Contact
Email: ruanjie@stu.pku.edu.cn jieruan75[at]gmail[dot]com
Twitter: @JieRuan75
Wechat: 1151334747
About Me
Hi, my name is Jie Ruan. Currently, I am a first-year PhD student at University of Michigan. I am fortunate to have Prof. Lu Wang as my advisor and I am a member of LAUNCH Lab.
I received my master degree from Peking University, advised by Prof. Yuesheng Zhu and Prof. Guibo Luo, and had the privilege of working closely with Prof. Xiaojun Wan.
My research interest broadly lies in AI and its constructive impacts on humanity. I aspire to deliver Trustworthy AI, including areas such as reliable evaluation, alignment, and scalable oversight. And I am dedicated to unleashing the full potential of imperfect AI in practical applications, especially in fields like psychology, medicine, education, social good, and science.
Feel free to reach out to me if you’d like to discuss research or explore potential collaboration!
I am seeking self-motivated undergraduate or master’s students to work on LLM evaluation. If you’re interested, I’d be happy to hear from you!
Selected Publications
Defining and Detecting Vulnerability in Human Evaluation Guidelines: A Preliminary Study Towards Reliable NLG Evaluation Paper Code
Jie Ruan, Wenqing Wang, Xiaojun Wan
🏆 NAACL 2024, Outstanding Paper Award
Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling. Paper Code
Jie Ruan, Xiao Pu, Mingqi Gao, Xiaojun Wan and Yuesheng Zhu
AAAI 2024
Benchmarking knowledge boundary for large language model: A different perspective on model evaluation. Paper Code
Xunjian Yin, Xu Zhang, Jie Ruan, Xiaojun Wan
ACL 2024
Describe Images in a Boring Way: Towards Cross-Modal Sarcasm Generation. Paper Code
Jie Ruan, Yue Wu, Xiaojun Wan and Yuesheng Zhu
WACV 2024
Human-like Summarization Evaluation with ChatGPT. Paper
Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, Xiaojun Wan
ReproHum# 0087-01: A Reproduction Study of the Human Evaluation of the Coverage of Fact Checking Explanations. paper
Mingqi Gao, Jie Ruan, Xiaojun Wan
Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024.
A Reproduction Study of the Human Evaluation of Role-Oriented Dialogue Summarization Models. Paper
Mingqi Gao, Jie Ruan, Xiaojun Wan
HumEval at RANLP 2023. The 3rd Workshop on Human Evaluation of NLP Systems (HumEval’23)
Selected Awards
NAACL 2024 Outstanding Paper Award (2024)
Peking University Outstanding Graduate (2024)
Peking University Outstanding Graduation Thesis (2024)
Academic Excellence Award of Peking University (2023)
Good Youth of Progressive and Kind in Guangdong Province (2022)
Outstanding Undergraduate Thesis (2021)
Outstanding Undergraduate Student (2021)
National Scholarship (2021)
Provincial Mathematical Modeling Competition - First Prize (2019)
City Award for Excellence in Computer Science Research - First Prize (2019)
Academic Service
Reviewer: ACL ARR, ICML, ICMR, IEEE Computational Intelligence Magazine
Program Committee: ICMR