About Me
I am Jiatong Li. I am a master’s student majoring in data science at University of Science and Technology of China (USTC), supervised by Prof. Qi Liu. I got my bachelor’s degree at USTC in 2022. In 2021–2022, I used to do internships at Alibaba and iFLYTEK. I am currently doing research internship about large language model evaluation in Alibaba Cloud, Platform of Artificial Intelligence (PAI). I am seeking a PhD position for the Fall 2025 semester.
Research Interest
- Trustworthy AI
- ML robustness & safety
- Large language model evaluation
- Large language model in intelligent education
Research Goal
- Current - I am currently devoted to developing reliable and fair LLM evaluation methodologies in order to overcome challenges in existing LLM evaluation benchmarks, such as data contamination and biases in LLM outputs.
- Previous - During my master’s study, I was dedicated to proposing a brand-new paradigm for personalized learner modeling in order to implement a more flexible and applicable learner modeling system. This goal has been primarily achieved in our KDD’22 and WWW’24 paper. In addition, this research has opened several new research directions in personalized learner modeling.
Selected Publications
- Jiatong Li, Renjun Hu, Kunzhe Huang, Yan Zhuang, Qi Liu*, Mengxiao Zhu, Xing Shi, Wei Lin (2024). PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations. In Arxiv
- Jiatong Li, Qi Liu*, Fei Wang, Jiayu Liu, Zhenya Huang, Fangzhou Yao, Linbo Zhu, Yu Su (2024). Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm. In TheWebConf 2024 (pp. 3420-3431). ACM. [Arxiv][中文解读]
- Jiatong Li$^+$, Rui Li$^+$, Qi Liu (2023). Beyond Static Datasets: A Deep Interaction Approach to LLM Evaluation. In Arxiv.
- Jiatong Li, Fei Wang, Qi Liu, Mengxiao Zhu, Wei Huang, Zhenya Huang*, Enhong Chen, Yu Su, Shijin Wang (2022). HierCDF: A Bayesian Network-based Hierarchical Cognitive Diagnosis Framework. In SIGKDD 2022 (pp. 904–913). ACM.
News
2024.1.23. One research paper Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm is accepted by TheWebConf 2024.
2023.9.18 Our research about evaluating LLMs’ task-solving abilities using dynamic games is available at https://arxiv.org/abs/2309.04369.
2023.3.18. A research entitled A Bayesian hierarchical item response theory model for estimating attributes of regular exams and students’ knowledge levels is accepted to make an oral presentation at IMPS 2023 conference.
2022.5.20. One research paper HierCDF: A Bayesian Network-based Hierarchical Cognitive Diagnosis Framework is accepted by SIGKDD 2022.Click here for detail.