⚠️ Note: This homepage was last updated in June 2024 and will remain on pause until I produce work of significant quality and impact. As a result, regular maintenance of the site is currently unnecessary.

I am a senior undergraduate student at Tianjin University.
My research interests lie in Large Language Model and Multimodal Learning.

Selected Papers

• (EMNLP 2024) Automated Evaluation Using Self-Adaptive Rubrics
  Zhiyuan Fan*, Weinong Wang, Xing Wu, Debing Zhang
  Under Review

• (NeurIPS 2024) Calibrated Self-Rewarding Vision Language Models
  Yiyang Zhou, Zhiyuan Fan*, … , Huaxiu Yao
  These authors contributed equally.
  Under Review

• (ACL 2024) Exploring the Potential of Dense Information in Multimodal Alignment
  Zhiyuan Fan*, Zhihong Chen, Benyou Wang
  Under Review

• (EMNLP 2023) Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
  Zhiyuan Fan*, Shizhu He
  [Arxiv]

• (Electronics) Cuckoo Matrix: A High Efficient and Accurate Graph Stream Summarization on Limited Memory
  Zhuo Li, Zhuoran Li, Zhiyuan Fan*, Jianli Zhao, Siming Zeng, Peng Luo and Kaihua Liu
  [MDPI]

Selected Engineering Projects

• Reimplemented the tensor parallel strategy from the ground up using PyTorch and NCCL.
• Successfully trained models exceeding 100 billion parameters from scratch.

Selected Experiences

• University of North Carolina at Chapel Hill
  Intern research (advisor: Asst. Prof. Huaxiu Yao)
  February 2024 - Present

• Institute of Neuroscience, Chinese Academy of Sciences
  Intern research (advisor: Prof. Xue Li)
  April 2023 - June 2023

• Baidu & Jina AI
  Intern research (advisor: CTO, Nan Wang)
  July 2022 - Jan 2023

• Institute of Automation, Chinese Academy of Sciences
  Intern research (advisor: Assoc. Researcher Shizhu He)
  April 2022 - January 2024

Selected Open Source Projects

• PaddleNLP (GitHub, 11.7k Stars)
  Contributions: Submitted two PRs (7k+ lines):
  • Implemented DPR training on a single-GPU with a batch size of 512 using gradient cache.
  • Developed a from-scratch DeBERTa pre-trained language model implementation.
• LLM Cookbook (GitHub, 10.2k Stars)
  Contributions: Authored tutorials on Retrieval-Augmented Generation (RAG), agent-based systems, and techniques for processing unstructured data.

Selected Book

• Statistical Learning Method Solutions Manual (GitHub, 1.4k Stars, Online)
  • First Author of Chapter 27: Pre-trained Language Model
  • Reviewer for Chapters 16, 19, 20, and 21.

Selected Awards

• Champion
  Baidu PaddlePaddle AI Hackathon: Pretrained Language Model.
  2023

• Gold Award
  China International College Students ‘Internet+’ Innovation and Entrepreneurship Competition.
  2023

• Silver Award
  Huawei Ascend Operator Development Competition, solo.
  2021