About Me

I am a senior undergraduate student at Tianjin University.
My research interests lie in Large Language Model and Multimodal Learning.

Selected Publications

  • Automated Evaluation Using Self-Adaptive Rubrics
    Zhiyuan Fan*, Weinong Wang, Xing Wu, Debing Zhang
    Under Review

  • Calibrated Self-Rewarding Vision Language Models
    Yiyang Zhou, Zhiyuan Fan*, … , Huaxiu Yao
    These authors contributed equally.
    Under Review

  • (ACL 2024) Exploring the Potential of Dense Information in Multimodal Alignment
    Zhiyuan Fan*, Zhihong Chen, Benyou Wang
    Under Review

  • (EMNLP 2023) Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
    Zhiyuan Fan*, Shizhu He

  • (Electronics) Cuckoo Matrix: A High Efficient and Accurate Graph Stream Summarization on Limited Memory
    Zhuo Li, Zhuoran Li, Zhiyuan Fan*, Jianli Zhao, Siming Zeng, Peng Luo and Kaihua Liu

Selected Engineering Projects

  • Reimplemented the tensor parallel strategy from the ground up using PyTorch and NCCL.
  • Successfully trained models exceeding 100 billion parameters from scratch.

Selected Experiences

  • Institute of Neuroscience, Chinese Academy of Sciences
    Intern research (advisor: Prof. Xue Li)
    April 2023 - June 2023

  • Baidu & Jina AI
    Intern research (advisor: CTO, Nan Wang)
    July 2022 - Jan 2023

  • Institute of Automation, Chinese Academy of Sciences
    Intern research (advisor: Assoc. Researcher Shizhu He)
    April 2022 - Present

Selected Open Source Projects

  • PaddleNLP (GitHub, 11.7k Stars)
    Contributions: Submitted two PRs (7k+ lines):
    • Implemented DPR training on a single-GPU with a batch size of 512 using gradient cache.
    • Developed a from-scratch DeBERTa pre-trained language model implementation.
  • LLM Cookbook (GitHub, 10.2k Stars)
    Contributions: Authored tutorials on Retrieval-Augmented Generation (RAG), agent-based systems, and techniques for processing unstructured data.

Selected Book

  • Statistical Learning Method Solutions Manual (GitHub, 1.4k Stars, Online)
    • First Author of Chapter 27: Pre-trained Language Model
    • Reviewer for Chapters 16, 19, 20, and 21.

Selected Awards

  • Champion
    Baidu PaddlePaddle AI Hackathon: Pretrained Language Model.

  • Gold Award
    China International College Students ‘Internet+’ Innovation and Entrepreneurship Competition.

  • Silver Award
    Huawei Ascend Operator Development Competition, solo.