Haotian Liu


I am a member of technical staff at xAI. I obtained my Ph.D. from University of Wisconsin-Madison in May 2024, under the supervision of Prof. Yong Jae Lee. During my Ph.D., I’ve been fortunate to work with and have long-term collaboration with Chunyuan Li at Microsoft Research. Before this, I obtained my bachler’s degree (with honor) at Zhejiang University, where I worked with Prof. Xiaogang Jin and Prof. Fei Wu.

I am generally interested in computer vision and machine learning. My recent focus is on building steerable large models. The first babies include LLaVA and Grok-1.5V.

selected publications

  1. Blog
    LLaVA-NeXT: Improved reasoning, OCR, and world knowledge
    Jan 2024
  2. llava_v15.jpg
    Improved Baselines with Visual Instruction Tuning (LLaVA-1.5)
    Haotian LiuChunyuan LiYuheng Li, and Yong Jae Lee
    CVPR, 2024
  3. llava2023.jpg
    Visual Instruction Tuning (LLaVA)
    Haotian Liu*Chunyuan Li*Qingyang Wu, and Yong Jae Lee
    NeurIPS, 2023 (Oral, top 0.5%)
  4. react2023.jpg
    Learning Customized Visual Models with Retrieval-Augmented Knowledge
    Haotian Liu, Kilho Son , Jianwei Yang , Ce Liu , Jianfeng GaoYong Jae Lee*, and Chunyuan Li*
    CVPR, 2023 (Highlight, top 2.5%)
  5. gligen2023.jpg
    GLIGEN: Open-Set Grounded Text-to-Image Generation
    Yuheng LiHaotian LiuQingyang Wu, Fangzhou Mu , Jianwei YangJianfeng GaoChunyuan Li, and Yong Jae Lee
    CVPR, 2023
  6. elevater2022.jpg
    ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
    Chunyuan Li*Haotian Liu* , Liunian Li , Pengchuan Zhang , Jyoti Aneja , Jianwei Yang, Ping Jin , Houdong Hu , Zicheng Liu , Yong Jae Lee, and Jianfeng Gao
    NeurIPS, Datasets and Benchmarks Track, 2022
  7. maskpoint2022.png
    Masked Discrimination for Self-Supervised Learning on Point Clouds
    Haotian LiuMu Cai, and Yong Jae Lee
    ECCV, 2022
  8. icra2021.jpg
    YolactEdge: Real-time Instance Segmentation on the Edge
    Haotian Liu*, Rafael A Rivera Soto* , Fanyi Xiao, and Yong Jae Lee
    ICRA, 2021