publications

2024

  1. Blog
    LLaVA-NeXT: Improved reasoning, OCR, and world knowledge
    Haotian Liu, Chunyuan Li , Yuheng Li , Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee
    Jan 2024
    Blog Code Demo
  2. llava_v15.jpg
    Improved Baselines with Visual Instruction Tuning (LLaVA-1.5)
    Haotian Liu, Chunyuan Li , Yuheng Li, and Yong Jae Lee
    CVPR, 2024
    arXiv HTML Code Demo
  3. vipllava2024.jpg
    Making large multimodal models understand arbitrary visual prompts
    Mu Cai , Haotian Liu, Siva Karthik Mustikovela , Gregory P Meyer , Yuning Chai , Dennis Park , and Yong Jae Lee
    CVPR, 2024
    arXiv HTML Code
  4. edit-one-for-all.jpg
    Edit One for All: Interactive Batch Image Editing
    Thao Nguyen, Utkarsh Ojha , Yuheng Li , Haotian Liu, and Yong Jae Lee
    CVPR, 2024
    arXiv HTML Code
  5. wacv2024.jpg
    Computer Vision on the Edge: Individual Cattle Identification in Real-Time With ReadMyCow System
    Moniek Smink , Haotian Liu, Dörte Döpfer , and Yong Jae Lee
    WACV, 2024
    PDF

2023

  1. arXiv
    LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
    Shilong Liu, Hao Cheng , Haotian Liu , Hao Zhang , Feng Li , Tianhe Ren , Xueyan Zou, Jianwei Yang, Hang Su , Jun Zhu , Lei Zhang , Jianfeng Gao, and Chunyuan Li
    arXiv, 2023
    arXiv
  2. arXiv
    Aligning Large Multimodal Models with Factually Augmented RLHF
    Zhiqing Sun, Sheng Shen, Shengcao Cao , Haotian Liu, Chunyuan Li , Yikang Shen , Chuang Gan , Liang-Yan Gui , Yu-Xiong Wang , Yiming Yang , Kurt Keutzer , and Trevor Darrell
    arXiv, 2023
    arXiv
  3. NeurIPS
    An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models
    Yadong Lu, Chunyuan Li , Haotian Liu, Jianwei Yang, Jianfeng Gao , and Yelong Shen
    NeurIPS, Workshop on Instruction Tuning and Instruction Following, 2023
    arXiv
  4. llavamed2023.jpg
    LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
    Chunyuan Li, Cliff Wong , Sheng Zhang , Naoto Usuyama , Haotian Liu, Jianwei Yang, Tristan Naumann , Hoifung Poon , and Jianfeng Gao
    NeurIPS, Datasets and Benchmarks Track, 2023 (Spotlight)
    arXiv
  5. llava2023.jpg
    Visual Instruction Tuning (LLaVA)
    Haotian Liu*, Chunyuan Li*, Qingyang Wu, and Yong Jae Lee
    NeurIPS, 2023 (Oral, top 0.5%)
    arXiv HTML Code Demo
  6. arXiv
    Benchmarking and Analyzing Generative Data for Visual Recognition
    Bo Li , Haotian Liu, Liangyu Chen , Yong Jae Lee, Chunyuan Li, and Ziwei Liu
    arXiv, 2023
    arXiv
  7. pacgen2023.jpg
    Generate Anything Anywhere in Any Scene
    Yuheng Li , Haotian Liu, Yangming Wen , and Yong Jae Lee
    arXiv, 2023
    arXiv
  8. react2023.jpg
    Learning Customized Visual Models with Retrieval-Augmented Knowledge
    Haotian Liu, Kilho Son , Jianwei Yang , Ce Liu , Jianfeng Gao, Yong Jae Lee*, and Chunyuan Li*
    CVPR, 2023 (Highlight, top 2.5%)
    arXiv HTML Code
  9. gligen2023.jpg
    GLIGEN: Open-Set Grounded Text-to-Image Generation
    Yuheng Li , Haotian Liu, Qingyang Wu, Fangzhou Mu , Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee
    CVPR, 2023
    arXiv HTML Code Demo

2022

  1. elevater2022.jpg
    ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
    Chunyuan Li* , Haotian Liu* , Liunian Li , Pengchuan Zhang , Jyoti Aneja , Jianwei Yang, Ping Jin , Houdong Hu , Zicheng Liu , Yong Jae Lee, and Jianfeng Gao
    NeurIPS, Datasets and Benchmarks Track, 2022
    arXiv HTML Video
  2. maskpoint2022.png
    Masked Discrimination for Self-Supervised Learning on Point Clouds
    Haotian Liu, Mu Cai, and Yong Jae Lee
    ECCV, 2022
    arXiv Video Code
  3. instedge2022.jpg
    End-to-End Instance Edge Detection
    Xueyan Zou , Haotian Liu*, and Yong Jae Lee
    arXiv, 2022
    arXiv

2021

  1. icra2021.jpg
    YolactEdge: Real-time Instance Segmentation on the Edge
    Haotian Liu*, Rafael A Rivera Soto* , Fanyi Xiao, and Yong Jae Lee
    ICRA, 2021
    arXiv Video Code

2019

  1. iccv2019.jpg
    Identity from here, Pose from there: Self-supervised Disentanglement and Generation of Objects using Unlabeled Videos
    Fanyi Xiao , Haotian Liu, and Yong Jae Lee
    ICCV, 2019
    PDF

2018

  1. JCLP
    Operation strategy of public building: Implications from trade-off between carbon emission and occupant satisfaction
    Yimeng Chen , Haotian Liu, and Lei Shi
    Journal of Cleaner Production, 2018