Haodong Duan
Researcher · Multimodal Learning & Evaluation
ByteDance Seed Singapore
欢迎围绕多模态学习、大模型评测与视频理解开展学术合作，也欢迎对开源评测工具和基准感兴趣的研究者与工程师联系我。
I am open to academic collaborations on multimodal learning, LLM/LMM evaluation, and video understanding. Please feel free to reach out by email.
Research Interests
My research focuses on multimodal learning, LLM/LMM evaluation, and video understanding. I build open-source evaluation infrastructure and benchmarks that make model capabilities easier to measure, compare, and reproduce.
Blog
Latest News
All News →-
Three papers are listed in the NeurIPS 2026 program: Castle-in-the-Air, RISE-Video, and Training Long-Context Vision-Language Models Effectively.
-
LEGO-Puzzles is published at ECCV 2026.
-
Seed2.1 is released, advancing multimodal reasoning and agentic productivity across the Seed model family.
-
Five papers are published at CVPR 2026, covering GUI-agent evaluation, spatial self-supervised RL, agentic reward modeling, visual reasoning, and poster intelligence.
-
Seed2.0 is officially launched, strengthening multimodal understanding, complex instruction execution, and real-world agent capabilities.
-
Three papers are published at ICLR 2026: MMSI-Bench, VisualPRM, and MM-HELIX.
-
Seed1.8 is officially released as a generalized agentic model with multimodal, search, coding, and GUI capabilities.
-
I joined ByteDance Seed in Singapore, after two years at Shanghai AI Laboratory working on OpenCompass and large-model evaluation.
-
RISEBench is accepted by NeurIPS 2025 Datasets & Benchmarks as an Oral presentation.
Selected Works
All Publications → First / Co-First Author Corresponding Author Project Lead
ICLR MMSI-Bench Spatial Reasoning · Multi-Image Reasoning · Multimodal Evaluation MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence
MMSI-Bench evaluates whether MLLMs can integrate evidence across multiple images to solve grounded, real-world spatial reasoning problems.
ICLR, 2026
ECCV LEGO-Puzzles Spatial Reasoning · Visual Reasoning · Multimodal Evaluation LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?
LEGO-Puzzles probes spatial understanding and multi-step planning through interpretable LEGO assembly tasks that remain difficult for frontier MLLMs.
ECCV, 2026
ICCV Visual-RFT Reinforcement Learning · Reward Modeling · Model Adaptation Visual-RFT: Visual Reinforcement Fine-Tuning
Visual-RFT brings reinforcement fine-tuning with verifiable rewards to visual perception tasks, improving data-efficient adaptation with limited examples.
IEEE/CVF International Conference on Computer Vision (ICCV), 2025
NeurIPS RISEBench Multimodal Generation · Visual Reasoning · Multimodal Evaluation Envisioning beyond the pixels: Benchmarking reasoning-informed visual editing
RISEBench measures whether image editors can follow reasoning-heavy temporal, causal, spatial, and logical instructions while preserving visual quality.
NeurIPS D&B Oral, 2025
ACL OmniAlign-V Model Alignment · Reward Modeling · Data-Centric Learning Omnialign-v: Towards enhanced alignment of mllms with human preference
OmniAlign-V improves multimodal assistants with diverse preference-alignment data and introduces MM-AlignBench for open-ended alignment evaluation.
ACL, 2025
ACL Redundancy Principle Benchmark Design · Model Efficiency · Multimodal Evaluation Redundancy Principles for MLLMs Benchmarks
This work quantifies dimension, instance, and cross-benchmark redundancy and turns the findings into practical principles for efficient MLLM evaluation.
ACL, 2025
ACL Condor Data-Centric Learning · Model Adaptation · Model Alignment Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement
Condor synthesizes and refines knowledge-intensive instruction data to strengthen language-model alignment without relying on a fixed seed dataset.
ACL, 2025
ECCV MMBench Multimodal Evaluation · Benchmark Design · Visual Reasoning MMBench: Is Your Multi-modal Model an All-around Player?
MMBench provides a bilingual, ability-structured benchmark and CircularEval protocol for robustly diagnosing the capabilities of multimodal models.
European Conference on Computer Vision (ECCV), 2024
NeurIPS MMStar Benchmark Design · Multimodal Evaluation · Visual Reasoning Are We on the Right Way for Evaluating Large Vision-Language Models?
MMStar curates genuinely vision-dependent questions to expose data leakage and inflated gains in large vision-language model evaluation.
Conference on Neural Information Processing Systems (NeurIPS), 2024
ACM MM VLMEvalKit Evaluation Infrastructure · Multimodal Evaluation VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
VLMEvalKit unifies model inference, benchmark access, and standardized scoring into a widely adopted toolkit for reproducible multimodal evaluation.
ACM International Conference on Multimedia (MM), 2024
NeurIPS ShareGPT4Video Video Understanding · Data-Centric Learning · Multimodal Generation ShareGPT4Video: Improving Video Understanding and Generation with Better Captions
ShareGPT4Video scales detailed video captioning to improve both video-language understanding and text-to-video generation.
Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track, 2024
EMNLP ProSA LLM Evaluation · Benchmark Design · Model Alignment ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs
ProSA measures prompt sensitivity at the instance level and analyzes why semantically equivalent instructions can produce unstable LLM performance.
Empirical Methods in Natural Language Processing (EMNLP), Findings, 2024
NeurIPS MMBench-Video Video Understanding · Long-Context Modeling · Multimodal Evaluation MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
MMBench-Video evaluates long-form, multi-shot video understanding with free-form questions and a fine-grained taxonomy of temporal capabilities.
Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track, 2024
ACL BotChat LLM Evaluation · Agentic Systems · Model Alignment BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues
BotChat evaluates multi-turn conversational ability through scalable bot-to-bot interaction and fine-grained dialogue-quality assessment.
Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
NeurIPS Prism Visual Reasoning · Multimodal Evaluation · Model Efficiency Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs
Prism decouples perception and reasoning to diagnose VLM capability bottlenecks and combine compact specialist components more effectively.
Conference on Neural Information Processing Systems (NeurIPS), 2024
ACL Ada-LEval Long-Context Modeling · LLM Evaluation · Benchmark Design Ada-LEval: Evaluating Long-Context LLMs with Length-Adaptable Benchmarks
Ada-LEval adapts task length to model context windows, enabling controlled evaluation of retrieval, ordering, and full-text comprehension at scale.
Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
NeurIPS JourneyDB Image Understanding · Multimodal Generation · Multimodal Evaluation JourneyDB: A Benchmark for Generative Image Understanding
JourneyDB pairs millions of generated images with prompts and annotations to benchmark captioning, prompt inversion, retrieval, and visual question answering.
Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track, 2023
ICCV SkeleTR Skeleton Recognition · Action Recognition · Human Pose SkeleTR: Towards Skeleton-based Action Recognition in the Wild
SkeleTR models both intra-person motion and inter-person relationships for skeleton action recognition in crowded, unconstrained scenes.
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
CVPR PoseC3D Skeleton Recognition · Action Recognition · Human Pose Revisiting Skeleton-based Action Recognition
PoseC3D represents skeletons as 3D heatmap volumes, yielding a simple, robust, and scalable alternative to graph-based action recognition.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
ACM MM PYSKL Evaluation Infrastructure · Skeleton Recognition · Action Recognition PYSKL: Towards Good Practices for Skeleton Action Recognition
PYSKL consolidates diverse skeleton-action algorithms, training practices, and benchmarks into one reproducible research toolbox.
ACM International Conference on Multimedia (MM), 2022
arXiv DG-STGCN Skeleton Recognition · Action Recognition · Human Pose Dg-stgcn: Dynamic spatial-temporal modeling for skeleton-based action recognition
DG-STGCN learns dynamic spatial graphs and multi-scale temporal structure instead of relying on a fixed skeleton topology.
arXiv preprint arXiv:2210.05895, 2022
CVPR TransRank Self-Supervised Learning · Video Understanding · Action Recognition TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition
TransRank reframes transformation recognition as relative ranking to learn stronger self-supervised video representations from temporal and spatial transformations.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
ECCV OmniSource Data-Centric Learning · Video Understanding · Action Recognition Omni-sourced Webly-supervised Learning for Video Recognition
OmniSource unifies images, short clips, and untrimmed web videos to make large-scale web supervision more data-efficient for video recognition.
European Conference on Computer Vision (ECCV), 2020
ICCV TRB Human Pose · Image Understanding TRB: A Novel Triplet Representation for Understanding 2D Human Body
TRB combines skeleton and contour keypoints to represent human pose and shape compactly for estimation, editing, and conditional generation.
IEEE/CVF International Conference on Computer Vision (ICCV), 2019
Open-Source Projects
VLMEvalKit
PythonAn all-in-one evaluation toolkit for large vision-language and multimodal models. It connects model inference, benchmark execution, scoring, and result comparison in one reproducible workflow used by both researchers and practitioners.
Multimodal EvaluationEvaluation Infrastructure
A comprehensive platform for evaluating large language and multimodal models. Its modular configs, task scheduling, evaluators, and reporting tools make broad and reproducible model comparisons easier to run and extend.
LLM EvaluationBenchmarking Platform
A unified toolbox for skeleton-based action recognition, with strong baselines spanning GCN- and CNN-based methods. It packages training recipes, pretrained models, and benchmark results so new methods can be compared on consistent footing.
Skeleton ActionVideo Recognition
OpenMMLab's modular video-understanding toolbox for action recognition, localization, skeleton-based recognition, and related tasks. It offers reusable components, extensive model coverage, and standardized training and inference pipelines.
Video UnderstandingOpenMMLab