MMDU 超长多图多轮对话理解数据集

MMDU (Multi-Turn Multi-Image Dialog Understanding) 是超长多图多轮对话理解数据集，由武汉大学、上海人工智能实验室、香港中文大学和摩尔线程于 2024 年联合推出。研究团队在论文「MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs」中提出了全新多图多轮评测基准 MMDU 及大规模指令微调数据集 MMDU-45k，旨在评估和提升 LVLMs 在多轮及多图像对话中的性能。MMDU (Multi-Turn Multi-Image Dialog Understanding) is an ultra-long multi-turn and multi-image dialog understanding dataset, jointly launched in 2024 by Wuhan University, Shanghai AI Laboratory, The Chinese University of Hong Kong, and Moore Threads. The research team proposed a novel multi-turn and multi-image evaluation benchmark MMDU and a large-scale instruction-tuning dataset MMDU-45k in their paper titled "MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs", aiming to evaluate and enhance the performance of LVLMs in multi-turn and multi-image dialog scenarios.

搜集汇总

背景与挑战

背景概述

MMDU是一个超长多图多轮对话理解数据集，包含110个高质量的多图像多轮对话和45k条指令调优数据对话，旨在评估和提升大型视觉语言模型在多轮及多图像对话中的性能。其特点包括多轮对话与多图像输入、长上下文处理能力以及开放式评估方法。