MM-Hallu/HumbleBench

HumbleBench是一个多模态幻觉基准测试，用于评估多模态大语言模型（MLLMs）中的认知谦逊。它测试模型是否能识别提供的答案选项中没有一个正确的情况——这种行为反映了认知谦逊。数据集包含22,831个示例，涉及3,582张独特图像，分为训练集，任务类型包括对象、属性和关系。每个示例包括图像、问题ID、问题（包含选项A-E及以上都不是的多选题）、正确答案标签和任务类型字段。此外，数据集还提供子集：标准评估的HumbleBench、带高斯噪声图像的HumbleBench-GN，以及仅评估以上都不是的HumbleBench-E。

HumbleBench is a multimodal hallucination benchmark for evaluating epistemic humility in Multimodal Large Language Models (MLLMs). It tests whether models can recognize when none of the provided answer options are correct -- a behavior reflecting epistemic humility. The dataset contains 22,831 examples with 3,582 unique images, split into train, and includes task types such as Object, Attribute, and Relation. Each example features an image, question ID, question (multiple-choice with options A-E including None of the above), ground truth label, and type field. Additionally, subsets are provided: HumbleBench for standard evaluation, HumbleBench-GN with Gaussian noise images, and HumbleBench-E for None of the above only evaluation.