QCRI/DisasterVQA

--- language: - en license: cc-by-sa-4.0 task_categories: - visual-question-answering - image-to-text - question-answering task_ids: - visual-question-answering modality: - image - text tags: - DisasterVQA - disaster-response - humanitarian - crisis-informatics - VQA - VLM - vision-language-models - damage-assessment - situational-awareness source_datasets: - Incidents1M - CrisisMMD - MEDIC size_categories: - 1K<n<10K pretty_name: DisasterVQA doi: 10.5281/zenodo.18365212 configs: - config_name: default data_files: - split: train path: DisasterVQA/** dataset_info: features: - name: question_id dtype: string - name: image_id dtype: string - name: image dtype: image - name: question dtype: string - name: question_type dtype: string - name: groundtruth_answer sequence: string - name: choices dtype: struct: - name: A dtype: string - name: B dtype: string - name: C dtype: string - name: D dtype: string - name: disaster_type dtype: string - name: dataset_source dtype: string - name: region dtype: string - name: crisis_info_type dtype: string - name: crisis_info_code dtype: string --- # DisasterVQA: A Visual Question Answering Benchmark Dataset for Disaster Scenes ## Dataset Summary DisasterVQA is a benchmark dataset for evaluating Vision-Language Models (VLMs) on disaster-response visual question answering. It contains **1,395 real-world disaster images** and **4,405 expert-curated question–answer pairs** covering floods, wildfires, and earthquakes. The dataset includes three question types: - **Binary** (Yes/No) - **Multiple-Choice** - **Open-Ended** Questions span situational awareness and operational decision-making tasks, grounded in humanitarian frameworks (FEMA ESF, OCHA MIRA). --- ## Dataset Structure ### Repository Layout ``` ├── README.md ├── disastervqa_annotations.jsonl ├── taxonomy.json └── DisasterVQA/ ├── Incidents1M/ │ └── ... ├── CrisisMMD/ │ └── ... └── MEDIC/ └── ... ``` ### Files | File | Description | |------|-------------| | `disastervqa_annotations.jsonl` | Benchmark annotations and metadata (question text, ground-truth answers, image paths, taxonomy labels) | | `taxonomy.json` | Final taxonomy definitions and references for each `crisis_info_code` | | `DisasterVQA/Incidents1M/` | Disaster images sourced from the Incidents1M dataset | | `DisasterVQA/CrisisMMD/` | Disaster images sourced from the CrisisMMD dataset | | `DisasterVQA/MEDIC/` | Disaster images sourced from the MEDIC dataset | --- ## License This dataset is released under the **Creative Commons Attribution Share Alike 4.0 International (CC BY-SA 4.0)** license. --- ## Citation If you use this dataset, please cite the accompanying paper: ```bibtex @inproceedings{disastervqa_icwsm2026, author = {Al-Mohannadi, Aisha and Firoz, Ayisha and Yang, Yin and Imran, Muhammad and Ofli, Ferda}, title = {DisasterVQA: A Visual Question Answering Benchmark Dataset for Disaster Scenes}, booktitle = {Proceedings of the International AAAI Conference on Web and Social Media (ICWSM)}, year = {2026}, address = {Los Angeles, California, USA}, url = {https://arxiv.org/abs/2601.13839} } ``` Paper: [arXiv:2601.13839](https://arxiv.org/abs/2601.13839) --- ## Links - 📦 Zenodo: [https://doi.org/10.5281/zenodo.18267769](https://doi.org/10.5281/zenodo.18267769) - 📄 Paper: [arXiv:2601.13839](https://arxiv.org/abs/2601.13839) - 🏛️ Conference: ICWSM 2026, Los Angeles, California, USADisasterVQA is a benchmark dataset for evaluating vision-language models on disaster response visual question answering (VQA) tasks. It comprises 1,395 real-world disaster images and 4,405 expert-annotated question-answer pairs, covering disaster types such as floods, wildfires, and earthquakes. The question types include binary (yes/no), multiple-choice, and open-ended questions, with content involving situational awareness and operational decision-making tasks, and is developed based on humanitarian frameworks such as FEMA ESF and OCHA MIRA.

搜集汇总

数据集介绍

构建方式

面向灾害危机信息学与视觉问答交叉领域，DisasterVQA的构建以真实灾害场景图像为基础，汇集Incidents1M、CrisisMMD与MEDIC等来源的灾害图像，经筛选保留1,395幅覆盖洪水、野火与地震的影像。围绕人道主义响应框架FEMA ESF与OCHA MIRA，领域专家设计并标注4,405组问答对，包含二元、多选与开放式三类问题，聚焦情境感知与操作决策。标注结果以JSONL格式组织，另设taxonomy.json界定危机信息代码，图像按原始来源分目录存放，形成可复现的基准资源。

特点

灾害场景下的多模态理解面临图像语义复杂、决策信息密集与标注稀缺等挑战，DisasterVQA由此提供兼具规模与专家知识的评测资源。其特点体现在多源真实图像与专家策展问答的耦合：数据覆盖洪水、野火、地震等灾害类型，问题涵盖二元、多选与开放式形式，并关联灾害类型、区域、来源数据集、危机信息类型与代码等元数据。字段设计包含图像、问题、选项与标准答案，可支撑视觉问答、图像到文本与问答任务，亦适配视觉语言模型的情境意识与损害评估研究。

使用方法

为评估视觉语言模型在灾害响应中的情境理解与辅助决策能力，DisasterVQA可采用标准化流程加载与评测。使用者可通过HuggingFace数据集接口加载default配置的train划分，或直接读取disastervqa_annotations.jsonl与对应图像目录。将图像、问题及可选选项输入模型后，依据二元、多选与开放式问题的标准答案分别计算准确率或语义匹配指标；taxonomy.json可用于解释危机信息代码。使用时应遵循CC BY-SA 4.0许可，并引用ICWSM 2026论文。

背景与挑战

背景概述

在危机信息学与人道主义响应智能化交汇的背景下，灾害场景的视觉问答正成为评估视觉语言模型情境理解能力的关键途径。DisasterVQA 由 Al-Mohannadi、Firoz、Yang、Imran 与 Ofli 等研究者构建，伴随 ICWSM 2026 论文发布，包含 1,395 幅真实灾害图像与 4,405 组专家策展问答对，覆盖洪水、野火与地震。其核心研究问题在于检验视觉语言模型能否依托 FEMA ESF、OCHA MIRA 等人道主义框架，完成灾害态势感知与操作决策相关的二元、多选及开放问答。该基准汇聚 Incidents1M、CrisisMMD 与 MEDIC 等来源，为危机信息学、损害评估和视觉语言模型鲁棒性研究提供了可复现的评测资源，并推动灾害响应场景中的多模态推理标准化。

当前挑战

面向灾害响应的视觉问答需应对高度异质、动态且标注稀缺的现实场景，这构成 DisasterVQA 所涉领域的首要难题。灾害图像常伴随低质量、遮挡、视角混乱与语义模糊，模型必须跨模态完成细粒度损害评估、情境推断和符合人道主义框架的操作决策，同时兼顾二元、多选与开放式问题的多样推理需求。构建过程中，研究者需整合 Incidents1M、CrisisMMD 与 MEDIC 等异构来源，协调许可、图像去重、灾害类型与区域平衡，并依赖专家策展确保问答对的事实一致性、taxonomy 映射准确性和危机信息编码可靠。多源数据分布差异与标注主观性亦对基准的公平性和泛化评估提出挑战，要求持续维护质量控制和可复现标准。

常用场景

经典使用场景

面向灾害响应与人道主义危机信息学的视觉问答研究，DisasterVQA 构筑了一个以真实灾后图像为载体的基准评测场域。其经典用法在于向视觉语言模型同时输入灾害图像与专家审订的问题，要求模型在是/否、多选与开放式三类题型中生成答案，从而检验模型对洪涝、野火与地震场景的细粒度理解。该数据集汇集 1,395 幅真实灾害图像与 4,405 组问答对，问题覆盖态势感知与操作决策，并锚定 FEMA ESF、OCHA MIRA 等人道主义框架，成为评估灾害场景多模态推理能力的基准资源。

解决学术问题

灾害场景的多模态理解长期受制于专家标注稀缺、灾种覆盖不均与评测协议缺位，DisasterVQA 的学术价值正体现于对此类瓶颈的系统性回应。该数据集以统一标注体系整合 Incidents1M、CrisisMMD 与 MEDIC 等来源，提供专家核验的答案、问题类型、灾种、区域及危机信息编码，使不同视觉语言模型能够在同一基准上比较灾害识别、损害评估与情境推演能力。其意义在于推动危机信息学从单模态分类转向可解释的视觉问答，提升模型评测的可复现性与人道主义任务适配度，为灾害响应中的可信人工智能研究奠定数据基础。

衍生相关工作

DisasterVQA 的学术谱系可追溯至 Incidents1M、CrisisMMD 与 MEDIC 等灾害图像与危机多媒体资源，其构建本身即是对既有经典工作的整合与基准化推进。该数据集直接衍生工作的核心是 ICWSM 2026 发表的 DisasterVQA 基准论文，并以此连接灾害视觉问答、视觉语言模型评测、损害评估与态势感知等研究脉络。其开放标注与统一协议为后续跨灾种泛化、专家知识注入、多模态危机信息抽取提供可复现基线，支撑危机信息学与多模态学习交叉地带相关经典工作群的发展。