Xunlun/DisasterM3

--- license: cc-by-nc-sa-4.0 task_categories: - visual-question-answering - image-segmentation - image-to-text - image-classification language: - en tags: - vision-language - remote-sensing - disaster - multi-task pretty_name: >- A multi-hazard, multi-sensor, and multi-task vision-language dataset for global-scale disaster assessment and response. size_categories: - 1B<n<10B --- <h2 align="center"> DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response </h2> <h5 align="center"><a href="https://junjue-wang.github.io/homepage/">Junjue Wang*</a>, <a href="https://weihaoxuan.com">Weihao Xuan*</a>, Heli Qi, Zhihao Liu, Kunyi Liu, Yuhan Wu, <a href="https://chrx97.com/"> Hongruixuan Chen</a>, <a href="https://jtrneo.github.io/"> Jian Song</a></h5> <h5 align="center"> Junshi Xia, <a href="https://zhuozheng.top/">Zhuo Zheng</a>, <a href="https://naotoyokoya.com/">Naoto Yokoya†</a></h5> <h5 align="center"> * Equal Contributions † Corresponding Author</h5> `Paper`: https://arxiv.org/abs/2505.21089 `Code`: https://github.com/Junjue-Wang/DisasterM3 <div align="center"> <img src="https://github.com/Junjue-Wang/resources/blob/main/DisasterM3/task_taxonomy.png?raw=true"> </div> ## Highlights DisasterM3 includes 26,988 bi-temporal satellite images and 123k instruction pairs across 5 continents, with three characteristics: 1. Multi-hazard: 36 historical disaster events with significant impacts, which are categorized into 10 common natural and man-made disasters 2. Multi-sensor: Extreme weather during disasters often hinders optical sensor imaging, making it necessary to combine Synthetic Aperture Radar (SAR) imagery for post-disaster scenes 3. Multi-task: 9 disaster-related visual perception and reasoning tasks, harnessing the full potential of VLM's reasoning ability ## News - 2025/10/23, We released the DisasterM3 instruct set. - 2025/10/17, We released the benchmark set of DisasterM3. - 2025/09/22, We are preparing the dataset and code. - 2025/09/22, Our paper got accepted by NeurIPS 2025. ## Benchmark Please run this code for benchmarking the DisasterM3 dataset. Two examples: Qwen2.5 VL: ``` python disaster_m3/pyscripts/run_vllm.py --model_id Qwen/Qwen2.5-VL-7B-Instruct --subset bearing_body ``` InternVL3: ``` python disaster_m3/pyscripts/run_vllm.py --model_id OpenGVLab/InternVL3-78B --subset report ``` ## Citation If you use DisasterM3 in your research, please cite our following papers. ```text @article{wang2025disasterm3, title={DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response}, author={Wang, Junjue and Xuan, Weihao and Qi, Heli and Liu, Zhihao and Liu, Kunyi and Wu, Yuhan and Chen, Hongruixuan and Song, Jian and Xia, Junshi and Zheng, Zhuo and Yokoya, Naoto}, booktitle={Proceedings of the Neural Information Processing Systems}, year={2025} } ``` ## Acknowledgments This dataset builds upon the following excellent open datasets: - **xBD dataset** by Ritwik Gupta - [Paper](https://openaccess.thecvf.com/content_CVPRW_2019/html/cv4gc/Gupta_Creating_xBD_A_Dataset_for_Assessing_Building_Damage_from_Satellite_CVPRW_2019_paper.html) - [Dataset](https://xview2.org/dataset) - License: [CC BY-NC-SA 4.0] - **BRIGHT dataset** by Hongruixuan Chen - [Repository](https://github.com/ChenHongruixuan/BRIGHT) - License: [CC BY-NC 4.0] ## License All images and their associated annotations in DisasterM3 can be used for academic purposes only, <font color="red"><b> but any commercial use is prohibited.</b></font> <a rel="license" href="https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en"> <img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> ## Star History [](https://www.star-history.com/#Junjue-Wang/DisasterM3&Date)许可证：CC BY-NC-SA 4.0 任务类别： - 视觉问答 - 图像分割 - 图像到文本 - 图像分类 语言： - 英语 标签： - 视觉语言 - 遥感 - 灾害 - 多任务 美观名称：面向全球灾害评估与响应的多灾害、多传感器、多任务视觉语言数据集 规模类别：10亿 < 数据集规模 < 100亿 <h2 align="center">DisasterM3：面向灾害损毁评估与响应的遥感视觉语言数据集</h2> <h5 align="center"><a href="https://junjue-wang.github.io/homepage/">王俊杰*</a>, <a href="https://weihaoxuan.com">宣炜豪*</a>, 齐赫利, 刘智浩, 刘坤毅, 吴雨涵, <a href="https://chrx97.com/">陈鸿睿轩</a>, <a href="https://jtrneo.github.io/">宋健</a></h5> <h5 align="center">夏俊诗, <a href="https://zhuozheng.top/">郑卓</a>, <a href="https://naotoyokoya.com/">横田直人†</a></h5> <h5 align="center">* 共同第一作者 † 通讯作者</h5> `论文`: https://arxiv.org/abs/2505.21089 `代码仓库`: https://github.com/Junjue-Wang/DisasterM3 <div align="center"> <img src="https://github.com/Junjue-Wang/resources/blob/main/DisasterM3/task_taxonomy.png?raw=true"> </div> ## 核心亮点 DisasterM3 覆盖全球5大洲，包含26988对双时相卫星影像与12.3万条指令样本对，具备三大特性： 1. 多灾害属性：收录36起影响深远的历史灾害事件，涵盖10类常见自然与人为灾害 2. 多传感器适配：灾害期间极端天气常会干扰光学传感器成像，因此本数据集融合合成孔径雷达（Synthetic Aperture Radar, SAR）影像以覆盖灾后场景 3. 多任务覆盖：包含9类与灾害相关的视觉感知与推理任务，充分挖掘视觉语言模型（Vision-Language Model, VLM）的推理潜力 ## 最新动态 - 2025/10/23：发布DisasterM3指令集 - 2025/10/17：发布DisasterM3基准测试集 - 2025/09/22：数据集与代码筹备中 - 2025/09/22：论文被神经信息处理系统大会（NeurIPS 2025）收录 ## 基准测试 可通过如下代码运行DisasterM3数据集的基准测试。示例如下： Qwen2.5 VL 模型： python disaster_m3/pyscripts/run_vllm.py --model_id Qwen/Qwen2.5-VL-7B-Instruct --subset bearing_body InternVL3 模型： python disaster_m3/pyscripts/run_vllm.py --model_id OpenGVLab/InternVL3-78B --subset report ## 引用方式 若您在研究中使用DisasterM3数据集，请引用如下论文： text @article{wang2025disasterm3, title={DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response}, author={Wang, Junjue and Xuan, Weihao and Qi, Heli and Liu, Zhihao and Liu, Kunyi and Wu, Yuhan and Chen, Hongruixuan and Song, Jian and Xia, Junshi and Zheng, Zhuo and Yokoya, Naoto}, booktitle={Proceedings of the Neural Information Processing Systems}, year={2025} } ## 致谢 本数据集基于以下优秀开源数据集构建： - **xBD数据集**：由Ritwik Gupta团队开发 - [论文链接](https://openaccess.thecvf.com/content_CVPRW_2019/html/cv4gc/Gupta_Creating_xBD_A_Dataset_for_Assessing_Building_Damage_from_Satellite_CVPRW_2019_paper.html) - [数据集链接](https://xview2.org/dataset) - 许可证：CC BY-NC-SA 4.0 - **BRIGHT数据集**：由陈鸿睿轩团队开发 - [代码仓库链接](https://github.com/ChenHongruixuan/BRIGHT) - 许可证：CC BY-NC 4.0 ## 许可证声明 DisasterM3 所有影像及配套标注仅可用于学术研究，**严禁任何商业用途**。 <a rel="license" href="https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en"> <img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> ## 星标历史 [](https://www.star-history.com/#Junjue-Wang/DisasterM3&Date)