Ling200424/DisasterM3

--- license: cc-by-nc-sa-4.0 task_categories: - visual-question-answering - image-segmentation - image-to-text - image-classification language: - en tags: - vision-language - remote-sensing - disaster - multi-task pretty_name: >- A multi-hazard, multi-sensor, and multi-task vision-language dataset for global-scale disaster assessment and response. size_categories: - 1B<n<10B --- <h2 align="center"> DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response </h2> <h5 align="center"><a href="https://junjue-wang.github.io/homepage/">Junjue Wang*</a>, <a href="https://weihaoxuan.com">Weihao Xuan*</a>, Heli Qi, Zhihao Liu, Kunyi Liu, Yuhan Wu, <a href="https://chrx97.com/"> Hongruixuan Chen</a>, <a href="https://jtrneo.github.io/"> Jian Song</a></h5> <h5 align="center"> Junshi Xia, <a href="https://zhuozheng.top/">Zhuo Zheng</a>, <a href="https://naotoyokoya.com/">Naoto Yokoya†</a></h5> <h5 align="center"> * Equal Contributions † Corresponding Author</h5> `Paper`: https://arxiv.org/abs/2505.21089 `Code`: https://github.com/Junjue-Wang/DisasterM3 <div align="center"> <img src="https://github.com/Junjue-Wang/resources/blob/main/DisasterM3/task_taxonomy.png?raw=true"> </div> ## Highlights DisasterM3 includes 26,988 bi-temporal satellite images and 123k instruction pairs across 5 continents, with three characteristics: 1. Multi-hazard: 36 historical disaster events with significant impacts, which are categorized into 10 common natural and man-made disasters 2. Multi-sensor: Extreme weather during disasters often hinders optical sensor imaging, making it necessary to combine Synthetic Aperture Radar (SAR) imagery for post-disaster scenes 3. Multi-task: 9 disaster-related visual perception and reasoning tasks, harnessing the full potential of VLM's reasoning ability ## News - 2025/10/23, We released the DisasterM3 instruct set. - 2025/10/17, We released the benchmark set of DisasterM3. - 2025/09/22, We are preparing the dataset and code. - 2025/09/22, Our paper got accepted by NeurIPS 2025. ## Benchmark Please run this code for benchmarking the DisasterM3 dataset. Two examples: Qwen2.5 VL: ``` python disaster_m3/pyscripts/run_vllm.py --model_id Qwen/Qwen2.5-VL-7B-Instruct --subset bearing_body ``` InternVL3: ``` python disaster_m3/pyscripts/run_vllm.py --model_id OpenGVLab/InternVL3-78B --subset report ``` ## Citation If you use DisasterM3 in your research, please cite our following papers. ```text @article{wang2025disasterm3, title={DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response}, author={Wang, Junjue and Xuan, Weihao and Qi, Heli and Liu, Zhihao and Liu, Kunyi and Wu, Yuhan and Chen, Hongruixuan and Song, Jian and Xia, Junshi and Zheng, Zhuo and Yokoya, Naoto}, booktitle={Proceedings of the Neural Information Processing Systems}, year={2025} } ``` ## Acknowledgments This dataset builds upon the following excellent open datasets: - **xBD dataset** by Ritwik Gupta - [Paper](https://openaccess.thecvf.com/content_CVPRW_2019/html/cv4gc/Gupta_Creating_xBD_A_Dataset_for_Assessing_Building_Damage_from_Satellite_CVPRW_2019_paper.html) - [Dataset](https://xview2.org/dataset) - License: [CC BY-NC-SA 4.0] - **BRIGHT dataset** by Hongruixuan Chen - [Repository](https://github.com/ChenHongruixuan/BRIGHT) - License: [CC BY-NC 4.0] ## License All images and their associated annotations in DisasterM3 can be used for academic purposes only, <font color="red"><b> but any commercial use is prohibited.</b></font> <a rel="license" href="https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en"> <img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> ## Star History [](https://www.star-history.com/#Junjue-Wang/DisasterM3&Date)--- 许可证：CC BY-NC-SA 4.0（知识共享署名-非商业性使用-相同方式共享4.0国际许可协议） 任务类别： - 视觉问答（Visual Question Answering, VQA） - 图像分割 - 图像到文本 - 图像分类 语言： - 英语 标签： - 视觉语言（Vision-Language） - 遥感 - 灾害 - 多任务 数据集展示名称：用于全球尺度灾害评估与响应的多灾害、多传感器、多任务视觉语言数据集 样本规模区间：10亿 < 样本量 < 100亿 --- <h2 align="center">DisasterM3：面向灾害损毁评估与响应的遥感视觉语言数据集</h2> <h5 align="center"><a href="https://junjue-wang.github.io/homepage/">王俊杰*</a>、<a href="https://weihaoxuan.com">宣伟豪*</a>、齐合力、刘智昊、刘坤毅、吴雨涵、<a href="https://chrx97.com/">陈鸿睿轩</a>、<a href="https://jtrneo.github.io/">宋健</a></h5> <h5 align="center">夏俊诗、<a href="https://zhuozheng.top/">郑卓</a>、<a href="https://naotoyokoya.com/">横田直人†</a></h5> <h5 align="center">* 共同第一作者 † 通讯作者</h5> `论文`：https://arxiv.org/abs/2505.21089 `代码`：https://github.com/Junjue-Wang/DisasterM3 <div align="center"> <img src="https://github.com/Junjue-Wang/resources/blob/main/DisasterM3/task_taxonomy.png?raw=true"> </div> ## 核心亮点 DisasterM3 涵盖全球5大洲的26988对双时相卫星影像与12.3万条指令样本，具备三大核心特性： 1. 多灾害覆盖：收录36起具有重大影响的历史灾害事件，涵盖10类常见自然与人为灾害 2. 多传感器融合：灾害期间极端天气常干扰光学传感器成像，因此需结合合成孔径雷达（Synthetic Aperture Radar, SAR）影像用于灾后场景分析 3. 多任务适配：包含9项灾害相关的视觉感知与推理任务，可充分挖掘视觉语言模型（Vision-Language Model, VLM）的推理潜能 ## 动态 - 2025/10/23：正式发布DisasterM3指令集 - 2025/10/17：正式发布DisasterM3基准测试集 - 2025/09/22：数据集与代码筹备中 - 2025/09/22：论文被NeurIPS 2025收录 ## 基准测试 如需对DisasterM3数据集开展基准测试，请运行以下代码。 两个示例如下： Qwen2.5 VL: python disaster_m3/pyscripts/run_vllm.py --model_id Qwen/Qwen2.5-VL-7B-Instruct --subset bearing_body InternVL3: python disaster_m3/pyscripts/run_vllm.py --model_id OpenGVLab/InternVL3-78B --subset report ## 引用 若您在研究工作中使用DisasterM3数据集，请引用如下论文： text @article{wang2025disasterm3, title={DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response}, author={Wang, Junjue and Xuan, Weihao and Qi, Heli and Liu, Zhihao and Liu, Kunyi and Wu, Yuhan and Chen, Hongruixuan and Song, Jian and Xia, Junshi and Zheng, Zhuo and Yokoya, Naoto}, booktitle={Proceedings of the Neural Information Processing Systems}, year={2025} } ## 致谢 本数据集依托以下优质开源数据集构建： - **xBD数据集** 由Ritwik Gupta发布 - [论文](https://openaccess.thecvf.com/content_CVPRW_2019/html/cv4gc/Gupta_Creating_xBD_A_Dataset_for_Assessing_Building_Damage_from_Satellite_CVPRW_2019_paper.html) - [数据集](https://xview2.org/dataset) - 许可协议：[CC BY-NC-SA 4.0] - **BRIGHT数据集** 由陈鸿睿轩发布 - [代码仓库](https://github.com/ChenHongruixuan/BRIGHT) - 许可协议：[CC BY-NC 4.0] ## 许可协议 DisasterM3数据集内的所有影像及其关联标注仅可用于学术研究场景，<font color="red"><b> 任何商业使用均被严格禁止。</b></font> <a rel="license" href="https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en"> <img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> ## 星标历史 [](https://www.star-history.com/#Junjue-Wang/DisasterM3&Date)