DeepMath-103K
收藏资源简介：
DeepMath-103K是一个精心策划的大规模数学数据集，旨在推动语言模型在数学推理方面的边界。其主要特点包括：1. 挑战性问题：专注于难度较高的数学问题（主要为5-9级），显著提高了复杂性；2. 广泛的主题多样性：涵盖代数、微积分、数论、几何、概率和离散数学等多个数学领域；3. 严格的净化：通过语义匹配对常见基准进行了细致的净化，最小化测试集泄漏并促进公平的模型评估；4. 丰富的数据格式：每个样本都包含问题、最终答案、难度、主题和三种不同的推理路径，以支持各种研究应用。
DeepMath-103K is a carefully curated large-scale mathematical dataset designed to push the boundaries of language models in mathematical reasoning. Its key features are as follows: 1. Challenging problems: Focusing on high-difficulty mathematical questions (primarily Levels 5-9), it significantly enhances overall complexity; 2. Wide thematic diversity: Covering multiple mathematical fields including algebra, calculus, number theory, geometry, probability theory, and discrete mathematics; 3. Strict data purification: It has undergone meticulous purification of common benchmarks via semantic matching, minimizing test set leakage and facilitating fair model evaluation; 4. Rich data format: Each sample contains the question, final answer, difficulty level, topic, and three distinct reasoning paths, supporting a wide range of research applications.
DeepMath-103K 数据集概述
数据集基本信息
- 名称: DeepMath-103K
- 发布日期: 2025年4月14日
- 数据地址: https://huggingface.co/datasets/zwhe99/DeepMath-103K
- 论文地址: https://arxiv.org/abs/2504.11456
- 代码地址: https://github.com/zwhe99/DeepMath
主要特点
- 高难度问题: 主要包含5-9级难度的数学问题，显著高于现有开源数据集的复杂度。
- 广泛主题覆盖: 涵盖代数、微积分、数论、几何、概率和离散数学等多个数学领域。
- 严格去污染处理: 通过语义匹配进行细致去污染处理，减少测试集泄露，确保公平评估。
- 丰富数据格式: 每个样本包含：
- 问题: 数学问题陈述
- 最终答案: 可验证的最终答案
- 难度: 数值化难度评分
- 主题: 层级分类
- 解决方案: 三种不同的推理路径
模型训练结果
7B模型表现
| 模型 | MATH 500 | AMC23 | Olympiad Bench | Miverva Math | AIME24 | AIME25 |
|---|---|---|---|---|---|---|
| Qwen2.5-7B-Base | 54.8 | 35.3 | 27.8 | 16.2 | 7.7 | 5.4 |
| Open-Reasoner-Zero-7B | 81.8 | 58.9 | 47.9 | 38.4 | 15.6 | 14.4 |
| Qwen-2.5-7B-SimpleRL-Zoo | 77.0 | 55.8 | 41.0 | 41.2 | 15.6 | 8.7 |
| DeepMath-Zero-7B | 85.5 | 64.7 | 51.0 | 45.3 | 20.4 | 17.5 |
1.5B模型表现
| 模型 | MATH 500 | AMC23 | Olympiad Bench | Miverva Math | AIME24 | AIME25 |
|---|---|---|---|---|---|---|
| R1-Distill-Qwen-1.5B | 84.7 | 72.0 | 53.1 | 36.6 | 29.4 | 24.8 |
| DeepScaleR-1.5B-Preview | 89.4 | 80.3 | 60.9 | 42.2 | 42.3 | 29.6 |
| Still-3-1.5B-Preview | 86.6 | 75.8 | 55.7 | 38.7 | 30.8 | 24.6 |
| DeepMath-1.5B | 89.0 | 81.6 | 60.1 | 40.6 | 39.8 | 30.8 |
快速开始
环境准备
shell git clone --recurse-submodules https://github.com/zwhe99/DeepMath.git && cd DeepMath conda create -y -n deepmath python=3.12.2 && conda activate deepmath pip3 install ray[default] pip3 install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124 pip3 install flash-attn==2.7.4.post1 --no-build-isolation pip3 install omegaconf==2.4.0.dev3 hydra-core==1.4.0.dev1 antlr4-python3-runtime==4.11.0 vllm==0.7.3 pip3 install math-verify[antlr4_11_0]==0.7.0 fire deepspeed tensorboardX prettytable datasets transformers==4.49.0 pip3 install -e verl
引用
bibtex @article{deepmath, title={DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning}, author={He, Zhiwei and Liang, Tian and Xu, Jiahao and Liu, Qiuzhi and Chen, Xingyu and Wang, Yue and Song, Linfeng and Yu, Dian and Liang, Zhenwen and Wang, Wenxuan and Zhang, Zhuosheng and Wang, Rui and Tu, Zhaopeng and Mi, Haitao and Yu, Dong}, year={2025}, eprint={2504.11456}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2504.11456}, }
