allenai/dolma3_dolmino_pool

---
license: odc-by
task_categories:
- text-generation
pretty_name: dolma3-dolmino Pool (October 2025)
language:
- en
configs:
- config_name: default
data_files:
- split: train
path: data/**/*

- config_name: stem_heavy_crawl
data_files:
- split: train
path: data/stem_heavy_crawl/**/*

- config_name: stack_edu_fim
data_files:
- split: train
path: data/stack_edu-fim-*/**/*

- config_name: cranecode
data_files:
- split: train
path: data/cranecode/**/*

- config_name: cranemath
data_files:
- split: train
path: data/cranemath/**/*

- config_name: megamatt
data_files:
- split: train
path: data/megamatt/**/*

- config_name: dolmino1_math
data_files:
- split: train
path: data/dolmino1_math/**/*

- config_name: omr_rewrite_fullthoughts
data_files:
- split: train
path: data/omr-rewrite-fullthoughts/**/*

- config_name: tinyMATH_mind
data_files:
- split: train
path: data/tinyMATH-mind/**/*

- config_name: tinyMATH_pot
data_files:
- split: train
path: data/tinyMATH-pot/**/*

- config_name: reddit_high
data_files:
- split: train
path: data/reddit_to_flashcards-high_relevance/**/*

- config_name: reddit_low
data_files:
- split: train
path: data/reddit_to_flashcards-low_relevance/**/*

- config_name: wiki_to_rcqa
data_files:
- split: train
path: data/wiki_to_rcqa/**/*

- config_name: nemotron_synth_qa
data_files:
- split: train
path: data/nemotron-synth_qa/**/*

- config_name: tulu_3_sft
data_files:
- split: train
path: data/tulu_3_sft/**/*

- config_name: dolmino_1_flan
data_files:
- split: train
path: data/dolmino_1-flan/**/*

- config_name: r1_reasoning_traces
data_files:
- split: train
path: data/r1_reasoning_traces/**/*

- config_name: qwq_reasoning_traces
data_files:
- split: train
path: data/qwq_reasoning_traces/**/*

- config_name: gemini_reasoning_traces
data_files:
- split: train
path: data/gemini_reasoning_traces/**/*

- config_name: llamanemotron_reasoning_traces
data_files:
- split: train
path: data/llamanemotron_reasoning_traces/**/*

- config_name: openthoughts2_reasoning_traces
data_files:
- split: train
path: data/openthoughts2_reasoning_traces/**/*

- config_name: verifiable_gpt41
data_files:
- split: train
path: data/verifiable-gpt41/**/*

- config_name: verifiable_o4mini
data_files:
- split: train
path: data/verifiable-o4mini/**/*

- config_name: math_meta_reasoning
data_files:
- split: train
path: data/math_meta_reasoning/**/*

- config_name: code_meta_reasoning
data_files:
- split: train
path: data/code_meta_reasoning/**/*

- config_name: olmocr_science_pdfs
data_files:
- split: train
path: data/olmocr_science_pdfs-high_quality-*/**/*

- config_name: common_crawl_hq
data_files:
- split: train
path: data/common_crawl-high_quality-*/**/*

---

⚠️ **IMPORTANT NOTICE** ⚠️

This is the Dolma 3 Dolmino **pool**; it hasn't been mixed.
If you are interested in *the data used* to train:
- [Olmo 3 7B](https://huggingface.co/allenai/Olmo-3-1025-7B): [**allenai/dolma3_dolmino_mix-100B-1025**](https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1025)
- [Olmo 3 32B](https://huggingface.co/allenai/Olmo-3-1025-32B): [**allenai/dolma3_dolmino_mix-100B-1125**](https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1125)

----

<img alt="Logo for Dolmino Pool" src="dolmino-pool.png" width="301px" style="margin-left:'auto' margin-right:'auto' display:'block'">

# Dolma 3 Dolmino dataset pool for Olmo 3 stage 2 annealing training

This dataset contains the high-quality pool of data considered for the second stage of Olmo 3 7B.

## Dataset Sources

| Source | Category | Tokens | Documents |
|--------|----------|--------|-----------|
| TinyMATH Mind | Math (synth) | 899M | 1.42M |
| TinyMATH PoT | Math (synth) | 241M | 729K |
| CraneMath | Math (synth) | 5.62B | 6.55M |
| MegaMatt | Math (synth) | 3.88B | 6.79M |
| Dolmino Math | Math (synth) | 10.7B | 21M |
| StackEdu (FIM) | Code | 21.4B | 32M |
| CraneCode | Python (synth) | 18.8B | 19.7M |
| Reddit To Flashcards | QA (synth) | 21.6B | 370M |
| Wiki To RCQA | QA (synth) | 4.22B | 22.3M |
| Nemotron Synth QA | QA (synth) | 487B | 972M |
| Math Meta-Reasoning | Thinking (synth) | 1.05B | 984K |
| Code Meta-Reasoning | Thinking (synth) | 1.27B | 910K |
| Program-Verifiable | Thinking (synth) | 438M | 384K |
| OMR Rewrite FullThoughts | Thinking (synth) | 850M | 291K |
| QWQ Reasoning Traces | Thinking (synth) | 4.77B | 438K |
| General Reasoning Mix | Thinking (synth) | 2.48B | 668K |
| Gemini Reasoning Traces | Thinking (synth) | 246M | 55.2K |
| Llama Nemotron Reasoning Traces | Thinking (synth) | 20.9B | 3.91M |
| OpenThoughts2 Reasoning Traces | Thinking (synth) | 5.6B | 1.11M |
| Tulu 3 SFT | Instruction (synth) | 1.61B | 1.95M |
| Dolmino 1 Flan | Instruction (synth) | 16.8B | 56.9M |
| OLMOCR Science PDFs (High Q.) | PDFs | 240B | 28.7M |
| STEM-Heavy Crawl | Web pages | 5.21B | 5.16M |
| Common Crawl (High Q.) | Web pages | 1.32T | 965M |
| **Total** | | **2.19T** | **2.52B** |

---

## Mix Compositions

| Source | 10B | | 100B | |
|--------|-----|-----|------|-----|
| | Source % | Mix % | Source % | Mix % |
| STEM-Heavy Crawl | - | - | 5.0% | 5.0% |
| StackEdu (FIM) | - | - | 10.0% | 10.0% |
| CraneCode | - | - | 10.0% | 10.0% |
| CraneMath | - | - | 5.63% | 5.63% |
| MegaMatt | - | - | 1.73% | 1.73% |
| Dolmino Math | - | - | 10.7% | 10.7% |
| OMR Rewrite FullThoughts | - | - | 0.85% | 0.85% |
| TinyMATH Mind | - | - | 0.9% | 0.9% |
| TinyMATH PoT | - | - | 0.24% | 0.24% |
| Reddit To Flashcards | - | - | 5.9% | 5.9% |
| Wiki To RCQA | - | - | 3.0% | 3.0% |
| Nemotron Synth QA | - | - | 5.0% | 5.0% |
| Tulu 3 SFT | - | - | 1.1% | 1.1% |
| Dolmino 1 Flan | - | - | 5.0% | 5.0% |
| QWQ Reasoning Traces | - | - | 1.87% | 1.87% |
| Gemini Reasoning Traces | - | - | 0.25% | 0.25% |
| Llama Nemotron Reasoning Traces | - | - | 1.25% | 1.25% |
| OpenThoughts2 Reasoning Traces | - | - | 1.25% | 1.25% |
| Program-Verifiable | - | - | 0.16% | 0.16% |
| Math Meta-Reasoning | - | - | 0.38% | 0.38% |
| Code Meta-Reasoning | - | - | 0.46% | 0.46% |
| General Reasoning Mix | - | - | 1.87% | 1.87% |
| OLMOCR Science PDFs (High Q.) | - | - | 5.0% | 5.0% |
| Common Crawl (High Q.) | - | - | 22.5% | 22.5% |

## Licensing Information

Dolma 3 Dolmino is licensed under the Open Data Commons Attribution License v1.0 (ODC-By). It is intended for research and educational use. For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).

## Citation
A technical manuscript is forthcoming!

许可证：开放数据共同体署名许可v1.0（Open Data Commons Attribution License v1.0，简称ODC-By）
任务类别：文本生成
友好名称：dolma3-dolmino池（2025年10月）
语言：英语
配置项：
- 配置名称：default
数据文件：
- 拆分方式：训练集
路径：data/**/*
- 配置名称：stem_heavy_crawl
数据文件：
- 拆分方式：训练集
路径：data/stem_heavy_crawl/**/*
- 配置名称：stack_edu_fim
数据文件：
- 拆分方式：训练集
路径：data/stack_edu-fim-*/**/*
- 配置名称：cranecode
数据文件：
- 拆分方式：训练集
路径：data/cranecode/**/*
- 配置名称：cranemath
数据文件：
- 拆分方式：训练集
路径：data/cranemath/**/*
- 配置名称：megamatt
数据文件：
- 拆分方式：训练集
路径：data/megamatt/**/*
- 配置名称：dolmino1_math
数据文件：
- 拆分方式：训练集
路径：data/dolmino1_math/**/*
- 配置名称：omr_rewrite_fullthoughts
数据文件：
- 拆分方式：训练集
路径：data/omr-rewrite-fullthoughts/**/*
- 配置名称：tinyMATH_mind
数据文件：
- 拆分方式：训练集
路径：data/tinyMATH-mind/**/*
- 配置名称：tinyMATH_pot
数据文件：
- 拆分方式：训练集
路径：data/tinyMATH-pot/**/*
- 配置名称：reddit_high
数据文件：
- 拆分方式：训练集
路径：data/reddit_to_flashcards-high_relevance/**/*
- 配置名称：reddit_low
数据文件：
- 拆分方式：训练集
路径：data/reddit_to_flashcards-low_relevance/**/*
- 配置名称：wiki_to_rcqa
数据文件：
- 拆分方式：训练集
路径：data/wiki_to_rcqa/**/*
- 配置名称：nemotron_synth_qa
数据文件：
- 拆分方式：训练集
路径：data/nemotron-synth_qa/**/*
- 配置名称：tulu_3_sft
数据文件：
- 拆分方式：训练集
路径：data/tulu_3_sft/**/*
- 配置名称：dolmino_1_flan
数据文件：
- 拆分方式：训练集
路径：data/dolmino_1-flan/**/*
- 配置名称：r1_reasoning_traces
数据文件：
- 拆分方式：训练集
路径：data/r1_reasoning_traces/**/*
- 配置名称：qwq_reasoning_traces
数据文件：
- 拆分方式：训练集
路径：data/qwq_reasoning_traces/**/*
- 配置名称：gemini_reasoning_traces
数据文件：
- 拆分方式：训练集
路径：data/gemini_reasoning_traces/**/*
- 配置名称：llamanemotron_reasoning_traces
数据文件：
- 拆分方式：训练集
路径：data/llamanemotron_reasoning_traces/**/*
- 配置名称：openthoughts2_reasoning_traces
数据文件：
- 拆分方式：训练集
路径：data/openthoughts2_reasoning_traces/**/*
- 配置名称：verifiable_gpt41
数据文件：
- 拆分方式：训练集
路径：data/verifiable-gpt41/**/*
- 配置名称：verifiable_o4mini
数据文件：
- 拆分方式：训练集
路径：data/verifiable-o4mini/**/*
- 配置名称：math_meta_reasoning
数据文件：
- 拆分方式：训练集
路径：data/math_meta_reasoning/**/*
- 配置名称：code_meta_reasoning
数据文件：
- 拆分方式：训练集
路径：data/code_meta_reasoning/**/*
- 配置名称：olmocr_science_pdfs
数据文件：
- 拆分方式：训练集
路径：data/olmocr_science_pdfs-high_quality-*/**/*
- 配置名称：common_crawl_hq
数据文件：
- 拆分方式：训练集
路径：data/common_crawl-high_quality-*/**/*

⚠️ 重要声明 ⚠️
本数据集为Dolma 3 Dolmino**候选语料池**，尚未完成混合。若您希望获取用于训练以下模型的语料：
- [Olmo 3 7B]("https://huggingface.co/allenai/Olmo-3-1025-7B")：请访问数据集 ["allenai/dolma3_dolmino_mix-100B-1025"]("https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1025")
- [Olmo 3 32B]("https://huggingface.co/allenai/Olmo-3-1025-32B")：请访问数据集 ["allenai/dolma3_dolmino_mix-100B-1125"]("https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1125")

<img alt="Dolmino Pool 标识" src="dolmino-pool.png" width="301px" style="margin-left:auto; margin-right:auto; display:block">

# 用于Olmo 3第二阶段退火训练的Dolma 3 Dolmino数据集池
本数据集包含用于Olmo 3 7B第二阶段训练的高质量候选语料集合。

## 数据集来源
| 数据源 | 类别 | Token数 | 文档数 |
|--------|------|--------|---------|
| TinyMATH Mind | 数学合成数据 | 899M（8.99亿Token） | 142万 |
| TinyMATH PoT | 数学合成数据 | 241M（2.41亿Token） | 72.9万 |
| CraneMath | 数学合成数据 | 5.62B（56.2亿Token） | 655万 |
| MegaMatt | 数学合成数据 | 3.88B（38.8亿Token） | 679万 |
| Dolmino Math | 数学合成数据 | 10.7B（107亿Token） | 2100万 |
| StackEdu (FIM) | 代码合成数据 | 21.4B（214亿Token） | 3200万 |
| CraneCode | Python代码合成数据 | 18.8B（188亿Token） | 1970万 |
| Reddit To Flashcards | 问答合成数据 | 21.6B（216亿Token） | 3.7亿 |
| Wiki To RCQA | 问答合成数据 | 4.22B（42.2亿Token） | 2230万 |
| Nemotron Synth QA | 问答合成数据 | 487B（4870亿Token） | 972万 |
| Math Meta-Reasoning | 思维链合成数据 | 1.05B（10.5亿Token） | 98.4万 |
| Code Meta-Reasoning | 思维链合成数据 | 1.27B（12.7亿Token） | 91万 |
| Program-Verifiable | 思维链合成数据 | 438M（4.38亿Token） | 38.4万 |
| OMR Rewrite FullThoughts | 思维链合成数据 | 850M（8.5亿Token） | 29.1万 |
| QWQ Reasoning Traces | 思维链合成数据 | 4.77B（47.7亿Token） | 43.8万 |
| General Reasoning Mix | 通用思维链混合数据 | 2.48B（24.8亿Token） | 66.8万 |
| Gemini Reasoning Traces | 思维链合成数据 | 246M（2.46亿Token） | 5.52万 |
| Llama Nemotron Reasoning Traces | 思维链合成数据 | 20.9B（209亿Token） | 391万 |
| OpenThoughts2 Reasoning Traces | 思维链合成数据 | 5.6B（56亿Token） | 111万 |
| Tulu 3 SFT | 指令微调合成数据 | 1.61B（16.1亿Token） | 195万 |
| Dolmino 1 Flan | 指令微调合成数据 | 16.8B（168亿Token） | 5690万 |
| OLMOCR Science PDFs (High Q.) | 高质量科学PDF数据 | 240B（2400亿Token） | 2870万 |
| STEM-Heavy Crawl | 高质量网页数据 | 5.21B（52.1亿Token） | 516万 |
| Common Crawl (High Q.) | 高质量通用网页数据 | 1.32T（1.32万亿Token） | 9.65亿 |
| **总计** | | **2.19T（2.19万亿Token）** | **25.2亿** |

## 混合组成比例
| 数据源 | 10B规模语料 | | 100B规模语料 | |
|--------|-----|-----|------|-----|
| | 数据源占比 | 混合占比 | 数据源占比 | 混合占比 |
| STEM-Heavy Crawl | - | - | 5.0% | 5.0% |
| StackEdu (FIM) | - | - | 10.0% | 10.0% |
| CraneCode | - | - | 10.0% | 10.0% |
| CraneMath | - | - | 5.63% | 5.63% |
| MegaMatt | - | - | 1.73% | 1.73% |
| Dolmino Math | - | - | 10.7% | 10.7% |
| OMR Rewrite FullThoughts | - | - | 0.85% | 0.85% |
| TinyMATH Mind | - | - | 0.9% | 0.9% |
| TinyMATH PoT | - | - | 0.24% | 0.24% |
| Reddit To Flashcards | - | - | 5.9% | 5.9% |
| Wiki To RCQA | - | - | 3.0% | 3.0% |
| Nemotron Synth QA | - | - | 5.0% | 5.0% |
| Tulu 3 SFT | - | - | 1.1% | 1.1% |
| Dolmino 1 Flan | - | - | 5.0% | 5.0% |
| QWQ Reasoning Traces | - | - | 1.87% | 1.87% |
| Gemini Reasoning Traces | - | - | 0.25% | 0.25% |
| Llama Nemotron Reasoning Traces | - | - | 1.25% | 1.25% |
| OpenThoughts2 Reasoning Traces | - | - | 1.25% | 1.25% |
| Program-Verifiable | - | - | 0.16% | 0.16% |
| Math Meta-Reasoning | - | - | 0.38% | 0.38% |
| Code Meta-Reasoning | - | - | 0.46% | 0.46% |
| General Reasoning Mix | - | - | 1.87% | 1.87% |
| OLMOCR Science PDFs (High Q.) | - | - | 5.0% | 5.0% |
| Common Crawl (High Q.) | - | - | 22.5% | 22.5% |

## 许可信息
Dolma 3 Dolmino数据集采用开放数据共同体署名许可v1.0（Open Data Commons Attribution License v1.0，简称ODC-By）授权，仅可用于研究与教育用途。更多相关信息请参阅[负责任使用指南]("https://allenai.org/responsible-use")。

## 引用说明
技术手稿即将发布！