openeurollm/dolci-instruct-sft-tokenized

---
license: apache-2.0
language:
- en
tags:
- olmo
- sft
- tokenized
- olmo-core
size_categories:
- 1M<n<10M
---

# Dolci-Instruct-SFT Tokenized

Pre-tokenized version of the [allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT) dataset, ready for training with [OLMo-core](https://github.com/allenai/OLMo-core).

See also: [openeurollm/dolci-think-sft-tokenized](https://huggingface.co/datasets/openeurollm/dolci-think-sft-tokenized) for the thinking variant.

## Dataset Details

| Property | Value |
|----------|-------|
| Source dataset | [allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT) |
| Tokenizer | [allenai/Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) |
| Max sequence length | 32,768 |
| Total instances | 2,152,111 |
| Total tokens | 1.7B |
| Trainable tokens | 789M (46.2%) |
| Avg tokens per instance | 793 |

During SFT, only assistant response tokens are trainable. System and user message tokens are masked out via `labels_mask` so the model sees them as context but is not trained to predict them. The lower trainable ratio (46.2%) reflects the shorter assistant responses in this dataset compared to the thinking variant, where system/user prompt tokens make up a larger share of each sequence.

## File Format

The dataset is stored as pre-merged NumPy arrays compatible with OLMo-core's data loading:

- `token_ids_part_XXXX.npy`: token ID arrays
- `labels_mask_part_XXXX.npy`: label mask arrays, where `1` = trainable (assistant response) and `0` = masked (system/user message)
- `tokenizer/`: tokenizer files used during tokenization
- `dataset_statistics.json`: detailed statistics about the tokenized dataset

## Usage with OLMo-core

Point your OLMo-core training config to this dataset directory. The format is directly compatible with the OLMo-core SFT data loader.

## License

Apache 2.0

许可证：Apache 2.0
语言：英语
标签：olmo、sft、tokenized、olmo-core
数据规模分类：100万 < 样本数 < 1000万

# Dolci-Instruct-SFT 分词版数据集
本数据集为[allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT)的预分词版本，可直接用于基于[OLMo-core](https://github.com/allenai/OLMo-core)的模型训练。

如需思维链变体版本，请参阅：[openeurollm/dolci-think-sft-tokenized](https://huggingface.co/datasets/openeurollm/dolci-think-sft-tokenized)。

## 数据集详情
| 属性项 | 属性值 |
|-------|-------|
| 源数据集 | [allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT) |
| 分词器 | [allenai/Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) |
| 最大序列长度 | 32768 |
| 总样本数 | 2,152,111 |
| 总Token数 | 17亿 |
| 可训练Token数 | 7.89亿（占比46.2%） |
| 单样本平均Token数 | 793 |

在监督微调（SFT，Supervised Fine-tuning）阶段，仅助手回复的Token可参与训练。系统提示与用户消息的Token将通过`labels_mask`进行掩码处理，使得模型可将其作为上下文读取，但无需对其进行预测。本数据集的可训练Token占比仅为46.2%，这是因为相较于思维链变体版本，本数据集内的助手回复长度更短，而思维链变体中系统/用户提示Token在每条序列中占比更高。

## 文件格式
本数据集以预合并的NumPy数组形式存储，与OLMo-core的数据加载流程兼容：
- `token_ids_part_XXXX.npy`：Token ID数组文件
- `labels_mask_part_XXXX.npy`：标签掩码数组文件，其中`1`代表可训练Token（对应助手回复内容），`0`代表掩码Token（对应系统/用户消息内容）
- `tokenizer/`：分词时使用的分词器文件目录
- `dataset_statistics.json`：包含分词后数据集详细统计信息的JSON文件

## OLMo-core 使用指南
将OLMo-core的训练配置指向本数据集目录即可直接使用，本数据集格式与OLMo-core的监督微调数据加载器完全兼容。

## 许可证
Apache 2.0