MichaelR207/rephraser_kimi_v1_0331

---
dataset_info:
features:
- name: messages
list:
- name: role
dtype: string
- name: content
dtype: string
- name: warc_file
dtype: string
- name: doc_id
dtype: string
- name: spec_id
dtype: string
- name: spec
dtype: string
- name: model
dtype: string
splits:
- name: train
num_examples: 57799
- name: validation
num_examples: 100
license: apache-2.0
---

# rephraser_kimi_v1_0331

Web extraction distillation dataset reprocessed with **moonshotai/Kimi-K2.5**.

This dataset contains HTML-to-text extraction results where each row has a system prompt,
a user prompt containing HTML + extraction spec, and an assistant response with
`<think>` reasoning followed by the extracted text.

## Source

Reprocessed from [MichaelR207/rephraser_late_check_0225](https://huggingface.co/datasets/MichaelR207/rephraser_late_check_0225)
using the same input prompts but replacing GPT-OSS-120B outputs with Kimi-K2.5 outputs.

## Token Statistics

Token counts computed using the `cl100k_base` tokenizer.

- **Input tokens**: tokens in the prompt sent to the model.
- **Reasoning tokens**: tokens used for chain-of-thought reasoning (`<think>...</think>`).
- **Output tokens**: non-reasoning completion tokens, i.e. the actual extracted text.
- **Total**: input + reasoning + output.

| Metric | Train | Validation | Total |
|--------|------:|----------:|------:|
| Input tokens | 988,863,730 | 1,656,610 | 990,520,340 |
| Reasoning tokens | 107,516,740 | 386,840 | 107,903,580 |
| Output tokens | 8,754,403 | 233,422 | 8,987,825 |
| Total | 1,105,134,873 | 2,276,872 | 1,107,411,745 |
| Rows | 57,799 | 100 | 57,899 |

## Schema

Each row contains:
- `messages`: list of 3 dicts (`system`, `user`, `assistant`) -- compatible with chat fine-tuning
- `warc_file`: source Common Crawl WARC file
- `doc_id`: document identifier within the WARC file
- `spec_id`: extraction specification identifier (0-999)
- `spec`: full text of the extraction specification
- `model`: model used to generate the assistant response

## Coverage

This dataset covers the first 7% of each extraction spec rows from the source dataset.