MultilingualUnigramLM/FineWeb2-khm_Khmr-100M
收藏Hugging Face2026-05-05 更新2026-05-31 收录
下载链接：
https://hf-mirror.com/datasets/MultilingualUnigramLM/FineWeb2-khm_Khmr-100M
下载链接
链接失效反馈官方服务：
资源简介：
FineWeb2-khm_Khmr-100M是一个包含1亿个token的高棉语数据集切片，源自HuggingFaceFW/fineweb-2数据集，配置为khm_Khmr。它包含1,008,681个文档，使用空格分隔的token数为100,000,211个，总字符数为2,077,801,439个。数据以单个Parquet文件（data/train-00000-of-00001.parquet）存储，模式为(text: string)，专为LAS分词器训练设计，并模仿了LangMap-TheStack-*-100M的布局。
FineWeb2-khm_Khmr-100M is a 100-million-token slice of the HuggingFaceFW/fineweb-2 dataset, specifically configured for khm_Khmr (Khmer language). It contains 1,008,681 documents with 100,000,211 whitespace-separated tokens and 2,077,801,439 characters. The data is stored in a single Parquet file (data/train-00000-of-00001.parquet) with a schema of (text: string), designed for LAS tokenizer training and mirrors the LangMap-TheStack-*-100M layout.
