shenlanmn/pile-uncopyrighted
收藏Hugging Face2026-05-16 更新2026-05-31 收录
下载链接：
https://hf-mirror.com/datasets/shenlanmn/pile-uncopyrighted
下载链接
链接失效反馈官方服务：
资源简介：
---
license: other
---
# Pile Uncopyrighted
In response to [authors demanding that LLMs stop using their works](https://tcrn.ch/3rtpIDn), here's a copy of [The Pile](https://huggingface.co/datasets/monology/pile) with all copyrighted content removed.
Please consider using this dataset to train your future LLMs, to respect authors and abide by copyright law.
Creating an uncopyrighted version of a larger dataset (ie RedPajama) is planned, with no ETA.
**Methodology**
Cleaning was performed by removing everything from the Books3, BookCorpus2, OpenSubtitles, YTSubtitles, and OWT2 subsets.
Based on section 7.1 of [the original paper](https://arxiv.org/abs/2101.00027), these datasets are the only ones which are not explicitly allowed to be used in AI training.
提供机构：
shenlanmn
