scientifi-papers/scientific-papers

这是一个包含约5700万篇科学论文全文的原始数据集，从多个大规模学术论文集合中提取。该数据集提供原始全文，适用于预训练、微调或构建科学文献搜索索引。数据集包括四个子集：papers-2（约1850万篇论文，来自S2ORC论文集合的未命名子集）、papers-3（约2740万篇论文，来自S2ORC科学论文集合）、pes2o（约820万篇论文，来自Pes2oX-fulltext，基于Semantic Scholar数据）和corex（约290万篇论文，来自CORE存储库）。数据模式包含9列，如paper_id、title、authors、year、venue、doi、abstract、raw_fulltext和text_length。数据集基于CC-BY-4.0许可证。

A raw full-text dataset of ~57 million scientific papers, extracted from multiple large-scale academic paper collections. This dataset provides raw full text suitable for pre-training, fine-tuning, or building search indices over scientific literature. It includes four subsets: papers-2 (~18.5M papers from the S2ORC papers collection untitled subset), papers-3 (~27.4M papers from the S2ORC scientific-papers collection), pes2o (~8.2M papers from Pes2oX-fulltext, based on Semantic Scholar data), and corex (~2.9M papers from the CORE repository). The schema consists of 9 columns, such as paper_id, title, authors, year, venue, doi, abstract, raw_fulltext, and text_length. The dataset is licensed under CC-BY-4.0.