five

allenai/scirepeval

收藏
Hugging Face2024-01-16 更新2024-03-04 收录
下载链接：
https://hf-mirror.com/datasets/allenai/scirepeval
下载链接
链接失效反馈
官方服务：
资源简介：
--- dataset_info: - config_name: biomimicry features: - name: doc_id dtype: string - name: doi dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: label dtype: uint32 - name: venue dtype: string splits: - name: evaluation num_bytes: 16652415 num_examples: 10991 download_size: 9314032 dataset_size: 16652415 - config_name: cite_count features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: venue dtype: string - name: n_citations dtype: int32 - name: log_citations dtype: float32 splits: - name: evaluation num_bytes: 45741032 num_examples: 30058 - name: train num_bytes: 265390284 num_examples: 175944 - name: validation num_bytes: 40997159 num_examples: 26830 download_size: 204760850 dataset_size: 352128475 - config_name: cite_prediction features: - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: sha dtype: string - name: corpus_id dtype: uint64 - name: pos struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: sha dtype: string - name: corpus_id dtype: uint64 - name: neg struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: sha dtype: string - name: corpus_id dtype: uint64 splits: - name: train num_bytes: 2582594392 num_examples: 676150 - name: validation num_bytes: 549599739 num_examples: 143686 download_size: 1854909838 dataset_size: 3132194131 - config_name: cite_prediction_aug2023refresh features: - name: query struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: pos struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: neg struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 splits: - name: train num_bytes: 2069439948 num_examples: 475656 download_size: 1222814801 dataset_size: 2069439948 - config_name: cite_prediction_new features: - name: query struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: pos struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: neg struct: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: int8 splits: - name: train num_bytes: 23829782726 num_examples: 6197963 - name: validation num_bytes: 609822308 num_examples: 176430 download_size: 14512970071 dataset_size: 24439605034 - config_name: drsm features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: label_type dtype: string - name: label dtype: string - name: class dtype: uint32 splits: - name: evaluation num_bytes: 12757612 num_examples: 8813 download_size: 7021949 dataset_size: 12757612 - config_name: feeds_1 features: - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: feed_id dtype: string - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 6488182 num_examples: 423 download_size: 6911928 dataset_size: 6488182 - config_name: feeds_m features: - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: feed_id dtype: string - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 135219457 num_examples: 9025 download_size: 149126628 dataset_size: 135219457 - config_name: feeds_title features: - name: query dtype: string - name: doc_id dtype: string - name: feed_id dtype: string - name: abbreviations dtype: string - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 5923757 num_examples: 424 download_size: 6228046 dataset_size: 5923757 - config_name: fos features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: labels sequence: int32 - name: labels_text sequence: string splits: - name: evaluation num_bytes: 63854253 num_examples: 68147 - name: train num_bytes: 509154623 num_examples: 541218 - name: validation num_bytes: 63947785 num_examples: 67631 download_size: 382411779 dataset_size: 636956661 - config_name: high_influence_cite features: - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 85746699 num_examples: 1199 - name: train num_bytes: 2607643584 num_examples: 58626 - name: validation num_bytes: 329589399 num_examples: 7356 download_size: 1622948830 dataset_size: 3022979682 - config_name: mesh_descriptors features: - name: doc_id dtype: string - name: mag_id dtype: uint64 - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: descriptor dtype: string - name: qualifier dtype: string splits: - name: evaluation num_bytes: 390178523 num_examples: 258678 - name: train num_bytes: 3120119117 num_examples: 2069065 - name: validation num_bytes: 390161743 num_examples: 258678 download_size: 2259106030 dataset_size: 3900459383 - config_name: nfcorpus features: - name: query dtype: string - name: doc_id dtype: string - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: score dtype: uint32 splits: - name: evaluation num_bytes: 72184049 num_examples: 323 download_size: 37626800 dataset_size: 72184049 - config_name: paper_reviewer_matching features: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 splits: - name: evaluation num_bytes: 76005977 num_examples: 73364 download_size: 41557009 dataset_size: 76005977 - config_name: peer_review_score_hIndex features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: rating sequence: int32 - name: confidence dtype: string - name: authors sequence: string - name: decision dtype: string - name: mean_rating dtype: float32 - name: hIndex sequence: string splits: - name: evaluation num_bytes: 18233937 num_examples: 12668 download_size: 10163532 dataset_size: 18233937 - config_name: pub_year features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: year dtype: int32 - name: venue dtype: string - name: norm_year dtype: float32 - name: scaled_year dtype: float32 - name: n_authors dtype: int32 - name: norm_authors dtype: float32 splits: - name: evaluation num_bytes: 46195045 num_examples: 30000 - name: train num_bytes: 301313882 num_examples: 198995 - name: validation num_bytes: 30493617 num_examples: 19869 download_size: 224105260 dataset_size: 378002544 - config_name: relish features: - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: int64 - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: int64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 338282942 num_examples: 3190 download_size: 171723654 dataset_size: 338282942 - config_name: same_author features: - name: dataset dtype: string - name: query struct: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 126843745 num_examples: 13585 - name: train num_bytes: 602167333 num_examples: 67493 - name: validation num_bytes: 84426967 num_examples: 8996 download_size: 104055242 dataset_size: 813438045 - config_name: scidocs_mag_mesh features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: authors sequence: string - name: cited_by sequence: string - name: references sequence: string - name: year dtype: int32 splits: - name: evaluation num_bytes: 74030118 num_examples: 48473 download_size: 47773142 dataset_size: 74030118 - config_name: scidocs_view_cite_read features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: authors sequence: string - name: cited_by sequence: string - name: references sequence: string - name: year dtype: int32 splits: - name: evaluation num_bytes: 240569108 num_examples: 142009 download_size: 159403764 dataset_size: 240569108 - config_name: search features: - name: query dtype: string - name: doc_id dtype: string - name: candidates list: - name: doc_id dtype: string - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: uint64 - name: venue dtype: string - name: year dtype: float64 - name: author_names sequence: string - name: n_citations dtype: int32 - name: n_key_citations dtype: int32 - name: score dtype: uint32 splits: - name: evaluation num_bytes: 39417912 num_examples: 2637 - name: train num_bytes: 6889691036 num_examples: 399878 - name: validation num_bytes: 1221360738 num_examples: 75382 download_size: 4495463131 dataset_size: 8150469686 - config_name: trec_covid features: - name: query dtype: string - name: doc_id dtype: string - name: candidates list: - name: title dtype: string - name: abstract dtype: string - name: corpus_id dtype: string - name: doc_id dtype: string - name: date dtype: string - name: doi dtype: string - name: iteration dtype: string - name: score dtype: int32 splits: - name: evaluation num_bytes: 98757931 num_examples: 50 download_size: 52359825 dataset_size: 98757931 - config_name: tweet_mentions features: - name: doc_id dtype: string - name: corpus_id dtype: uint64 - name: title dtype: string - name: abstract dtype: string - name: index dtype: int32 - name: retweets dtype: float32 - name: count dtype: int32 - name: mentions dtype: float32 splits: - name: evaluation num_bytes: 25895172 num_examples: 25655 download_size: 14991004 dataset_size: 25895172 configs: - config_name: biomimicry data_files: - split: evaluation path: biomimicry/evaluation-* - config_name: cite_count data_files: - split: evaluation path: cite_count/evaluation-* - split: train path: cite_count/train-* - split: validation path: cite_count/validation-* - config_name: cite_prediction data_files: - split: train path: cite_prediction/train-* - split: validation path: cite_prediction/validation-* - config_name: cite_prediction_aug2023refresh data_files: - split: train path: cite_prediction_aug2023refresh/train-* - config_name: cite_prediction_new data_files: - split: train path: cite_prediction_new/train-* - split: validation path: cite_prediction_new/validation-* - config_name: drsm data_files: - split: evaluation path: drsm/evaluation-* - config_name: fos data_files: - split: evaluation path: fos/evaluation-* - split: train path: fos/train-* - split: validation path: fos/validation-* - config_name: high_influence_cite data_files: - split: evaluation path: high_influence_cite/evaluation-* - split: train path: high_influence_cite/train-* - split: validation path: high_influence_cite/validation-* - config_name: mesh_descriptors data_files: - split: evaluation path: mesh_descriptors/evaluation-* - split: train path: mesh_descriptors/train-* - split: validation path: mesh_descriptors/validation-* - config_name: nfcorpus data_files: - split: evaluation path: nfcorpus/evaluation-* - config_name: paper_reviewer_matching data_files: - split: evaluation path: paper_reviewer_matching/evaluation-* - config_name: peer_review_score_hIndex data_files: - split: evaluation path: peer_review_score_hIndex/evaluation-* - config_name: pub_year data_files: - split: evaluation path: pub_year/evaluation-* - split: train path: pub_year/train-* - split: validation path: pub_year/validation-* - config_name: relish data_files: - split: evaluation path: relish/evaluation-* - config_name: same_author data_files: - split: evaluation path: same_author/evaluation-* - split: train path: same_author/train-* - split: validation path: same_author/validation-* - config_name: scidocs_mag_mesh data_files: - split: evaluation path: scidocs_mag_mesh/evaluation-* - config_name: scidocs_view_cite_read data_files: - split: evaluation path: scidocs_view_cite_read/evaluation-* - config_name: search data_files: - split: evaluation path: search/evaluation-* - split: train path: search/train-* - split: validation path: search/validation-* - config_name: trec_covid data_files: - split: evaluation path: trec_covid/evaluation-* - config_name: tweet_mentions data_files: - split: evaluation path: tweet_mentions/evaluation-* ---
提供机构：
allenai
原始信息汇总

数据集概述

数据集配置

1. biomimicry

  • 特征:
    • doc_id: string
    • doi: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • label: uint32
    • venue: string
  • 分割:
    • evaluation: 16652415 bytes, 10991 examples
  • 下载大小: 9314032 bytes
  • 数据集大小: 16652415 bytes

2. cite_count

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • venue: string
    • n_citations: int32
    • log_citations: float32
  • 分割:
    • evaluation: 45741032 bytes, 30058 examples
    • train: 265390284 bytes, 175944 examples
    • validation: 40997159 bytes, 26830 examples
  • 下载大小: 204760850 bytes
  • 数据集大小: 352128475 bytes

3. cite_prediction

  • 特征:
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • sha: string
      • corpus_id: uint64
    • pos: struct
      • doc_id: string
      • title: string
      • abstract: string
      • sha: string
      • corpus_id: uint64
    • neg: struct
      • doc_id: string
      • title: string
      • abstract: string
      • sha: string
      • corpus_id: uint64
  • 分割:
    • train: 2582594392 bytes, 676150 examples
    • validation: 549599739 bytes, 143686 examples
  • 下载大小: 1854909838 bytes
  • 数据集大小: 3132194131 bytes

4. cite_prediction_aug2023refresh

  • 特征:
    • query: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
    • pos: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
    • neg: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
  • 分割:
    • train: 2069439948 bytes, 475656 examples
  • 下载大小: 1222814801 bytes
  • 数据集大小: 2069439948 bytes

5. cite_prediction_new

  • 特征:
    • query: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
    • pos: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
    • neg: struct
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: int8
  • 分割:
    • train: 23829782726 bytes, 6197963 examples
    • validation: 609822308 bytes, 176430 examples
  • 下载大小: 14512970071 bytes
  • 数据集大小: 24439605034 bytes

6. drsm

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • label_type: string
    • label: string
    • class: uint32
  • 分割:
    • evaluation: 12757612 bytes, 8813 examples
  • 下载大小: 7021949 bytes
  • 数据集大小: 12757612 bytes

7. feeds_1

  • 特征:
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
    • feed_id: string
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: uint32
  • 分割:
    • evaluation: 6488182 bytes, 423 examples
  • 下载大小: 6911928 bytes
  • 数据集大小: 6488182 bytes

8. feeds_m

  • 特征:
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
    • feed_id: string
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: uint32
  • 分割:
    • evaluation: 135219457 bytes, 9025 examples
  • 下载大小: 149126628 bytes
  • 数据集大小: 135219457 bytes

9. feeds_title

  • 特征:
    • query: string
    • doc_id: string
    • feed_id: string
    • abbreviations: string
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: uint32
  • 分割:
    • evaluation: 5923757 bytes, 424 examples
  • 下载大小: 6228046 bytes
  • 数据集大小: 5923757 bytes

10. fos

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • labels: sequence: int32
    • labels_text: sequence: string
  • 分割:
    • evaluation: 63854253 bytes, 68147 examples
    • train: 509154623 bytes, 541218 examples
    • validation: 63947785 bytes, 67631 examples
  • 下载大小: 382411779 bytes
  • 数据集大小: 636956661 bytes

11. high_influence_cite

  • 特征:
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: uint32
  • 分割:
    • evaluation: 85746699 bytes, 1199 examples
    • train: 2607643584 bytes, 58626 examples
    • validation: 329589399 bytes, 7356 examples
  • 下载大小: 1622948830 bytes
  • 数据集大小: 3022979682 bytes

12. mesh_descriptors

  • 特征:
    • doc_id: string
    • mag_id: uint64
    • corpus_id: uint64
    • title: string
    • abstract: string
    • descriptor: string
    • qualifier: string
  • 分割:
    • evaluation: 390178523 bytes, 258678 examples
    • train: 3120119117 bytes, 2069065 examples
    • validation: 390161743 bytes, 258678 examples
  • 下载大小: 2259106030 bytes
  • 数据集大小: 3900459383 bytes

13. nfcorpus

  • 特征:
    • query: string
    • doc_id: string
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • score: uint32
  • 分割:
    • evaluation: 72184049 bytes, 323 examples
  • 下载大小: 37626800 bytes
  • 数据集大小: 72184049 bytes

14. paper_reviewer_matching

  • 特征:
    • doc_id: string
    • title: string
    • abstract: string
    • corpus_id: uint64
  • 分割:
    • evaluation: 76005977 bytes, 73364 examples
  • 下载大小: 41557009 bytes
  • 数据集大小: 76005977 bytes

15. peer_review_score_hIndex

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • rating: sequence: int32
    • confidence: string
    • authors: sequence: string
    • decision: string
    • mean_rating: float32
    • hIndex: sequence: string
  • 分割:
    • evaluation: 18233937 bytes, 12668 examples
  • 下载大小: 10163532 bytes
  • 数据集大小: 18233937 bytes

16. pub_year

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • year: int32
    • venue: string
    • norm_year: float32
    • scaled_year: float32
    • n_authors: int32
    • norm_authors: float32
  • 分割:
    • evaluation: 46195045 bytes, 30000 examples
    • train: 301313882 bytes, 198995 examples
    • validation: 30493617 bytes, 19869 examples
  • 下载大小: 224105260 bytes
  • 数据集大小: 378002544 bytes

17. relish

  • 特征:
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: int64
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: int64
      • score: uint32
  • 分割:
    • evaluation: 338282942 bytes, 3190 examples
  • 下载大小: 171723654 bytes
  • 数据集大小: 338282942 bytes

18. same_author

  • 特征:
    • dataset: string
    • query: struct
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • score: uint32
  • 分割:
    • evaluation: 126843745 bytes, 13585 examples
    • train: 602167333 bytes, 67493 examples
    • validation: 84426967 bytes, 8996 examples
  • 下载大小: 104055242 bytes
  • 数据集大小: 813438045 bytes

19. scidocs_mag_mesh

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • authors: sequence: string
    • cited_by: sequence: string
    • references: sequence: string
    • year: int32
  • 分割:
    • evaluation: 74030118 bytes, 48473 examples
  • 下载大小: 47773142 bytes
  • 数据集大小: 74030118 bytes

20. scidocs_view_cite_read

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • authors: sequence: string
    • cited_by: sequence: string
    • references: sequence: string
    • year: int32
  • 分割:
    • evaluation: 240569108 bytes, 142009 examples
  • 下载大小: 159403764 bytes
  • 数据集大小: 240569108 bytes

21. search

  • 特征:
    • query: string
    • doc_id: string
    • candidates: list
      • doc_id: string
      • title: string
      • abstract: string
      • corpus_id: uint64
      • venue: string
      • year: float64
      • author_names: sequence: string
      • n_citations: int32
      • n_key_citations: int32
      • score: uint32
  • 分割:
    • evaluation: 39417912 bytes, 2637 examples
    • train: 6889691036 bytes, 399878 examples
    • validation: 1221360738 bytes, 75382 examples
  • 下载大小: 4495463131 bytes
  • 数据集大小: 8150469686 bytes

22. trec_covid

  • 特征:
    • query: string
    • doc_id: string
    • candidates: list
      • title: string
      • abstract: string
      • corpus_id: string
      • doc_id: string
      • date: string
      • doi: string
      • iteration: string
      • score: int32
  • 分割:
    • evaluation: 98757931 bytes, 50 examples
  • 下载大小: 52359825 bytes
  • 数据集大小: 98757931 bytes

23. tweet_mentions

  • 特征:
    • doc_id: string
    • corpus_id: uint64
    • title: string
    • abstract: string
    • index:
搜集汇总
数据集介绍
main_image_url
构建方式
在科学文献计量学与自然语言处理交叉领域，数据集构建的严谨性直接决定了模型评估的可靠性。allenai/scirepeval数据集汇聚了超过20项子任务，涵盖引用预测、文献分类、作者识别等多个维度。其构建方式基于Semantic Scholar学术图谱，从海量论文元数据中提取标题、摘要、作者、引用关系等结构化信息，并针对不同任务设计了差异化的标注策略。例如，cite_prediction子集通过构建查询-正例-负例三元组来模拟文献推荐场景，而fos子集则借助领域本体对论文进行多标签分类。数据划分遵循标准训练-验证-评估流程，确保各子集规模均衡且任务定义清晰。
特点
该数据集最显著的特点在于其多任务覆盖与精细化的标签体系。与单一任务的语料库不同，scirepeval整合了从微观的引用计数回归到宏观的学科分类、从同作者识别到同行评议评分等多样化的学术信息抽取挑战。每个子任务均包含结构化的特征字段，如论文的语料库唯一标识、发表年份、被引次数等数值型与文本型属性。此外，数据集特别引入了归一化指标（如norm_year、scaled_year），为模型训练提供了尺度统一的特征空间，从而支持跨任务、跨领域的迁移学习研究。
使用方法
使用此数据集时，研究者可通过HuggingFace Datasets库按需加载特定子配置。例如，调用load_dataset('allenai/scirepeval', 'cite_count')即可获取包含论文标题、摘要及引用计数的回归任务数据。对于检索类任务（如search、nfcorpus），数据以查询文档对形式组织，并附有候选列表及相关性评分。模型训练可依托train与validation划分，评估则使用evaluation集。建议根据具体任务选择对应配置，并利用数据集中提供的归一化特征（如log_citations）进行预处理，以提升模型训练的稳定性与收敛效率。
背景与挑战
背景概述
在科学文献计量与自然语言处理交叉领域，学术文本的语义表征与信息检索任务始终是研究焦点。由艾伦人工智能研究所（Allen Institute for AI）主导构建的SciRepEval数据集，诞生于2023年前后，旨在系统性地评估科学文献表示的泛化能力。该数据集整合了超过20个子任务，涵盖文献分类、引用预测、作者匹配、主题检索等多元维度，其核心研究问题在于探索统一的科学文本嵌入模型能否在不同下游任务间实现高效迁移。凭借其大规模、多任务、跨领域的特性，SciRepEval已成为验证科学NLP模型鲁棒性的重要基准，对推动学术搜索引擎、科研影响力分析及同行评议自动化等应用产生了深远影响。
当前挑战
SciRepEval所面临的挑战首先体现在领域问题的复杂性上：科学文献的语义理解需同时应对多标签分类、长尾知识分布及跨学科概念映射，例如在FOS（研究领域分类）与MeSH描述符预测任务中，标签空间的高度稀疏性与层级性对模型泛化构成严峻考验。构建过程中，数据采集与标注的异质性成为另一核心难题——来自不同来源的文献需统一处理摘要、作者、引用网络等异构信息，同时保证跨任务（如引用预测与论文-审稿人匹配）的标注一致性。此外，大规模数据集（如cite_prediction_new含超600万样本）的存储与高效加载，以及部分子任务（如trec_covid仅50条评估样本）的极低资源场景，进一步加剧了模型评估的偏差控制与计算效率挑战。
常用场景
经典使用场景
在科学文献计量学与自然语言处理交叉领域，SciRepEval数据集被广泛用于评估学术文本表示模型的泛化能力。其经典使用场景涵盖引用预测、论文发表年份回归、学科领域分类（FOS）以及作者身份识别等多元任务。研究者借助该数据集中的cite_prediction、pub_year、fos等子集，可系统性地测试模型在捕捉学术论文语义关联、时间演化特征及学科归属上的表现。这种多任务评估框架不仅揭示了模型在不同学术信息检索场景下的鲁棒性，也为设计更贴合科研需求的嵌入方法提供了基准参考。
解决学术问题
该数据集核心解决了学术文本表示学习中缺乏统一、多维度评估标准的问题。传统基准往往聚焦单一任务，难以全面反映模型在真实科研场景中的效能。SciRepEval通过整合引用预测、高影响力论文识别、Mesh主题词标注等任务，使研究者能够在同一框架下对比不同模型对学术知识结构、引用动力学及领域细粒度语义的编码能力。其意义在于推动了从单一指标到综合评测的范式转变，为理解学术文本的深层语义、跨学科关联及影响力传播机制提供了量化实验平台。
衍生相关工作
基于SciRepEval数据集，学术界衍生了一系列具有影响力的工作。例如，SPECTER模型及其后续版本利用该数据集的citation prediction任务进行训练，提出了能够捕捉论文间引用关系的强大嵌入表示。在信息检索领域，研究者借鉴其search子集构建了针对科学文档的稠密检索模型。此外，该数据集还被用于验证对比学习、图神经网络等先进技术在学术文本分析中的有效性，催生了诸如SciNCL等专注于科学文献对比表示学习的框架。这些衍生工作共同丰富了科学文献挖掘的方法论体系。
以上内容由遇见数据集搜集并总结生成
关于我们
遇见数据集——让每个数据集都被发现，让每一次遇见都有价值。
数发科官网www.sfktec.com
联系我们
youjian
selectdataset@iotsh.com.cn
商业合作
数据驱动未来，携手共赢发展
二维码
社区交流群
二维码
科研交流群
商业服务