open-llm-leaderboard/details_EleutherAI__pythia-2.7b
收藏Hugging Face2023-10-13 更新2024-03-04 收录
下载链接：
https://hf-mirror.com/datasets/open-llm-leaderboard/details_EleutherAI__pythia-2.7b
下载链接
链接失效反馈官方服务：
资源简介：
---
pretty_name: Evaluation run of EleutherAI/pythia-2.7b
dataset_summary: "Dataset automatically created during the evaluation run of model\
\ [EleutherAI/pythia-2.7b](https://huggingface.co/EleutherAI/pythia-2.7b) on the\
\ [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\
\nThe dataset is composed of 64 configuration, each one coresponding to one of the\
\ evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can be\
\ found as a specific split in each configuration, the split being named using the\
\ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\
\nAn additional configuration \"results\" store all the aggregated results of the\
\ run (and is used to compute and display the agregated metrics on the [Open LLM\
\ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\
\nTo load the details from a run, you can for instance do the following:\n```python\n\
from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_EleutherAI__pythia-2.7b\"\
,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\
These are the [latest results from run 2023-10-13T09:56:45.840530](https://huggingface.co/datasets/open-llm-leaderboard/details_EleutherAI__pythia-2.7b/blob/main/results_2023-10-13T09-56-45.840530.json)(note\
\ that their might be results for other tasks in the repos if successive evals didn't\
\ cover the same tasks. You find each in the results and the \"latest\" split for\
\ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.0010486577181208054,\n\
\ \"em_stderr\": 0.0003314581465219123,\n \"f1\": 0.045142617449664524,\n\
\ \"f1_stderr\": 0.0011715787583837264,\n \"acc\": 0.315883214963382,\n\
\ \"acc_stderr\": 0.008228218962784018\n },\n \"harness|drop|3\": {\n\
\ \"em\": 0.0010486577181208054,\n \"em_stderr\": 0.0003314581465219123,\n\
\ \"f1\": 0.045142617449664524,\n \"f1_stderr\": 0.0011715787583837264\n\
\ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.01061410159211524,\n \
\ \"acc_stderr\": 0.002822713322387704\n },\n \"harness|winogrande|5\"\
: {\n \"acc\": 0.6211523283346487,\n \"acc_stderr\": 0.013633724603180334\n\
\ }\n}\n```"
repo_url: https://huggingface.co/EleutherAI/pythia-2.7b
leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
point_of_contact: clementine@hf.co
configs:
- config_name: harness_arc_challenge_25
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|arc:challenge|25_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|arc:challenge|25_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_drop_3
data_files:
- split: 2023_10_13T09_56_45.840530
path:
- '**/details_harness|drop|3_2023-10-13T09-56-45.840530.parquet'
- split: latest
path:
- '**/details_harness|drop|3_2023-10-13T09-56-45.840530.parquet'
- config_name: harness_gsm8k_5
data_files:
- split: 2023_10_13T09_56_45.840530
path:
- '**/details_harness|gsm8k|5_2023-10-13T09-56-45.840530.parquet'
- split: latest
path:
- '**/details_harness|gsm8k|5_2023-10-13T09-56-45.840530.parquet'
- config_name: harness_hellaswag_10
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hellaswag|10_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hellaswag|10_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-management|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-virology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-management|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-virology|5_2023-07-19T16:50:21.612353.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_abstract_algebra_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_anatomy_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-anatomy|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-anatomy|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_astronomy_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-astronomy|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-astronomy|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_business_ethics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_clinical_knowledge_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_biology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_biology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_biology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_chemistry_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_computer_science_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_mathematics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_medicine_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_college_physics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-college_physics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_physics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_computer_security_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-computer_security|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-computer_security|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_conceptual_physics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_econometrics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-econometrics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-econometrics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_electrical_engineering_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_elementary_mathematics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_formal_logic_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_global_facts_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-global_facts|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-global_facts|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_biology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_chemistry_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_computer_science_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_european_history_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_geography_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_government_and_politics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_macroeconomics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_mathematics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_microeconomics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_physics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_psychology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_statistics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_us_history_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_high_school_world_history_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_human_aging_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-human_aging|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_aging|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_human_sexuality_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_international_law_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-international_law|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-international_law|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_jurisprudence_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_logical_fallacies_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_machine_learning_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_management_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-management|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-management|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_marketing_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-marketing|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-marketing|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_medical_genetics_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_miscellaneous_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_moral_disputes_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_moral_scenarios_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_nutrition_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-nutrition|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-nutrition|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_philosophy_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-philosophy|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-philosophy|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_prehistory_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-prehistory|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-prehistory|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_professional_accounting_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_professional_law_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-professional_law|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_law|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_professional_medicine_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_professional_psychology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_public_relations_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-public_relations|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-public_relations|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_security_studies_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-security_studies|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-security_studies|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_sociology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-sociology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-sociology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_us_foreign_policy_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_virology_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-virology|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-virology|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_hendrycksTest_world_religions_5
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|hendrycksTest-world_religions|5_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-world_religions|5_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_truthfulqa_mc_0
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- '**/details_harness|truthfulqa:mc|0_2023-07-19T16:50:21.612353.parquet'
- split: latest
path:
- '**/details_harness|truthfulqa:mc|0_2023-07-19T16:50:21.612353.parquet'
- config_name: harness_winogrande_5
data_files:
- split: 2023_10_13T09_56_45.840530
path:
- '**/details_harness|winogrande|5_2023-10-13T09-56-45.840530.parquet'
- split: latest
path:
- '**/details_harness|winogrande|5_2023-10-13T09-56-45.840530.parquet'
- config_name: results
data_files:
- split: 2023_07_19T16_50_21.612353
path:
- results_2023-07-19T16:50:21.612353.parquet
- split: 2023_10_13T09_56_45.840530
path:
- results_2023-10-13T09-56-45.840530.parquet
- split: latest
path:
- results_2023-10-13T09-56-45.840530.parquet
---
# Dataset Card for Evaluation run of EleutherAI/pythia-2.7b
## Dataset Description
- **Homepage:**
- **Repository:** https://huggingface.co/EleutherAI/pythia-2.7b
- **Paper:**
- **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
- **Point of Contact:** clementine@hf.co
### Dataset Summary
Dataset automatically created during the evaluation run of model [EleutherAI/pythia-2.7b](https://huggingface.co/EleutherAI/pythia-2.7b) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).
To load the details from a run, you can for instance do the following:
```python
from datasets import load_dataset
data = load_dataset("open-llm-leaderboard/details_EleutherAI__pythia-2.7b",
"harness_winogrande_5",
split="train")
```
## Latest results
These are the [latest results from run 2023-10-13T09:56:45.840530](https://huggingface.co/datasets/open-llm-leaderboard/details_EleutherAI__pythia-2.7b/blob/main/results_2023-10-13T09-56-45.840530.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval):
```python
{
"all": {
"em": 0.0010486577181208054,
"em_stderr": 0.0003314581465219123,
"f1": 0.045142617449664524,
"f1_stderr": 0.0011715787583837264,
"acc": 0.315883214963382,
"acc_stderr": 0.008228218962784018
},
"harness|drop|3": {
"em": 0.0010486577181208054,
"em_stderr": 0.0003314581465219123,
"f1": 0.045142617449664524,
"f1_stderr": 0.0011715787583837264
},
"harness|gsm8k|5": {
"acc": 0.01061410159211524,
"acc_stderr": 0.002822713322387704
},
"harness|winogrande|5": {
"acc": 0.6211523283346487,
"acc_stderr": 0.013633724603180334
}
}
```
### Supported Tasks and Leaderboards
[More Information Needed]
### Languages
[More Information Needed]
## Dataset Structure
### Data Instances
[More Information Needed]
### Data Fields
[More Information Needed]
### Data Splits
[More Information Needed]
## Dataset Creation
### Curation Rationale
[More Information Needed]
### Source Data
#### Initial Data Collection and Normalization
[More Information Needed]
#### Who are the source language producers?
[More Information Needed]
### Annotations
#### Annotation process
[More Information Needed]
#### Who are the annotators?
[More Information Needed]
### Personal and Sensitive Information
[More Information Needed]
## Considerations for Using the Data
### Social Impact of Dataset
[More Information Needed]
### Discussion of Biases
[More Information Needed]
### Other Known Limitations
[More Information Needed]
## Additional Information
### Dataset Curators
[More Information Needed]
### Licensing Information
[More Information Needed]
### Citation Information
[More Information Needed]
### Contributions
[More Information Needed]
提供机构：
open-llm-leaderboard原始信息汇总
数据集概述
数据集是在模型 EleutherAI/pythia-2.7b 在 Open LLM Leaderboard 上的评估运行期间自动创建的。
数据集组成
- 数据集包含 64 个配置，每个配置对应一个评估任务。
- 数据集从 2 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
- "train" 分割始终指向最新的结果。
- 额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。
数据加载示例
python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_EleutherAI__pythia-2.7b", "harness_winogrande_5", split="train")
最新结果
以下是 2023-10-13T09:56:45.840530 运行 的最新结果：
python { "all": { "em": 0.0010486577181208054, "em_stderr": 0.0003314581465219123, "f1": 0.045142617449664524, "f1_stderr": 0.0011715787583837264, "acc": 0.315883214963382, "acc_stderr": 0.008228218962784018 }, "harness|drop|3": { "em": 0.0010486577181208054, "em_stderr": 0.0003314581465219123, "f1": 0.045142617449664524, "f1_stderr": 0.0011715787583837264 }, "harness|gsm8k|5": { "acc": 0.01061410159211524, "acc_stderr": 0.002822713322387704 }, "harness|winogrande|5": { "acc": 0.6211523283346487, "acc_stderr": 0.013633724603180334 } }
配置详情
-
harness_arc_challenge_25
- 分割：2023_07_19T16_50_21.612353
- 路径：
**/details_harness|arc:challenge|25_2023-07-19T16:50:21.612353.parquet - 分割：latest
- 路径：
**/details_harness|arc:challenge|25_2023-07-19T16:50:21.612353.parquet
-
harness_drop_3
- 分割：2023_10_13T09_56_45.840530
- 路径：
**/details_harness|drop|3_2023-10-13T09-56-45.840530.parquet - 分割：latest
- 路径：
**/details_harness|drop|3_2023-10-13T09-56-45.840530.parquet
-
harness_gsm8k_5
- 分割：2023_10_13T09_56_45.840530
- 路径：
**/details_harness|gsm8k|5_2023-10-13T09-56-45.840530.parquet - 分割：latest
- 路径：
**/details_harness|gsm8k|5_2023-10-13T09-56-45.840530.parquet
-
harness_hellaswag_10
- 分割：2023_07_19T16_50_21.612353
- 路径：
**/details_harness|hellaswag|10_2023-07-19T16:50:21.612353.parquet - 分割：latest
- 路径：
**/details_harness|hellaswag|10_2023-07-19T16:50:21.612353.parquet
-
harness_hendrycksTest_5
- 分割：2023_07_19T16_50_21.612353
- 路径：
**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-anatomy|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-astronomy|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-business_ethics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_biology|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_chemistry|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_computer_science|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_mathematics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_medicine|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-college_physics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-computer_security|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-econometrics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-formal_logic|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-global_facts|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_biology|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_geography|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_physics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-human_aging|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-human_sexuality|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-international_law|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-jurisprudence|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-machine_learning|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-management|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-marketing|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-medical_genetics|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-miscellaneous|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-moral_disputes|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-nutrition|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-philosophy|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-prehistory|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-professional_accounting|5_2023-07-19T16:50:21.612353.parquet**/details_harness|hendrycksTest-professional_law|5_2023-07-19T16:50:21.612353.parquet- `**/details_harness|hendrycksTest-professional_medicine|5
搜集汇总
数据集介绍
构建方式
在大规模语言模型评估的浪潮中，Open LLM Leaderboard为模型性能的横向对比提供了标准化平台。该数据集正是针对EleutherAI团队开发的pythia-2.7b模型在排行榜上的评估过程而自动生成。其构建方式围绕64个不同的评测任务配置展开，每个配置对应一个具体的评估基准。数据源自两次独立的评估运行，每次运行的结果以时间戳命名，作为特定分割存储在对应配置中。此外，系统自动维护一个名为“train”的分割，始终指向最新一次运行的评估结果。一个名为“results”的额外配置汇总了所有运行的聚合指标，为排行榜上最终得分的计算与展示提供支撑。
特点
该数据集的核心特点在于其结构化的多任务评估体系。它囊括了从常识推理（如Winogrande）到数学推理（如GSM8K）再到阅读理解（如DROP）等丰富多样的任务，能够全面反映模型在不同能力维度上的表现。每个任务配置内包含多个时间戳分割，使得研究者可以追踪模型性能随评估轮次的变化趋势。数据以Parquet格式存储，兼顾了高效读写与跨平台兼容性。最新的运行结果提供了如准确率、F1分数等关键指标及其标准误差，为模型间的精细比较提供了量化依据。
使用方法
研究者可通过Hugging Face的datasets库便捷地加载与使用该数据集。具体而言，调用load_dataset函数并指定数据集名称与目标任务配置，即可获取对应评估任务的详细结果。例如，通过设置参数“harness_winogrande_5”并选择“train”分割，用户能直接访问Winogrande任务的最新评估数据。每个配置下的“latest”分割始终指向最新结果，便于快速获取最新性能指标。对于需要历史对比的分析，则可利用时间戳命名的分割加载特定运行的数据。这种设计使得对模型进行纵向追踪与横向对比都变得直观而高效。
背景与挑战
背景概述
在大型语言模型（LLM）迅猛发展的浪潮中，如何系统、公正地评估模型的多维度能力，成为推动该领域前行的关键瓶颈。为此，Hugging Face社区于2023年推出了Open LLM Leaderboard，旨在构建一个开放、透明的模型性能竞技场。由Clémentine Fourrier主导的该数据集，专门记录了EleutherAI团队开发的Pythia-2.7B模型在排行榜上的评估运行细节。Pythia系列模型以其可复现、规模递进的特性闻名，而此评估数据集则聚焦于2.7B参数版本在ARC、HellaSwag、MMLU等数十项基准任务上的表现，涵盖了从常识推理到数学求解的广泛能力。该数据集不仅为研究者提供了细粒度的模型性能快照，更通过自动化的评估流程，为LLM的横向对比设立了标准化参照，对理解模型规模与能力之间的复杂关系产生了深远影响。
当前挑战
该数据集所面临的核心挑战之一，源于LLM评估本身的领域复杂性。传统基准如MMLU或GSM8K虽能部分反映模型的知识广度与推理能力，却难以捕捉模型在开放域对话、长文本理解或指令遵循等新兴诉求上的真实水平，导致评估结果与实用价值间存在鸿沟。此外，数据集的构建过程亦充满技术挑战：如何确保64个不同配置下的任务评估在时间戳不同的运行间保持一致性，避免因环境差异或模型版本迭代引入的偏差，是数据质量的关键。同时，面对LLM快速演进的现状，该数据集需不断纳入新任务以保持时效性，但新基准的标准化接入与历史数据的兼容性维护，构成了资源与工程上的双重考验。
常用场景
经典使用场景
在大型语言模型评估领域，open-llm-leaderboard/details_EleutherAI__pythia-2.7b 数据集为研究者提供了对 EleutherAI/pythia-2.7b 模型进行标准化性能评测的完整记录。该数据集涵盖了 64 个不同的任务配置，包括 ARC-Challenge、HellaSwag、MMLU 等广泛使用的基准测试，每个任务都通过统一的评估框架（如 Language Model Evaluation Harness）进行测试，并存储了详细的评分结果。这种结构化的评估数据使得研究者能够深入分析模型在不同认知维度上的表现，如常识推理、数学解题和阅读理解。经典用法包括加载特定任务的评测结果，例如使用 datasets 库获取 winogrande 任务的准确性指标，从而对比模型在代词消歧任务中的能力。
衍生相关工作
该数据集衍生了一系列重要的后续工作，特别是在模型评估基准和排行榜研究方面。基于 Open LLM Leaderboard 的框架，研究者开发了针对不同语言和领域的扩展评估套件，例如多语言推理基准和多模态理解测试。此外，该数据集的细粒度结果（如每个测试样本的准确率）被用于分析模型的行为一致性，催生了关于模型鲁棒性和公平性的研究。例如，有工作利用这些数据构建了模型能力图谱，揭示 pythia-2.7b 在特定学科（如法律和医学）上的知识盲区，从而推动了知识增强型模型的发展。这些衍生工作进一步巩固了标准化评估在语言模型研究中的核心地位。
数据集最近研究
最新研究方向
在大规模语言模型评测领域，Open LLM Leaderboard已成为衡量模型综合能力的重要基准平台。围绕EleutherAI/pythia-2.7b这一参数规模的模型，最新研究方向聚焦于在多样化任务上的细粒度性能剖析，涵盖从常识推理（如Winogrande）到数学求解（如GSM8K）再到阅读理解（如DROP）等维度。该数据集通过标准化评测流程，揭示了中等规模模型在复杂推理任务上仍存在显著瓶颈，例如在DROP任务中F1值仅达0.045，而在Winogrande上准确率却达到0.621，这种任务间能力的显著分化正成为当前研究的热点。这一发现不仅为模型架构优化指明了方向，也推动了针对特定能力短板的数据增强与训练策略革新，对构建更均衡、更强健的语言智能具有深远影响。
以上内容由遇见数据集搜集并总结生成
