open-llm-leaderboard-old/details_Fredithefish__ScarletPajama-3B-HF

--- pretty_name: Evaluation run of Fredithefish/ScarletPajama-3B-HF dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [Fredithefish/ScarletPajama-3B-HF](https://huggingface.co/Fredithefish/ScarletPajama-3B-HF)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 64 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 4 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_Fredithefish__ScarletPajama-3B-HF\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2023-10-17T04:53:29.822366](https://huggingface.co/datasets/open-llm-leaderboard/details_Fredithefish__ScarletPajama-3B-HF/blob/main/results_2023-10-17T04-53-29.822366.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.004404362416107382,\n\ \ \"em_stderr\": 0.000678145162047974,\n \"f1\": 0.05973783557047,\n\ \ \"f1_stderr\": 0.00145869394982755,\n \"acc\": 0.3235523790774504,\n\ \ \"acc_stderr\": 0.00738110264721833\n },\n \"harness|drop|3\": {\n\ \ \"em\": 0.004404362416107382,\n \"em_stderr\": 0.000678145162047974,\n\ \ \"f1\": 0.05973783557047,\n \"f1_stderr\": 0.00145869394982755\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.002274450341167551,\n \ \ \"acc_stderr\": 0.0013121578148674066\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.6448303078137332,\n \"acc_stderr\": 0.013450047479569254\n\ \ }\n}\n```" repo_url: https://huggingface.co/Fredithefish/ScarletPajama-3B-HF leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|arc:challenge|25_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|arc:challenge|25_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|arc:challenge|25_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-07-18T11:22:36.276384.parquet' - config_name: harness_drop_3 data_files: - split: 2023_10_17T04_53_29.822366 path: - '**/details_harness|drop|3_2023-10-17T04-53-29.822366.parquet' - split: latest path: - '**/details_harness|drop|3_2023-10-17T04-53-29.822366.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2023_10_17T04_53_29.822366 path: - '**/details_harness|gsm8k|5_2023-10-17T04-53-29.822366.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2023-10-17T04-53-29.822366.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hellaswag|10_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hellaswag|10_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hellaswag|10_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-management|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-07-18T10:40:07.998848.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-management|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-07-18T10:59:29.744691.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-management|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-management|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-07-18T11:22:36.276384.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-management|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-management|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-management|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-virology|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-virology|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-virology|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-07-18T11:22:36.276384.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_07_18T10_40_07.998848 path: - '**/details_harness|truthfulqa:mc|0_2023-07-18T10:40:07.998848.parquet' - split: 2023_07_18T10_59_29.744691 path: - '**/details_harness|truthfulqa:mc|0_2023-07-18T10:59:29.744691.parquet' - split: 2023_07_18T11_22_36.276384 path: - '**/details_harness|truthfulqa:mc|0_2023-07-18T11:22:36.276384.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-07-18T11:22:36.276384.parquet' - config_name: harness_winogrande_5 data_files: - split: 2023_10_17T04_53_29.822366 path: - '**/details_harness|winogrande|5_2023-10-17T04-53-29.822366.parquet' - split: latest path: - '**/details_harness|winogrande|5_2023-10-17T04-53-29.822366.parquet' - config_name: results data_files: - split: 2023_07_18T10_40_07.998848 path: - results_2023-07-18T10:40:07.998848.parquet - split: 2023_07_18T10_59_29.744691 path: - results_2023-07-18T10:59:29.744691.parquet - split: 2023_07_18T11_22_36.276384 path: - results_2023-07-18T11:22:36.276384.parquet - split: 2023_10_17T04_53_29.822366 path: - results_2023-10-17T04-53-29.822366.parquet - split: latest path: - results_2023-10-17T04-53-29.822366.parquet --- # Dataset Card for Evaluation run of Fredithefish/ScarletPajama-3B-HF ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/Fredithefish/ScarletPajama-3B-HF - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [Fredithefish/ScarletPajama-3B-HF](https://huggingface.co/Fredithefish/ScarletPajama-3B-HF) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 4 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_Fredithefish__ScarletPajama-3B-HF", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2023-10-17T04:53:29.822366](https://huggingface.co/datasets/open-llm-leaderboard/details_Fredithefish__ScarletPajama-3B-HF/blob/main/results_2023-10-17T04-53-29.822366.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "em": 0.004404362416107382, "em_stderr": 0.000678145162047974, "f1": 0.05973783557047, "f1_stderr": 0.00145869394982755, "acc": 0.3235523790774504, "acc_stderr": 0.00738110264721833 }, "harness|drop|3": { "em": 0.004404362416107382, "em_stderr": 0.000678145162047974, "f1": 0.05973783557047, "f1_stderr": 0.00145869394982755 }, "harness|gsm8k|5": { "acc": 0.002274450341167551, "acc_stderr": 0.0013121578148674066 }, "harness|winogrande|5": { "acc": 0.6448303078137332, "acc_stderr": 0.013450047479569254 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型 Fredithefish/ScarletPajama-3B-HF 在 Open LLM Leaderboard 上的自动创建的。数据集包含 64 个配置，每个配置对应一个评估任务。

数据集结构

数据集由 4 次运行创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

额外配置

一个额外的配置 "results" 存储所有运行的聚合结果，用于计算并在 Open LLM Leaderboard 上显示聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_Fredithefish__ScarletPajama-3B-HF", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-17T04:53:29.822366 运行的最新结果： python { "all": { "em": 0.004404362416107382, "em_stderr": 0.000678145162047974, "f1": 0.05973783557047, "f1_stderr": 0.00145869394982755, "acc": 0.3235523790774504, "acc_stderr": 0.00738110264721833 }, "harness|drop|3": { "em": 0.004404362416107382, "em_stderr": 0.000678145162047974, "f1": 0.05973783557047, "f1_stderr": 0.00145869394982755 }, "harness|gsm8k|5": { "acc": 0.002274450341167551, "acc_stderr": 0.0013121578148674066 }, "harness|winogrande|5": { "acc": 0.6448303078137332, "acc_stderr": 0.013450047479569254 } }

配置详情

• config_name: harness_arc_challenge_25

  • split: 2023_07_18T10_40_07.998848
    • path: **/details_harness|arc:challenge|25_2023-07-18T10:40:07.998848.parquet
  • split: 2023_07_18T10_59_29.744691
    • path: **/details_harness|arc:challenge|25_2023-07-18T10:59:29.744691.parquet
  • split: 2023_07_18T11_22_36.276384
    • path: **/details_harness|arc:challenge|25_2023-07-18T11:22:36.276384.parquet
  • split: latest
    • path: **/details_harness|arc:challenge|25_2023-07-18T11:22:36.276384.parquet

• config_name: harness_drop_3

  • split: 2023_10_17T04_53_29.822366
    • path: **/details_harness|drop|3_2023-10-17T04-53-29.822366.parquet
  • split: latest
    • path: **/details_harness|drop|3_2023-10-17T04-53-29.822366.parquet

• config_name: harness_gsm8k_5

  • split: 2023_10_17T04_53_29.822366
    • path: **/details_harness|gsm8k|5_2023-10-17T04-53-29.822366.parquet
  • split: latest
    • path: **/details_harness|gsm8k|5_2023-10-17T04-53-29.822366.parquet

• config_name: harness_hellaswag_10

  • split: 2023_07_18T10_40_07.998848
    • path: **/details_harness|hellaswag|10_2023-07-18T10:40:07.998848.parquet
  • split: 2023_07_18T10_59_29.744691
    • path: **/details_harness|hellaswag|10_2023-07-18T10:59:29.744691.parquet
  • split: 2023_07_18T11_22_36.276384
    • path: **/details_harness|hellaswag|10_2023-07-18T11:22:36.276384.parquet
  • split: latest
    • path: **/details_harness|hellaswag|10_2023-07-18T11:22:36.276384.parquet

• config_name: harness_hendrycksTest_5

  • split: 2023_07_18T10_40_07.998848
    • path: **/details_harness|hendrycksTest-abstract_algebra|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-anatomy|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-astronomy|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-business_ethics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-clinical_knowledge|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_biology|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_chemistry|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_computer_science|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_mathematics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_medicine|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-college_physics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-computer_security|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-conceptual_physics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-econometrics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-electrical_engineering|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-elementary_mathematics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-formal_logic|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-global_facts|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_biology|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_chemistry|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_computer_science|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_european_history|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_geography|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_mathematics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_physics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_psychology|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_statistics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_us_history|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-high_school_world_history|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-human_aging|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-human_sexuality|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-international_law|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-jurisprudence|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-logical_fallacies|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-machine_learning|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-management|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-marketing|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-medical_genetics|5_2023-07-18T10:40:07.998848.parquet
    • path: **/details_harness|hendrycksTest-miscellaneous|5_2023-0

搜集汇总

数据集介绍

构建方式

该数据集是在Open LLM Leaderboard评测框架下，对模型Fredithefish/ScarletPajama-3B-HF进行自动化评估过程中生成的。数据集共包含64个配置，每个配置对应一项评测任务，这些任务涵盖ARC Challenge、DROP、GSM8K、HellaSwag、MMLU（Hendrycks Test）等多个基准。数据集的构建基于4次独立的运行，每次运行的结果以时间戳命名的分割（split）形式存储在相应配置中，而“train”分割则始终指向最新一次运行的结果。此外，数据集还包含一个名为“results”的配置，用于存储所有运行的聚合指标，这些指标被用于在Open LLM Leaderboard上计算和展示模型的综合表现。

特点

该数据集的一个显著特点在于其结构化的多任务覆盖能力，它系统性地记录了模型在数十种不同自然语言理解与推理任务上的细粒度表现。每个评测任务作为一个独立的配置，便于研究者针对性地分析模型在特定领域的优劣。数据集通过时间戳分割保留了历史运行记录，使得模型的性能演变轨迹得以追溯，为模型迭代提供了宝贵的对比依据。同时，“results”配置中的聚合指标（如准确率、精确匹配率、F1分数）为快速评估模型整体水平提供了便捷的概览。这种设计既保证了数据的完整性，也兼顾了分析灵活性。

使用方法

使用该数据集时，研究者可通过Hugging Face的datasets库进行加载。例如，加载特定任务（如Winogrande）的最新结果，只需调用load_dataset函数，指定数据集名称与相应配置（如“harness_winogrande_5”），并选择分割为“train”即可。若需访问历史运行数据，则可将分割参数指定为对应的时间戳字符串。对于聚合结果的获取，可直接加载“results”配置。这种加载方式支持对模型在各项任务上的详细输出进行深入分析，包括每个样本的预测与真实标签，从而为模型诊断、误差分析及性能比较提供坚实的数据基础。

背景与挑战

背景概述

该数据集诞生于大语言模型评估体系蓬勃发展的2023年，由Hugging Face团队在Open LLM Leaderboard框架下创建，核心研究人员包括Clémentine Fourrier等。其核心研究问题在于系统性地评估Fredithefish/ScarletPajama-3B-HF这一3B参数模型在多项自然语言理解与推理任务上的表现，涵盖ARC挑战集、HellaSwag常识推理、GSM8K数学推理、Winogrande代词消歧以及MMLU多学科知识等基准测试。作为Open LLM Leaderboard生态的重要组成部分，该数据集通过标准化流程记录模型在64个配置下的详细评测结果，为社区提供了可复现的评估基准，推动了开源大模型性能比较的透明化与规范化，对促进大语言模型的可信评估具有里程碑意义。

当前挑战

当前数据集面临的核心挑战包括：1) 领域问题层面，大语言模型评估存在任务维度碎片化与指标单一化困境，如GSM8K上仅0.23%的准确率暴露出模型在数学推理上的显著短板，而Winogrande上64.48%的准确率则显示常识推理的相对优势，这种能力不均衡性亟需更细粒度的诊断性评估框架；2) 构建过程中，评测结果的时效性与可复现性难以平衡，多次运行的时间戳分片设计虽便于追溯，但不同批次间的硬件环境与框架版本差异可能引入隐性偏差，且Parquet格式的存储结构对大规模多轮评测的检索效率构成挑战，同时如何确保leaderboard评分规则与社区最新研究进展同步更新也是一项持续考验。

常用场景

经典使用场景

在开放大语言模型评估的浪潮中，Open LLM Leaderboard 的评估数据集已成为衡量模型性能的标杆。该数据集专为模型 ScarletPajama-3B-HF 的自动化评估而构建，涵盖了 ARC、HellaSwag、MMLU、TruthfulQA 等经典基准任务。其核心用途在于提供细粒度的评估细节，包括每项任务在多次运行中的准确率、F1 分数等指标，使研究者能够深入剖析模型在常识推理、知识理解与数学求解等维度的表现。通过标准化评估流程，该数据集为大语言模型的能力对比与迭代优化提供了坚实的数据基础。

衍生相关工作

围绕该数据集衍生了一系列重要的学术工作。例如，HuggingFace 团队基于 Leaderboard 数据发布了多篇关于模型规模与性能关系的分析报告，揭示了参数数量与特定任务表现之间的非线性规律。同时，研究者利用该数据集对 ScarletPajama-3B-HF 等模型进行了深入剖析，推动了关于训练数据质量与模型泛化能力关联性的研究。这些工作不仅丰富了大语言模型评估的理论体系，也启发了后续如 LLaMA 系列模型的训练策略优化，形成了从评估到训练的正向循环。

数据集最近研究

最新研究方向

当前，大语言模型的性能评估已成为自然语言处理领域的前沿热点，而Open LLM Leaderboard作为业界公认的基准平台，其评估数据集的构建与分析方法正引领着模型透明化与可复现性的研究浪潮。针对Fredithefish/ScarletPajama-3B-HF模型的评估运行，该数据集不仅涵盖了ARC、HellaSwag、MMLU等多样化的推理与知识任务，还通过多轮次、多时间戳的精细化管理，为模型的动态性能追踪与历史对比提供了坚实的数据基础。这一研究方向的核心意义在于，它推动了从单一静态评分向多维、时序化评估范式的转变，使得研究者能够深入洞察模型在不同能力维度上的演化轨迹与潜在瓶颈。结合近期社区对模型公平性、鲁棒性及泛化能力的广泛关注，此类细粒度评估数据集的出现，无疑为构建更可靠、更透明的大语言模型评价体系提供了关键支撑，也预示着未来模型评估将朝着更加标准化、动态化与可解释性的方向迈进。