open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO

--- pretty_name: Evaluation run of 0-hero/Matter-0.1-7B-boost-DPO dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [0-hero/Matter-0.1-7B-boost-DPO](https://huggingface.co/0-hero/Matter-0.1-7B-boost-DPO)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 63 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 1 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the aggregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2024-03-22T17:52:36.450234](https://huggingface.co/datasets/open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO/blob/main/results_2024-03-22T17-52-36.450234.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"acc\": 0.6196341771509315,\n\ \ \"acc_stderr\": 0.03280520004973222,\n \"acc_norm\": 0.6228652576735157,\n\ \ \"acc_norm_stderr\": 0.033458484779768545,\n \"mc1\": 0.43451652386780903,\n\ \ \"mc1_stderr\": 0.017352738749259564,\n \"mc2\": 0.6029363233989143,\n\ \ \"mc2_stderr\": 0.015526892800807802\n },\n \"harness|arc:challenge|25\"\ : {\n \"acc\": 0.6271331058020477,\n \"acc_stderr\": 0.01413117676013117,\n\ \ \"acc_norm\": 0.6501706484641638,\n \"acc_norm_stderr\": 0.013936809212158298\n\ \ },\n \"harness|hellaswag|10\": {\n \"acc\": 0.6459868552081258,\n\ \ \"acc_stderr\": 0.0047723583951304474,\n \"acc_norm\": 0.8308105954989046,\n\ \ \"acc_norm_stderr\": 0.0037415289563158434\n },\n \"harness|hendrycksTest-abstract_algebra|5\"\ : {\n \"acc\": 0.27,\n \"acc_stderr\": 0.044619604333847394,\n \ \ \"acc_norm\": 0.27,\n \"acc_norm_stderr\": 0.044619604333847394\n \ \ },\n \"harness|hendrycksTest-anatomy|5\": {\n \"acc\": 0.5925925925925926,\n\ \ \"acc_stderr\": 0.04244633238353227,\n \"acc_norm\": 0.5925925925925926,\n\ \ \"acc_norm_stderr\": 0.04244633238353227\n },\n \"harness|hendrycksTest-astronomy|5\"\ : {\n \"acc\": 0.6776315789473685,\n \"acc_stderr\": 0.03803510248351585,\n\ \ \"acc_norm\": 0.6776315789473685,\n \"acc_norm_stderr\": 0.03803510248351585\n\ \ },\n \"harness|hendrycksTest-business_ethics|5\": {\n \"acc\": 0.55,\n\ \ \"acc_stderr\": 0.049999999999999996,\n \"acc_norm\": 0.55,\n \ \ \"acc_norm_stderr\": 0.049999999999999996\n },\n \"harness|hendrycksTest-clinical_knowledge|5\"\ : {\n \"acc\": 0.6754716981132075,\n \"acc_stderr\": 0.028815615713432115,\n\ \ \"acc_norm\": 0.6754716981132075,\n \"acc_norm_stderr\": 0.028815615713432115\n\ \ },\n \"harness|hendrycksTest-college_biology|5\": {\n \"acc\": 0.6875,\n\ \ \"acc_stderr\": 0.038760854559127644,\n \"acc_norm\": 0.6875,\n\ \ \"acc_norm_stderr\": 0.038760854559127644\n },\n \"harness|hendrycksTest-college_chemistry|5\"\ : {\n \"acc\": 0.42,\n \"acc_stderr\": 0.049604496374885836,\n \ \ \"acc_norm\": 0.42,\n \"acc_norm_stderr\": 0.049604496374885836\n \ \ },\n \"harness|hendrycksTest-college_computer_science|5\": {\n \"\ acc\": 0.5,\n \"acc_stderr\": 0.050251890762960605,\n \"acc_norm\"\ : 0.5,\n \"acc_norm_stderr\": 0.050251890762960605\n },\n \"harness|hendrycksTest-college_mathematics|5\"\ : {\n \"acc\": 0.35,\n \"acc_stderr\": 0.0479372485441102,\n \ \ \"acc_norm\": 0.35,\n \"acc_norm_stderr\": 0.0479372485441102\n },\n\ \ \"harness|hendrycksTest-college_medicine|5\": {\n \"acc\": 0.5838150289017341,\n\ \ \"acc_stderr\": 0.03758517775404947,\n \"acc_norm\": 0.5838150289017341,\n\ \ \"acc_norm_stderr\": 0.03758517775404947\n },\n \"harness|hendrycksTest-college_physics|5\"\ : {\n \"acc\": 0.35294117647058826,\n \"acc_stderr\": 0.04755129616062946,\n\ \ \"acc_norm\": 0.35294117647058826,\n \"acc_norm_stderr\": 0.04755129616062946\n\ \ },\n \"harness|hendrycksTest-computer_security|5\": {\n \"acc\":\ \ 0.75,\n \"acc_stderr\": 0.04351941398892446,\n \"acc_norm\": 0.75,\n\ \ \"acc_norm_stderr\": 0.04351941398892446\n },\n \"harness|hendrycksTest-conceptual_physics|5\"\ : {\n \"acc\": 0.5191489361702127,\n \"acc_stderr\": 0.032662042990646775,\n\ \ \"acc_norm\": 0.5191489361702127,\n \"acc_norm_stderr\": 0.032662042990646775\n\ \ },\n \"harness|hendrycksTest-econometrics|5\": {\n \"acc\": 0.4649122807017544,\n\ \ \"acc_stderr\": 0.04692008381368909,\n \"acc_norm\": 0.4649122807017544,\n\ \ \"acc_norm_stderr\": 0.04692008381368909\n },\n \"harness|hendrycksTest-electrical_engineering|5\"\ : {\n \"acc\": 0.5862068965517241,\n \"acc_stderr\": 0.04104269211806232,\n\ \ \"acc_norm\": 0.5862068965517241,\n \"acc_norm_stderr\": 0.04104269211806232\n\ \ },\n \"harness|hendrycksTest-elementary_mathematics|5\": {\n \"acc\"\ : 0.3439153439153439,\n \"acc_stderr\": 0.024464426625596437,\n \"\ acc_norm\": 0.3439153439153439,\n \"acc_norm_stderr\": 0.024464426625596437\n\ \ },\n \"harness|hendrycksTest-formal_logic|5\": {\n \"acc\": 0.38095238095238093,\n\ \ \"acc_stderr\": 0.043435254289490965,\n \"acc_norm\": 0.38095238095238093,\n\ \ \"acc_norm_stderr\": 0.043435254289490965\n },\n \"harness|hendrycksTest-global_facts|5\"\ : {\n \"acc\": 0.44,\n \"acc_stderr\": 0.04988876515698589,\n \ \ \"acc_norm\": 0.44,\n \"acc_norm_stderr\": 0.04988876515698589\n \ \ },\n \"harness|hendrycksTest-high_school_biology|5\": {\n \"acc\": 0.7516129032258064,\n\ \ \"acc_stderr\": 0.024580028921481,\n \"acc_norm\": 0.7516129032258064,\n\ \ \"acc_norm_stderr\": 0.024580028921481\n },\n \"harness|hendrycksTest-high_school_chemistry|5\"\ : {\n \"acc\": 0.5024630541871922,\n \"acc_stderr\": 0.035179450386910616,\n\ \ \"acc_norm\": 0.5024630541871922,\n \"acc_norm_stderr\": 0.035179450386910616\n\ \ },\n \"harness|hendrycksTest-high_school_computer_science|5\": {\n \ \ \"acc\": 0.67,\n \"acc_stderr\": 0.04725815626252609,\n \"acc_norm\"\ : 0.67,\n \"acc_norm_stderr\": 0.04725815626252609\n },\n \"harness|hendrycksTest-high_school_european_history|5\"\ : {\n \"acc\": 0.7636363636363637,\n \"acc_stderr\": 0.03317505930009181,\n\ \ \"acc_norm\": 0.7636363636363637,\n \"acc_norm_stderr\": 0.03317505930009181\n\ \ },\n \"harness|hendrycksTest-high_school_geography|5\": {\n \"acc\"\ : 0.7727272727272727,\n \"acc_stderr\": 0.029857515673386414,\n \"\ acc_norm\": 0.7727272727272727,\n \"acc_norm_stderr\": 0.029857515673386414\n\ \ },\n \"harness|hendrycksTest-high_school_government_and_politics|5\": {\n\ \ \"acc\": 0.8652849740932642,\n \"acc_stderr\": 0.02463978909770944,\n\ \ \"acc_norm\": 0.8652849740932642,\n \"acc_norm_stderr\": 0.02463978909770944\n\ \ },\n \"harness|hendrycksTest-high_school_macroeconomics|5\": {\n \ \ \"acc\": 0.6230769230769231,\n \"acc_stderr\": 0.024570975364225995,\n\ \ \"acc_norm\": 0.6230769230769231,\n \"acc_norm_stderr\": 0.024570975364225995\n\ \ },\n \"harness|hendrycksTest-high_school_mathematics|5\": {\n \"\ acc\": 0.3148148148148148,\n \"acc_stderr\": 0.028317533496066465,\n \ \ \"acc_norm\": 0.3148148148148148,\n \"acc_norm_stderr\": 0.028317533496066465\n\ \ },\n \"harness|hendrycksTest-high_school_microeconomics|5\": {\n \ \ \"acc\": 0.6554621848739496,\n \"acc_stderr\": 0.030868682604121622,\n\ \ \"acc_norm\": 0.6554621848739496,\n \"acc_norm_stderr\": 0.030868682604121622\n\ \ },\n \"harness|hendrycksTest-high_school_physics|5\": {\n \"acc\"\ : 0.33112582781456956,\n \"acc_stderr\": 0.038425817186598696,\n \"\ acc_norm\": 0.33112582781456956,\n \"acc_norm_stderr\": 0.038425817186598696\n\ \ },\n \"harness|hendrycksTest-high_school_psychology|5\": {\n \"acc\"\ : 0.8165137614678899,\n \"acc_stderr\": 0.0165952597103993,\n \"acc_norm\"\ : 0.8165137614678899,\n \"acc_norm_stderr\": 0.0165952597103993\n },\n\ \ \"harness|hendrycksTest-high_school_statistics|5\": {\n \"acc\": 0.4675925925925926,\n\ \ \"acc_stderr\": 0.03402801581358966,\n \"acc_norm\": 0.4675925925925926,\n\ \ \"acc_norm_stderr\": 0.03402801581358966\n },\n \"harness|hendrycksTest-high_school_us_history|5\"\ : {\n \"acc\": 0.7598039215686274,\n \"acc_stderr\": 0.02998373305591362,\n\ \ \"acc_norm\": 0.7598039215686274,\n \"acc_norm_stderr\": 0.02998373305591362\n\ \ },\n \"harness|hendrycksTest-high_school_world_history|5\": {\n \"\ acc\": 0.7848101265822784,\n \"acc_stderr\": 0.02675082699467618,\n \ \ \"acc_norm\": 0.7848101265822784,\n \"acc_norm_stderr\": 0.02675082699467618\n\ \ },\n \"harness|hendrycksTest-human_aging|5\": {\n \"acc\": 0.672645739910314,\n\ \ \"acc_stderr\": 0.03149384670994131,\n \"acc_norm\": 0.672645739910314,\n\ \ \"acc_norm_stderr\": 0.03149384670994131\n },\n \"harness|hendrycksTest-human_sexuality|5\"\ : {\n \"acc\": 0.7709923664122137,\n \"acc_stderr\": 0.036853466317118506,\n\ \ \"acc_norm\": 0.7709923664122137,\n \"acc_norm_stderr\": 0.036853466317118506\n\ \ },\n \"harness|hendrycksTest-international_law|5\": {\n \"acc\":\ \ 0.743801652892562,\n \"acc_stderr\": 0.03984979653302872,\n \"acc_norm\"\ : 0.743801652892562,\n \"acc_norm_stderr\": 0.03984979653302872\n },\n\ \ \"harness|hendrycksTest-jurisprudence|5\": {\n \"acc\": 0.7870370370370371,\n\ \ \"acc_stderr\": 0.0395783547198098,\n \"acc_norm\": 0.7870370370370371,\n\ \ \"acc_norm_stderr\": 0.0395783547198098\n },\n \"harness|hendrycksTest-logical_fallacies|5\"\ : {\n \"acc\": 0.6932515337423313,\n \"acc_stderr\": 0.036230899157241474,\n\ \ \"acc_norm\": 0.6932515337423313,\n \"acc_norm_stderr\": 0.036230899157241474\n\ \ },\n \"harness|hendrycksTest-machine_learning|5\": {\n \"acc\": 0.45535714285714285,\n\ \ \"acc_stderr\": 0.047268355537191,\n \"acc_norm\": 0.45535714285714285,\n\ \ \"acc_norm_stderr\": 0.047268355537191\n },\n \"harness|hendrycksTest-management|5\"\ : {\n \"acc\": 0.7864077669902912,\n \"acc_stderr\": 0.040580420156460344,\n\ \ \"acc_norm\": 0.7864077669902912,\n \"acc_norm_stderr\": 0.040580420156460344\n\ \ },\n \"harness|hendrycksTest-marketing|5\": {\n \"acc\": 0.8418803418803419,\n\ \ \"acc_stderr\": 0.023902325549560406,\n \"acc_norm\": 0.8418803418803419,\n\ \ \"acc_norm_stderr\": 0.023902325549560406\n },\n \"harness|hendrycksTest-medical_genetics|5\"\ : {\n \"acc\": 0.72,\n \"acc_stderr\": 0.045126085985421276,\n \ \ \"acc_norm\": 0.72,\n \"acc_norm_stderr\": 0.045126085985421276\n \ \ },\n \"harness|hendrycksTest-miscellaneous|5\": {\n \"acc\": 0.7994891443167306,\n\ \ \"acc_stderr\": 0.014317653708594209,\n \"acc_norm\": 0.7994891443167306,\n\ \ \"acc_norm_stderr\": 0.014317653708594209\n },\n \"harness|hendrycksTest-moral_disputes|5\"\ : {\n \"acc\": 0.6763005780346821,\n \"acc_stderr\": 0.02519018132760841,\n\ \ \"acc_norm\": 0.6763005780346821,\n \"acc_norm_stderr\": 0.02519018132760841\n\ \ },\n \"harness|hendrycksTest-moral_scenarios|5\": {\n \"acc\": 0.43910614525139663,\n\ \ \"acc_stderr\": 0.016598022120580425,\n \"acc_norm\": 0.43910614525139663,\n\ \ \"acc_norm_stderr\": 0.016598022120580425\n },\n \"harness|hendrycksTest-nutrition|5\"\ : {\n \"acc\": 0.7026143790849673,\n \"acc_stderr\": 0.02617390850671858,\n\ \ \"acc_norm\": 0.7026143790849673,\n \"acc_norm_stderr\": 0.02617390850671858\n\ \ },\n \"harness|hendrycksTest-philosophy|5\": {\n \"acc\": 0.7009646302250804,\n\ \ \"acc_stderr\": 0.02600330111788514,\n \"acc_norm\": 0.7009646302250804,\n\ \ \"acc_norm_stderr\": 0.02600330111788514\n },\n \"harness|hendrycksTest-prehistory|5\"\ : {\n \"acc\": 0.7407407407407407,\n \"acc_stderr\": 0.024383665531035454,\n\ \ \"acc_norm\": 0.7407407407407407,\n \"acc_norm_stderr\": 0.024383665531035454\n\ \ },\n \"harness|hendrycksTest-professional_accounting|5\": {\n \"\ acc\": 0.4645390070921986,\n \"acc_stderr\": 0.029752389657427047,\n \ \ \"acc_norm\": 0.4645390070921986,\n \"acc_norm_stderr\": 0.029752389657427047\n\ \ },\n \"harness|hendrycksTest-professional_law|5\": {\n \"acc\": 0.4439374185136897,\n\ \ \"acc_stderr\": 0.012689708167787682,\n \"acc_norm\": 0.4439374185136897,\n\ \ \"acc_norm_stderr\": 0.012689708167787682\n },\n \"harness|hendrycksTest-professional_medicine|5\"\ : {\n \"acc\": 0.6102941176470589,\n \"acc_stderr\": 0.0296246635811597,\n\ \ \"acc_norm\": 0.6102941176470589,\n \"acc_norm_stderr\": 0.0296246635811597\n\ \ },\n \"harness|hendrycksTest-professional_psychology|5\": {\n \"\ acc\": 0.6405228758169934,\n \"acc_stderr\": 0.01941253924203216,\n \ \ \"acc_norm\": 0.6405228758169934,\n \"acc_norm_stderr\": 0.01941253924203216\n\ \ },\n \"harness|hendrycksTest-public_relations|5\": {\n \"acc\": 0.6727272727272727,\n\ \ \"acc_stderr\": 0.0449429086625209,\n \"acc_norm\": 0.6727272727272727,\n\ \ \"acc_norm_stderr\": 0.0449429086625209\n },\n \"harness|hendrycksTest-security_studies|5\"\ : {\n \"acc\": 0.7142857142857143,\n \"acc_stderr\": 0.028920583220675606,\n\ \ \"acc_norm\": 0.7142857142857143,\n \"acc_norm_stderr\": 0.028920583220675606\n\ \ },\n \"harness|hendrycksTest-sociology|5\": {\n \"acc\": 0.8258706467661692,\n\ \ \"acc_stderr\": 0.026814951200421603,\n \"acc_norm\": 0.8258706467661692,\n\ \ \"acc_norm_stderr\": 0.026814951200421603\n },\n \"harness|hendrycksTest-us_foreign_policy|5\"\ : {\n \"acc\": 0.81,\n \"acc_stderr\": 0.03942772444036625,\n \ \ \"acc_norm\": 0.81,\n \"acc_norm_stderr\": 0.03942772444036625\n \ \ },\n \"harness|hendrycksTest-virology|5\": {\n \"acc\": 0.5301204819277109,\n\ \ \"acc_stderr\": 0.03885425420866767,\n \"acc_norm\": 0.5301204819277109,\n\ \ \"acc_norm_stderr\": 0.03885425420866767\n },\n \"harness|hendrycksTest-world_religions|5\"\ : {\n \"acc\": 0.8011695906432749,\n \"acc_stderr\": 0.03061111655743253,\n\ \ \"acc_norm\": 0.8011695906432749,\n \"acc_norm_stderr\": 0.03061111655743253\n\ \ },\n \"harness|truthfulqa:mc|0\": {\n \"mc1\": 0.43451652386780903,\n\ \ \"mc1_stderr\": 0.017352738749259564,\n \"mc2\": 0.6029363233989143,\n\ \ \"mc2_stderr\": 0.015526892800807802\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.7561168113654302,\n \"acc_stderr\": 0.01206892327890819\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.5003790750568613,\n \ \ \"acc_stderr\": 0.013772480761626175\n }\n}\n```" repo_url: https://huggingface.co/0-hero/Matter-0.1-7B-boost-DPO leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|arc:challenge|25_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2024-03-22T17-52-36.450234.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|gsm8k|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hellaswag|10_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-international_law|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-management|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-marketing|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-sociology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-virology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-international_law|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-management|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-marketing|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-sociology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-virology|5_2024-03-22T17-52-36.450234.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-astronomy|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-business_ethics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_biology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_medicine|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-college_physics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-computer_security|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-econometrics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-formal_logic|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-global_facts|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-human_aging|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-international_law|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-machine_learning|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-management|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-marketing|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-nutrition|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-philosophy|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-prehistory|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-professional_law|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-public_relations|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-security_studies|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-sociology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-virology|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|hendrycksTest-world_religions|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2024-03-22T17-52-36.450234.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|truthfulqa:mc|0_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2024-03-22T17-52-36.450234.parquet' - config_name: harness_winogrande_5 data_files: - split: 2024_03_22T17_52_36.450234 path: - '**/details_harness|winogrande|5_2024-03-22T17-52-36.450234.parquet' - split: latest path: - '**/details_harness|winogrande|5_2024-03-22T17-52-36.450234.parquet' - config_name: results data_files: - split: 2024_03_22T17_52_36.450234 path: - results_2024-03-22T17-52-36.450234.parquet - split: latest path: - results_2024-03-22T17-52-36.450234.parquet --- # Dataset Card for Evaluation run of 0-hero/Matter-0.1-7B-boost-DPO <!-- Provide a quick summary of the dataset. --> Dataset automatically created during the evaluation run of model [0-hero/Matter-0.1-7B-boost-DPO](https://huggingface.co/0-hero/Matter-0.1-7B-boost-DPO) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the aggregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2024-03-22T17:52:36.450234](https://huggingface.co/datasets/open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO/blob/main/results_2024-03-22T17-52-36.450234.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "acc": 0.6196341771509315, "acc_stderr": 0.03280520004973222, "acc_norm": 0.6228652576735157, "acc_norm_stderr": 0.033458484779768545, "mc1": 0.43451652386780903, "mc1_stderr": 0.017352738749259564, "mc2": 0.6029363233989143, "mc2_stderr": 0.015526892800807802 }, "harness|arc:challenge|25": { "acc": 0.6271331058020477, "acc_stderr": 0.01413117676013117, "acc_norm": 0.6501706484641638, "acc_norm_stderr": 0.013936809212158298 }, "harness|hellaswag|10": { "acc": 0.6459868552081258, "acc_stderr": 0.0047723583951304474, "acc_norm": 0.8308105954989046, "acc_norm_stderr": 0.0037415289563158434 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.27, "acc_stderr": 0.044619604333847394, "acc_norm": 0.27, "acc_norm_stderr": 0.044619604333847394 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5925925925925926, "acc_stderr": 0.04244633238353227, "acc_norm": 0.5925925925925926, "acc_norm_stderr": 0.04244633238353227 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6776315789473685, "acc_stderr": 0.03803510248351585, "acc_norm": 0.6776315789473685, "acc_norm_stderr": 0.03803510248351585 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.049999999999999996, "acc_norm": 0.55, "acc_norm_stderr": 0.049999999999999996 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6754716981132075, "acc_stderr": 0.028815615713432115, "acc_norm": 0.6754716981132075, "acc_norm_stderr": 0.028815615713432115 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6875, "acc_stderr": 0.038760854559127644, "acc_norm": 0.6875, "acc_norm_stderr": 0.038760854559127644 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.42, "acc_stderr": 0.049604496374885836, "acc_norm": 0.42, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.5, "acc_stderr": 0.050251890762960605, "acc_norm": 0.5, "acc_norm_stderr": 0.050251890762960605 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.35, "acc_stderr": 0.0479372485441102, "acc_norm": 0.35, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5838150289017341, "acc_stderr": 0.03758517775404947, "acc_norm": 0.5838150289017341, "acc_norm_stderr": 0.03758517775404947 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.35294117647058826, "acc_stderr": 0.04755129616062946, "acc_norm": 0.35294117647058826, "acc_norm_stderr": 0.04755129616062946 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5191489361702127, "acc_stderr": 0.032662042990646775, "acc_norm": 0.5191489361702127, "acc_norm_stderr": 0.032662042990646775 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4649122807017544, "acc_stderr": 0.04692008381368909, "acc_norm": 0.4649122807017544, "acc_norm_stderr": 0.04692008381368909 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5862068965517241, "acc_stderr": 0.04104269211806232, "acc_norm": 0.5862068965517241, "acc_norm_stderr": 0.04104269211806232 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3439153439153439, "acc_stderr": 0.024464426625596437, "acc_norm": 0.3439153439153439, "acc_norm_stderr": 0.024464426625596437 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.38095238095238093, "acc_stderr": 0.043435254289490965, "acc_norm": 0.38095238095238093, "acc_norm_stderr": 0.043435254289490965 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.44, "acc_stderr": 0.04988876515698589, "acc_norm": 0.44, "acc_norm_stderr": 0.04988876515698589 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7516129032258064, "acc_stderr": 0.024580028921481, "acc_norm": 0.7516129032258064, "acc_norm_stderr": 0.024580028921481 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5024630541871922, "acc_stderr": 0.035179450386910616, "acc_norm": 0.5024630541871922, "acc_norm_stderr": 0.035179450386910616 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.67, "acc_stderr": 0.04725815626252609, "acc_norm": 0.67, "acc_norm_stderr": 0.04725815626252609 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7636363636363637, "acc_stderr": 0.03317505930009181, "acc_norm": 0.7636363636363637, "acc_norm_stderr": 0.03317505930009181 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7727272727272727, "acc_stderr": 0.029857515673386414, "acc_norm": 0.7727272727272727, "acc_norm_stderr": 0.029857515673386414 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8652849740932642, "acc_stderr": 0.02463978909770944, "acc_norm": 0.8652849740932642, "acc_norm_stderr": 0.02463978909770944 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6230769230769231, "acc_stderr": 0.024570975364225995, "acc_norm": 0.6230769230769231, "acc_norm_stderr": 0.024570975364225995 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3148148148148148, "acc_stderr": 0.028317533496066465, "acc_norm": 0.3148148148148148, "acc_norm_stderr": 0.028317533496066465 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6554621848739496, "acc_stderr": 0.030868682604121622, "acc_norm": 0.6554621848739496, "acc_norm_stderr": 0.030868682604121622 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.33112582781456956, "acc_stderr": 0.038425817186598696, "acc_norm": 0.33112582781456956, "acc_norm_stderr": 0.038425817186598696 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.8165137614678899, "acc_stderr": 0.0165952597103993, "acc_norm": 0.8165137614678899, "acc_norm_stderr": 0.0165952597103993 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.4675925925925926, "acc_stderr": 0.03402801581358966, "acc_norm": 0.4675925925925926, "acc_norm_stderr": 0.03402801581358966 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.7598039215686274, "acc_stderr": 0.02998373305591362, "acc_norm": 0.7598039215686274, "acc_norm_stderr": 0.02998373305591362 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.7848101265822784, "acc_stderr": 0.02675082699467618, "acc_norm": 0.7848101265822784, "acc_norm_stderr": 0.02675082699467618 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.672645739910314, "acc_stderr": 0.03149384670994131, "acc_norm": 0.672645739910314, "acc_norm_stderr": 0.03149384670994131 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.7709923664122137, "acc_stderr": 0.036853466317118506, "acc_norm": 0.7709923664122137, "acc_norm_stderr": 0.036853466317118506 }, "harness|hendrycksTest-international_law|5": { "acc": 0.743801652892562, "acc_stderr": 0.03984979653302872, "acc_norm": 0.743801652892562, "acc_norm_stderr": 0.03984979653302872 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.7870370370370371, "acc_stderr": 0.0395783547198098, "acc_norm": 0.7870370370370371, "acc_norm_stderr": 0.0395783547198098 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.6932515337423313, "acc_stderr": 0.036230899157241474, "acc_norm": 0.6932515337423313, "acc_norm_stderr": 0.036230899157241474 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.45535714285714285, "acc_stderr": 0.047268355537191, "acc_norm": 0.45535714285714285, "acc_norm_stderr": 0.047268355537191 }, "harness|hendrycksTest-management|5": { "acc": 0.7864077669902912, "acc_stderr": 0.040580420156460344, "acc_norm": 0.7864077669902912, "acc_norm_stderr": 0.040580420156460344 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8418803418803419, "acc_stderr": 0.023902325549560406, "acc_norm": 0.8418803418803419, "acc_norm_stderr": 0.023902325549560406 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.72, "acc_stderr": 0.045126085985421276, "acc_norm": 0.72, "acc_norm_stderr": 0.045126085985421276 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.7994891443167306, "acc_stderr": 0.014317653708594209, "acc_norm": 0.7994891443167306, "acc_norm_stderr": 0.014317653708594209 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.6763005780346821, "acc_stderr": 0.02519018132760841, "acc_norm": 0.6763005780346821, "acc_norm_stderr": 0.02519018132760841 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.43910614525139663, "acc_stderr": 0.016598022120580425, "acc_norm": 0.43910614525139663, "acc_norm_stderr": 0.016598022120580425 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.7026143790849673, "acc_stderr": 0.02617390850671858, "acc_norm": 0.7026143790849673, "acc_norm_stderr": 0.02617390850671858 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.7009646302250804, "acc_stderr": 0.02600330111788514, "acc_norm": 0.7009646302250804, "acc_norm_stderr": 0.02600330111788514 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.7407407407407407, "acc_stderr": 0.024383665531035454, "acc_norm": 0.7407407407407407, "acc_norm_stderr": 0.024383665531035454 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.4645390070921986, "acc_stderr": 0.029752389657427047, "acc_norm": 0.4645390070921986, "acc_norm_stderr": 0.029752389657427047 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.4439374185136897, "acc_stderr": 0.012689708167787682, "acc_norm": 0.4439374185136897, "acc_norm_stderr": 0.012689708167787682 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.6102941176470589, "acc_stderr": 0.0296246635811597, "acc_norm": 0.6102941176470589, "acc_norm_stderr": 0.0296246635811597 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.6405228758169934, "acc_stderr": 0.01941253924203216, "acc_norm": 0.6405228758169934, "acc_norm_stderr": 0.01941253924203216 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.6727272727272727, "acc_stderr": 0.0449429086625209, "acc_norm": 0.6727272727272727, "acc_norm_stderr": 0.0449429086625209 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.7142857142857143, "acc_stderr": 0.028920583220675606, "acc_norm": 0.7142857142857143, "acc_norm_stderr": 0.028920583220675606 }, "harness|hendrycksTest-sociology|5": { "acc": 0.8258706467661692, "acc_stderr": 0.026814951200421603, "acc_norm": 0.8258706467661692, "acc_norm_stderr": 0.026814951200421603 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.81, "acc_stderr": 0.03942772444036625, "acc_norm": 0.81, "acc_norm_stderr": 0.03942772444036625 }, "harness|hendrycksTest-virology|5": { "acc": 0.5301204819277109, "acc_stderr": 0.03885425420866767, "acc_norm": 0.5301204819277109, "acc_norm_stderr": 0.03885425420866767 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.8011695906432749, "acc_stderr": 0.03061111655743253, "acc_norm": 0.8011695906432749, "acc_norm_stderr": 0.03061111655743253 }, "harness|truthfulqa:mc|0": { "mc1": 0.43451652386780903, "mc1_stderr": 0.017352738749259564, "mc2": 0.6029363233989143, "mc2_stderr": 0.015526892800807802 }, "harness|winogrande|5": { "acc": 0.7561168113654302, "acc_stderr": 0.01206892327890819 }, "harness|gsm8k|5": { "acc": 0.5003790750568613, "acc_stderr": 0.013772480761626175 } } ``` ## Dataset Details ### Dataset Description <!-- Provide a longer summary of what this dataset is. --> - **Curated by:** [More Information Needed] - **Funded by [optional]:** [More Information Needed] - **Shared by [optional]:** [More Information Needed] - **Language(s) (NLP):** [More Information Needed] - **License:** [More Information Needed] ### Dataset Sources [optional] <!-- Provide the basic links for the dataset. --> - **Repository:** [More Information Needed] - **Paper [optional]:** [More Information Needed] - **Demo [optional]:** [More Information Needed] ## Uses <!-- Address questions around how the dataset is intended to be used. --> ### Direct Use <!-- This section describes suitable use cases for the dataset. --> [More Information Needed] ### Out-of-Scope Use <!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. --> [More Information Needed] ## Dataset Structure <!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. --> [More Information Needed] ## Dataset Creation ### Curation Rationale <!-- Motivation for the creation of this dataset. --> [More Information Needed] ### Source Data <!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). --> #### Data Collection and Processing <!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. --> [More Information Needed] #### Who are the source data producers? <!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. --> [More Information Needed] ### Annotations [optional] <!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. --> #### Annotation process <!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. --> [More Information Needed] #### Who are the annotators? <!-- This section describes the people or systems who created the annotations. --> [More Information Needed] #### Personal and Sensitive Information <!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. --> [More Information Needed] ## Bias, Risks, and Limitations <!-- This section is meant to convey both technical and sociotechnical limitations. --> [More Information Needed] ### Recommendations <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations. ## Citation [optional] <!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. --> **BibTeX:** [More Information Needed] **APA:** [More Information Needed] ## Glossary [optional] <!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. --> [More Information Needed] ## More Information [optional] [More Information Needed] ## Dataset Card Authors [optional] [More Information Needed] ## Dataset Card Contact [More Information Needed]该数据集是在模型0-hero/Matter-0.1-7B-boost-DPO评估运行期间自动创建的，包含63个配置，每个配置对应一个评估任务。数据集来源于一次运行，每次运行作为每个配置中的特定拆分，以运行的时间戳命名。train拆分始终指向最新结果。额外配置results存储了运行的所有聚合结果，用于在Open LLM Leaderboard上计算和显示聚合指标。数据集涵盖了多种任务和配置的指标，如准确度、准确度标准误差、归一化准确度、归一化准确度标准误差、mc1、mc1标准误差、mc2和mc2标准误差。

原始信息汇总

数据集概述

数据集名称

• pretty_name: Evaluation run of 0-hero/Matter-0.1-7B-boost-DPO

数据集描述

• dataset_summary: 该数据集是在评估模型0-hero/Matter-0.1-7B-boost-DPO的过程中自动创建的，用于Open LLM Leaderboard。

数据集结构

• 组成: 包含63个配置，每个配置对应一个评估任务。
• 创建来源: 数据集由1次运行创建，每次运行在每个配置中作为一个特定的分割存在，分割名称使用运行的时间戳命名。
• 额外配置: “results”配置存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。

数据集加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO", "harness_winogrande_5", split="train")

最新结果

• 结果来源: 最新结果来自run 2024-03-22T17:52:36.450234。
• 结果内容: 包括多个任务的准确率(acc)和标准误差(acc_stderr)等指标。

数据集配置详情

配置列表

• config_name: harness_arc_challenge_25

  • data_files:
    • split: 2024_03_22T17_52_36.450234
      • path: /details_harness|arc:challenge|25_2024-03-22T17-52-36.450234.parquet
    • split: latest
      • path: /details_harness|arc:challenge|25_2024-03-22T17-52-36.450234.parquet

• config_name: harness_gsm8k_5

  • data_files:
    • split: 2024_03_22T17_52_36.450234
      • path: /details_harness|gsm8k|5_2024-03-22T17-52-36.450234.parquet
    • split: latest
      • path: /details_harness|gsm8k|5_2024-03-22T17-52-36.450234.parquet

• config_name: harness_hellaswag_10

  • data_files:
    • split: 2024_03_22T17_52_36.450234
      • path: /details_harness|hellaswag|10_2024-03-22T17-52-36.450234.parquet
    • split: latest
      • path: /details_harness|hellaswag|10_2024-03-22T17-52-36.450234.parquet

• config_name: harness_hendrycksTest_5

  • data_files:
    • split: 2024_03_22T17_52_36.450234
      • path:
        • /details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-astronomy|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-business_ethics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-clinical_knowledge|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_biology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_chemistry|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_computer_science|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_mathematics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_medicine|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-college_physics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-computer_security|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-conceptual_physics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-econometrics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-electrical_engineering|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-elementary_mathematics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-formal_logic|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-global_facts|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_biology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_chemistry|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_computer_science|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_european_history|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_geography|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_mathematics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_physics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_psychology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_statistics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_us_history|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-high_school_world_history|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-human_aging|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-human_sexuality|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-international_law|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-jurisprudence|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-logical_fallacies|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-machine_learning|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-management|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-marketing|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-medical_genetics|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-miscellaneous|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-moral_disputes|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-moral_scenarios|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-nutrition|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-philosophy|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-prehistory|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-professional_accounting|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-professional_law|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-professional_medicine|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-professional_psychology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-public_relations|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-security_studies|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-sociology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-us_foreign_policy|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-virology|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-world_religions|5_2024-03-22T17-52-36.450234.parquet
    • split: latest
      • path:
        • /details_harness|hendrycksTest-abstract_algebra|5_2024-03-22T17-52-36.450234.parquet
        • /details_harness|hendrycksTest-anatomy|5_2024-03-22T17-52-36.450234.parquet
        • **/details_harness|hendrycksTest-astronomy|

搜集汇总

数据集介绍

构建方式

在大型语言模型评估领域，Open LLM Leaderboard 为模型性能的横向比较提供了标准化平台。该数据集是针对 0-hero/Matter-0.1-7B-boost-DPO 模型在 Leaderboard 上的一次完整评估运行而自动生成的产物。其构建方式基于单一评估流程，覆盖了 63 个不同的任务配置，这些配置对应着模型所接受的各项评测任务，例如 ARC-Challenge、HellaSwag、GSM8K 以及涵盖 57 个学科的 MMLU 子集。每次运行的详细结果被存储为独立的 parquet 文件，并按时间戳标记为特定数据分割，而 'train' 分割则始终指向最新一次运行的成果。此外，一个名为 'results' 的汇总配置整合了所有任务的聚合指标，用于在 Leaderboard 上计算并展示模型的综合表现。

特点

该数据集的核心特色在于其结构化的多任务评估记录方式。它并非单一维度的评分集合，而是将模型在 63 个细粒度任务上的原始评测细节（如准确率及其标准误差）分门别类地存储于独立配置中，这为研究者深入分析模型在不同能力维度（如推理、常识、数学及专业知识）上的优劣提供了可能。数据集的版本控制机制亦是一大亮点：通过保留每次运行的时间戳分割，它完整记录了模型性能的演变轨迹，而 'latest' 分割的设定则确保了用户总能便捷地获取最新评估结果。这种设计兼顾了历史追溯与实时更新的需求，体现了评测数据管理的严谨性与灵活性。

使用方法

研究人员可通过 Hugging Face 的 datasets 库便捷地加载与使用该数据集。具体而言，用户需调用 load_dataset 函数，并指定数据集名称 'open-llm-leaderboard/details_0-hero__Matter-0.1-7B-boost-DPO' 以及目标任务配置（如 'harness_winogrande_5'）和数据分割（例如 'train' 以获取最新结果）。加载后的数据以 Parquet 格式存储，包含了模型在该任务上的详细评测指标。对于需要跨任务综合分析的场景，用户可遍历所有配置，或直接读取 'results' 配置中的聚合数据，以获取模型在全部评测任务上的完整性能图谱，从而支撑深入的模型能力对比与学术研究。

背景与挑战

背景概述

随着大规模语言模型在自然语言处理领域的迅猛发展，如何系统性地评估模型在多样化任务上的综合性能成为研究焦点。由HuggingFace社区主导的Open LLM Leaderboard应运而生，旨在为开源语言模型提供标准化、可复现的评估基准。该数据集作为对0-hero/Matter-0.1-7B-boost-DPO模型的一次完整评估记录，创建于2024年3月，由Clementine等研究人员维护，涵盖63个配置项，对应ARC-Challenge、HellaSwag、GSM8K、TruthfulQA、WinoGrande以及涵盖57个学科的MMLU等核心评测任务。该数据集通过自动化的评估流程，将模型在推理、常识、数学、事实一致性及多学科知识上的表现以细粒度指标形式呈现，为社区提供了可横向对比的实证依据，有力推动了开源大模型透明化评估体系的建设。

当前挑战

该数据集所解决的领域挑战在于为大语言模型构建一个全面、公平且可复现的评测框架。传统评估方式常因任务单一、评测标准不统一而难以反映模型真实能力，Open LLM Leaderboard通过整合多个权威基准并统一运行环境，有效缓解了评测碎片化问题。然而，构建过程本身面临多重挑战：其一，需确保每次评估的硬件环境与超参数严格一致，以避免外部因素干扰结果可比性；其二，面对MMLU等包含57个学科的子任务，如何高效管理数据文件版本与运行时间戳，保证不同批次评测结果的可追溯性；其三，模型输出具有随机性，需通过多次运行取统计量以降低方差，这对计算资源与时间成本提出了较高要求。

常用场景

经典使用场景

该数据集作为Open LLM Leaderboard的评估记录，核心用途在于系统性地追踪与比较大型语言模型在多样化自然语言理解与生成任务上的表现。其经典使用方式是通过加载特定配置（如harness_arc_challenge_25或harness_gsm8k_5）及对应时间戳的分割，重现模型在ARC挑战、HellaSwag常识推理、MMLU多学科知识、TruthfulQA真实性判别、WinoGrande指代消解及GSM8K数学推理等任务上的细粒度得分，为模型性能的横向对比与纵向演进分析提供结构化数据支撑。

解决学术问题

该数据集从根本上解决了大语言模型评估过程中结果可复现性不足与跨任务性能对齐困难的核心学术难题。通过标准化的评估框架与统一的数据格式，它使得研究者能够精确衡量模型在常识推理、知识掌握、数学计算及对抗性问答等多个维度的能力边界，从而揭示不同训练策略（如DPO偏好优化）对模型泛化性能的差异化影响。其意义在于为模型改进方向提供实证依据，推动评估基准的透明化与公平性，并促进了关于模型鲁棒性与知识覆盖度的系统性研究。

衍生相关工作

该数据集衍生了一系列关于评估方法论与模型优化的经典工作。例如，研究者利用其多任务得分分析DPO（直接偏好优化）算法相较于标准监督微调的优势，并基于细粒度结果改进奖励模型设计；另有工作以此数据集为基准，探索少样本提示与思维链推理对模型性能的增益效果。此外，该数据集的时序分割支持对模型能力进化轨迹的追踪，催生了关于持续学习与灾难性遗忘的研究，并成为验证新型评估指标（如校准误差）有效性的重要平台。