open-llm-leaderboard-old/details_yeontaek__llama-2-13B-ensemble-v1

--- pretty_name: Evaluation run of yeontaek/llama-2-13B-ensemble-v1 dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [yeontaek/llama-2-13B-ensemble-v1](https://huggingface.co/yeontaek/llama-2-13B-ensemble-v1)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 60 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 1 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v1\"\ ,\n\t\"harness_truthfulqa_mc_0\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\ \nThese are the [latest results from run 2023-08-24T06:11:39.305449](https://huggingface.co/datasets/open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v1/blob/main/results_2023-08-24T06%3A11%3A39.305449.json)\ \ (note that their might be results for other tasks in the repos if successive evals\ \ didn't cover the same tasks. You find each in the results and the \"latest\" split\ \ for each eval):\n\n```python\n{\n \"all\": {\n \"acc\": 0.5771705795394726,\n\ \ \"acc_stderr\": 0.03407019951989153,\n \"acc_norm\": 0.5809029426034396,\n\ \ \"acc_norm_stderr\": 0.0340502352653076,\n \"mc1\": 0.3598531211750306,\n\ \ \"mc1_stderr\": 0.016801860466677157,\n \"mc2\": 0.5015608436175973,\n\ \ \"mc2_stderr\": 0.015270015874980385\n },\n \"harness|arc:challenge|25\"\ : {\n \"acc\": 0.5998293515358362,\n \"acc_stderr\": 0.014317197787809169,\n\ \ \"acc_norm\": 0.6228668941979523,\n \"acc_norm_stderr\": 0.0141633668961926\n\ \ },\n \"harness|hellaswag|10\": {\n \"acc\": 0.6264688309101772,\n\ \ \"acc_stderr\": 0.004827526584889677,\n \"acc_norm\": 0.8236407090221072,\n\ \ \"acc_norm_stderr\": 0.0038034664560544717\n },\n \"harness|hendrycksTest-abstract_algebra|5\"\ : {\n \"acc\": 0.32,\n \"acc_stderr\": 0.04688261722621504,\n \ \ \"acc_norm\": 0.32,\n \"acc_norm_stderr\": 0.04688261722621504\n \ \ },\n \"harness|hendrycksTest-anatomy|5\": {\n \"acc\": 0.45925925925925926,\n\ \ \"acc_stderr\": 0.04304979692464243,\n \"acc_norm\": 0.45925925925925926,\n\ \ \"acc_norm_stderr\": 0.04304979692464243\n },\n \"harness|hendrycksTest-astronomy|5\"\ : {\n \"acc\": 0.5394736842105263,\n \"acc_stderr\": 0.04056242252249034,\n\ \ \"acc_norm\": 0.5394736842105263,\n \"acc_norm_stderr\": 0.04056242252249034\n\ \ },\n \"harness|hendrycksTest-business_ethics|5\": {\n \"acc\": 0.58,\n\ \ \"acc_stderr\": 0.049604496374885836,\n \"acc_norm\": 0.58,\n \ \ \"acc_norm_stderr\": 0.049604496374885836\n },\n \"harness|hendrycksTest-clinical_knowledge|5\"\ : {\n \"acc\": 0.6075471698113207,\n \"acc_stderr\": 0.03005258057955784,\n\ \ \"acc_norm\": 0.6075471698113207,\n \"acc_norm_stderr\": 0.03005258057955784\n\ \ },\n \"harness|hendrycksTest-college_biology|5\": {\n \"acc\": 0.6319444444444444,\n\ \ \"acc_stderr\": 0.040329990539607175,\n \"acc_norm\": 0.6319444444444444,\n\ \ \"acc_norm_stderr\": 0.040329990539607175\n },\n \"harness|hendrycksTest-college_chemistry|5\"\ : {\n \"acc\": 0.37,\n \"acc_stderr\": 0.048523658709391,\n \ \ \"acc_norm\": 0.37,\n \"acc_norm_stderr\": 0.048523658709391\n },\n\ \ \"harness|hendrycksTest-college_computer_science|5\": {\n \"acc\": 0.48,\n\ \ \"acc_stderr\": 0.050211673156867795,\n \"acc_norm\": 0.48,\n \ \ \"acc_norm_stderr\": 0.050211673156867795\n },\n \"harness|hendrycksTest-college_mathematics|5\"\ : {\n \"acc\": 0.34,\n \"acc_stderr\": 0.04760952285695236,\n \ \ \"acc_norm\": 0.34,\n \"acc_norm_stderr\": 0.04760952285695236\n \ \ },\n \"harness|hendrycksTest-college_medicine|5\": {\n \"acc\": 0.5491329479768786,\n\ \ \"acc_stderr\": 0.03794012674697031,\n \"acc_norm\": 0.5491329479768786,\n\ \ \"acc_norm_stderr\": 0.03794012674697031\n },\n \"harness|hendrycksTest-college_physics|5\"\ : {\n \"acc\": 0.3235294117647059,\n \"acc_stderr\": 0.046550104113196177,\n\ \ \"acc_norm\": 0.3235294117647059,\n \"acc_norm_stderr\": 0.046550104113196177\n\ \ },\n \"harness|hendrycksTest-computer_security|5\": {\n \"acc\":\ \ 0.75,\n \"acc_stderr\": 0.04351941398892446,\n \"acc_norm\": 0.75,\n\ \ \"acc_norm_stderr\": 0.04351941398892446\n },\n \"harness|hendrycksTest-conceptual_physics|5\"\ : {\n \"acc\": 0.46382978723404256,\n \"acc_stderr\": 0.032600385118357715,\n\ \ \"acc_norm\": 0.46382978723404256,\n \"acc_norm_stderr\": 0.032600385118357715\n\ \ },\n \"harness|hendrycksTest-econometrics|5\": {\n \"acc\": 0.30701754385964913,\n\ \ \"acc_stderr\": 0.04339138322579861,\n \"acc_norm\": 0.30701754385964913,\n\ \ \"acc_norm_stderr\": 0.04339138322579861\n },\n \"harness|hendrycksTest-electrical_engineering|5\"\ : {\n \"acc\": 0.5448275862068965,\n \"acc_stderr\": 0.04149886942192118,\n\ \ \"acc_norm\": 0.5448275862068965,\n \"acc_norm_stderr\": 0.04149886942192118\n\ \ },\n \"harness|hendrycksTest-elementary_mathematics|5\": {\n \"acc\"\ : 0.3306878306878307,\n \"acc_stderr\": 0.02422996529842507,\n \"\ acc_norm\": 0.3306878306878307,\n \"acc_norm_stderr\": 0.02422996529842507\n\ \ },\n \"harness|hendrycksTest-formal_logic|5\": {\n \"acc\": 0.35714285714285715,\n\ \ \"acc_stderr\": 0.042857142857142816,\n \"acc_norm\": 0.35714285714285715,\n\ \ \"acc_norm_stderr\": 0.042857142857142816\n },\n \"harness|hendrycksTest-global_facts|5\"\ : {\n \"acc\": 0.35,\n \"acc_stderr\": 0.047937248544110196,\n \ \ \"acc_norm\": 0.35,\n \"acc_norm_stderr\": 0.047937248544110196\n \ \ },\n \"harness|hendrycksTest-high_school_biology|5\": {\n \"acc\"\ : 0.6580645161290323,\n \"acc_stderr\": 0.026985289576552746,\n \"\ acc_norm\": 0.6580645161290323,\n \"acc_norm_stderr\": 0.026985289576552746\n\ \ },\n \"harness|hendrycksTest-high_school_chemistry|5\": {\n \"acc\"\ : 0.42857142857142855,\n \"acc_stderr\": 0.03481904844438803,\n \"\ acc_norm\": 0.42857142857142855,\n \"acc_norm_stderr\": 0.03481904844438803\n\ \ },\n \"harness|hendrycksTest-high_school_computer_science|5\": {\n \ \ \"acc\": 0.59,\n \"acc_stderr\": 0.04943110704237102,\n \"acc_norm\"\ : 0.59,\n \"acc_norm_stderr\": 0.04943110704237102\n },\n \"harness|hendrycksTest-high_school_european_history|5\"\ : {\n \"acc\": 0.7090909090909091,\n \"acc_stderr\": 0.03546563019624336,\n\ \ \"acc_norm\": 0.7090909090909091,\n \"acc_norm_stderr\": 0.03546563019624336\n\ \ },\n \"harness|hendrycksTest-high_school_geography|5\": {\n \"acc\"\ : 0.7424242424242424,\n \"acc_stderr\": 0.03115626951964683,\n \"\ acc_norm\": 0.7424242424242424,\n \"acc_norm_stderr\": 0.03115626951964683\n\ \ },\n \"harness|hendrycksTest-high_school_government_and_politics|5\": {\n\ \ \"acc\": 0.8238341968911918,\n \"acc_stderr\": 0.02749350424454806,\n\ \ \"acc_norm\": 0.8238341968911918,\n \"acc_norm_stderr\": 0.02749350424454806\n\ \ },\n \"harness|hendrycksTest-high_school_macroeconomics|5\": {\n \ \ \"acc\": 0.5692307692307692,\n \"acc_stderr\": 0.02510682066053975,\n \ \ \"acc_norm\": 0.5692307692307692,\n \"acc_norm_stderr\": 0.02510682066053975\n\ \ },\n \"harness|hendrycksTest-high_school_mathematics|5\": {\n \"\ acc\": 0.2962962962962963,\n \"acc_stderr\": 0.02784081149587192,\n \ \ \"acc_norm\": 0.2962962962962963,\n \"acc_norm_stderr\": 0.02784081149587192\n\ \ },\n \"harness|hendrycksTest-high_school_microeconomics|5\": {\n \ \ \"acc\": 0.6092436974789915,\n \"acc_stderr\": 0.03169380235712996,\n \ \ \"acc_norm\": 0.6092436974789915,\n \"acc_norm_stderr\": 0.03169380235712996\n\ \ },\n \"harness|hendrycksTest-high_school_physics|5\": {\n \"acc\"\ : 0.31788079470198677,\n \"acc_stderr\": 0.038020397601079024,\n \"\ acc_norm\": 0.31788079470198677,\n \"acc_norm_stderr\": 0.038020397601079024\n\ \ },\n \"harness|hendrycksTest-high_school_psychology|5\": {\n \"acc\"\ : 0.7651376146788991,\n \"acc_stderr\": 0.018175110510343564,\n \"\ acc_norm\": 0.7651376146788991,\n \"acc_norm_stderr\": 0.018175110510343564\n\ \ },\n \"harness|hendrycksTest-high_school_statistics|5\": {\n \"acc\"\ : 0.44907407407407407,\n \"acc_stderr\": 0.03392238405321617,\n \"\ acc_norm\": 0.44907407407407407,\n \"acc_norm_stderr\": 0.03392238405321617\n\ \ },\n \"harness|hendrycksTest-high_school_us_history|5\": {\n \"acc\"\ : 0.7598039215686274,\n \"acc_stderr\": 0.02998373305591361,\n \"\ acc_norm\": 0.7598039215686274,\n \"acc_norm_stderr\": 0.02998373305591361\n\ \ },\n \"harness|hendrycksTest-high_school_world_history|5\": {\n \"\ acc\": 0.759493670886076,\n \"acc_stderr\": 0.027820781981149685,\n \ \ \"acc_norm\": 0.759493670886076,\n \"acc_norm_stderr\": 0.027820781981149685\n\ \ },\n \"harness|hendrycksTest-human_aging|5\": {\n \"acc\": 0.672645739910314,\n\ \ \"acc_stderr\": 0.03149384670994131,\n \"acc_norm\": 0.672645739910314,\n\ \ \"acc_norm_stderr\": 0.03149384670994131\n },\n \"harness|hendrycksTest-human_sexuality|5\"\ : {\n \"acc\": 0.6793893129770993,\n \"acc_stderr\": 0.04093329229834278,\n\ \ \"acc_norm\": 0.6793893129770993,\n \"acc_norm_stderr\": 0.04093329229834278\n\ \ },\n \"harness|hendrycksTest-international_law|5\": {\n \"acc\":\ \ 0.7768595041322314,\n \"acc_stderr\": 0.03800754475228732,\n \"\ acc_norm\": 0.7768595041322314,\n \"acc_norm_stderr\": 0.03800754475228732\n\ \ },\n \"harness|hendrycksTest-jurisprudence|5\": {\n \"acc\": 0.75,\n\ \ \"acc_stderr\": 0.04186091791394607,\n \"acc_norm\": 0.75,\n \ \ \"acc_norm_stderr\": 0.04186091791394607\n },\n \"harness|hendrycksTest-logical_fallacies|5\"\ : {\n \"acc\": 0.7055214723926381,\n \"acc_stderr\": 0.03581165790474082,\n\ \ \"acc_norm\": 0.7055214723926381,\n \"acc_norm_stderr\": 0.03581165790474082\n\ \ },\n \"harness|hendrycksTest-machine_learning|5\": {\n \"acc\": 0.39285714285714285,\n\ \ \"acc_stderr\": 0.04635550135609976,\n \"acc_norm\": 0.39285714285714285,\n\ \ \"acc_norm_stderr\": 0.04635550135609976\n },\n \"harness|hendrycksTest-management|5\"\ : {\n \"acc\": 0.7669902912621359,\n \"acc_stderr\": 0.04185832598928315,\n\ \ \"acc_norm\": 0.7669902912621359,\n \"acc_norm_stderr\": 0.04185832598928315\n\ \ },\n \"harness|hendrycksTest-marketing|5\": {\n \"acc\": 0.8290598290598291,\n\ \ \"acc_stderr\": 0.024662496845209807,\n \"acc_norm\": 0.8290598290598291,\n\ \ \"acc_norm_stderr\": 0.024662496845209807\n },\n \"harness|hendrycksTest-medical_genetics|5\"\ : {\n \"acc\": 0.67,\n \"acc_stderr\": 0.04725815626252609,\n \ \ \"acc_norm\": 0.67,\n \"acc_norm_stderr\": 0.04725815626252609\n \ \ },\n \"harness|hendrycksTest-miscellaneous|5\": {\n \"acc\": 0.776500638569604,\n\ \ \"acc_stderr\": 0.01489723522945071,\n \"acc_norm\": 0.776500638569604,\n\ \ \"acc_norm_stderr\": 0.01489723522945071\n },\n \"harness|hendrycksTest-moral_disputes|5\"\ : {\n \"acc\": 0.653179190751445,\n \"acc_stderr\": 0.025624723994030454,\n\ \ \"acc_norm\": 0.653179190751445,\n \"acc_norm_stderr\": 0.025624723994030454\n\ \ },\n \"harness|hendrycksTest-moral_scenarios|5\": {\n \"acc\": 0.4324022346368715,\n\ \ \"acc_stderr\": 0.01656897123354861,\n \"acc_norm\": 0.4324022346368715,\n\ \ \"acc_norm_stderr\": 0.01656897123354861\n },\n \"harness|hendrycksTest-nutrition|5\"\ : {\n \"acc\": 0.630718954248366,\n \"acc_stderr\": 0.02763417668960266,\n\ \ \"acc_norm\": 0.630718954248366,\n \"acc_norm_stderr\": 0.02763417668960266\n\ \ },\n \"harness|hendrycksTest-philosophy|5\": {\n \"acc\": 0.6527331189710611,\n\ \ \"acc_stderr\": 0.027040745502307336,\n \"acc_norm\": 0.6527331189710611,\n\ \ \"acc_norm_stderr\": 0.027040745502307336\n },\n \"harness|hendrycksTest-prehistory|5\"\ : {\n \"acc\": 0.6265432098765432,\n \"acc_stderr\": 0.026915003011380154,\n\ \ \"acc_norm\": 0.6265432098765432,\n \"acc_norm_stderr\": 0.026915003011380154\n\ \ },\n \"harness|hendrycksTest-professional_accounting|5\": {\n \"\ acc\": 0.4219858156028369,\n \"acc_stderr\": 0.029462189233370593,\n \ \ \"acc_norm\": 0.4219858156028369,\n \"acc_norm_stderr\": 0.029462189233370593\n\ \ },\n \"harness|hendrycksTest-professional_law|5\": {\n \"acc\": 0.4178617992177314,\n\ \ \"acc_stderr\": 0.012596744108998565,\n \"acc_norm\": 0.4178617992177314,\n\ \ \"acc_norm_stderr\": 0.012596744108998565\n },\n \"harness|hendrycksTest-professional_medicine|5\"\ : {\n \"acc\": 0.5330882352941176,\n \"acc_stderr\": 0.030306257722468304,\n\ \ \"acc_norm\": 0.5330882352941176,\n \"acc_norm_stderr\": 0.030306257722468304\n\ \ },\n \"harness|hendrycksTest-professional_psychology|5\": {\n \"\ acc\": 0.5849673202614379,\n \"acc_stderr\": 0.01993362777685742,\n \ \ \"acc_norm\": 0.5849673202614379,\n \"acc_norm_stderr\": 0.01993362777685742\n\ \ },\n \"harness|hendrycksTest-public_relations|5\": {\n \"acc\": 0.6363636363636364,\n\ \ \"acc_stderr\": 0.046075820907199756,\n \"acc_norm\": 0.6363636363636364,\n\ \ \"acc_norm_stderr\": 0.046075820907199756\n },\n \"harness|hendrycksTest-security_studies|5\"\ : {\n \"acc\": 0.636734693877551,\n \"acc_stderr\": 0.030789051139030806,\n\ \ \"acc_norm\": 0.636734693877551,\n \"acc_norm_stderr\": 0.030789051139030806\n\ \ },\n \"harness|hendrycksTest-sociology|5\": {\n \"acc\": 0.7263681592039801,\n\ \ \"acc_stderr\": 0.031524391865554,\n \"acc_norm\": 0.7263681592039801,\n\ \ \"acc_norm_stderr\": 0.031524391865554\n },\n \"harness|hendrycksTest-us_foreign_policy|5\"\ : {\n \"acc\": 0.83,\n \"acc_stderr\": 0.03775251680686371,\n \ \ \"acc_norm\": 0.83,\n \"acc_norm_stderr\": 0.03775251680686371\n \ \ },\n \"harness|hendrycksTest-virology|5\": {\n \"acc\": 0.4578313253012048,\n\ \ \"acc_stderr\": 0.038786267710023595,\n \"acc_norm\": 0.4578313253012048,\n\ \ \"acc_norm_stderr\": 0.038786267710023595\n },\n \"harness|hendrycksTest-world_religions|5\"\ : {\n \"acc\": 0.783625730994152,\n \"acc_stderr\": 0.03158149539338734,\n\ \ \"acc_norm\": 0.783625730994152,\n \"acc_norm_stderr\": 0.03158149539338734\n\ \ },\n \"harness|truthfulqa:mc|0\": {\n \"mc1\": 0.3598531211750306,\n\ \ \"mc1_stderr\": 0.016801860466677157,\n \"mc2\": 0.5015608436175973,\n\ \ \"mc2_stderr\": 0.015270015874980385\n }\n}\n```" repo_url: https://huggingface.co/yeontaek/llama-2-13B-ensemble-v1 leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|arc:challenge|25_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hellaswag|10_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-24T06:11:39.305449.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-management|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-virology|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-24T06:11:39.305449.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_08_24T06_11_39.305449 path: - '**/details_harness|truthfulqa:mc|0_2023-08-24T06:11:39.305449.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-08-24T06:11:39.305449.parquet' --- # Dataset Card for Evaluation run of yeontaek/llama-2-13B-ensemble-v1 ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/yeontaek/llama-2-13B-ensemble-v1 - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [yeontaek/llama-2-13B-ensemble-v1](https://huggingface.co/yeontaek/llama-2-13B-ensemble-v1) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 60 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v1", "harness_truthfulqa_mc_0", split="train") ``` ## Latest results These are the [latest results from run 2023-08-24T06:11:39.305449](https://huggingface.co/datasets/open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v1/blob/main/results_2023-08-24T06%3A11%3A39.305449.json) (note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "acc": 0.5771705795394726, "acc_stderr": 0.03407019951989153, "acc_norm": 0.5809029426034396, "acc_norm_stderr": 0.0340502352653076, "mc1": 0.3598531211750306, "mc1_stderr": 0.016801860466677157, "mc2": 0.5015608436175973, "mc2_stderr": 0.015270015874980385 }, "harness|arc:challenge|25": { "acc": 0.5998293515358362, "acc_stderr": 0.014317197787809169, "acc_norm": 0.6228668941979523, "acc_norm_stderr": 0.0141633668961926 }, "harness|hellaswag|10": { "acc": 0.6264688309101772, "acc_stderr": 0.004827526584889677, "acc_norm": 0.8236407090221072, "acc_norm_stderr": 0.0038034664560544717 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.32, "acc_stderr": 0.04688261722621504, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621504 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.45925925925925926, "acc_stderr": 0.04304979692464243, "acc_norm": 0.45925925925925926, "acc_norm_stderr": 0.04304979692464243 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5394736842105263, "acc_stderr": 0.04056242252249034, "acc_norm": 0.5394736842105263, "acc_norm_stderr": 0.04056242252249034 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.58, "acc_stderr": 0.049604496374885836, "acc_norm": 0.58, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6075471698113207, "acc_stderr": 0.03005258057955784, "acc_norm": 0.6075471698113207, "acc_norm_stderr": 0.03005258057955784 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6319444444444444, "acc_stderr": 0.040329990539607175, "acc_norm": 0.6319444444444444, "acc_norm_stderr": 0.040329990539607175 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.37, "acc_stderr": 0.048523658709391, "acc_norm": 0.37, "acc_norm_stderr": 0.048523658709391 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.34, "acc_stderr": 0.04760952285695236, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695236 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5491329479768786, "acc_stderr": 0.03794012674697031, "acc_norm": 0.5491329479768786, "acc_norm_stderr": 0.03794012674697031 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3235294117647059, "acc_stderr": 0.046550104113196177, "acc_norm": 0.3235294117647059, "acc_norm_stderr": 0.046550104113196177 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.46382978723404256, "acc_stderr": 0.032600385118357715, "acc_norm": 0.46382978723404256, "acc_norm_stderr": 0.032600385118357715 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.30701754385964913, "acc_stderr": 0.04339138322579861, "acc_norm": 0.30701754385964913, "acc_norm_stderr": 0.04339138322579861 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192118, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192118 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3306878306878307, "acc_stderr": 0.02422996529842507, "acc_norm": 0.3306878306878307, "acc_norm_stderr": 0.02422996529842507 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.35714285714285715, "acc_stderr": 0.042857142857142816, "acc_norm": 0.35714285714285715, "acc_norm_stderr": 0.042857142857142816 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6580645161290323, "acc_stderr": 0.026985289576552746, "acc_norm": 0.6580645161290323, "acc_norm_stderr": 0.026985289576552746 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.42857142857142855, "acc_stderr": 0.03481904844438803, "acc_norm": 0.42857142857142855, "acc_norm_stderr": 0.03481904844438803 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.59, "acc_stderr": 0.04943110704237102, "acc_norm": 0.59, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7090909090909091, "acc_stderr": 0.03546563019624336, "acc_norm": 0.7090909090909091, "acc_norm_stderr": 0.03546563019624336 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7424242424242424, "acc_stderr": 0.03115626951964683, "acc_norm": 0.7424242424242424, "acc_norm_stderr": 0.03115626951964683 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8238341968911918, "acc_stderr": 0.02749350424454806, "acc_norm": 0.8238341968911918, "acc_norm_stderr": 0.02749350424454806 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5692307692307692, "acc_stderr": 0.02510682066053975, "acc_norm": 0.5692307692307692, "acc_norm_stderr": 0.02510682066053975 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.2962962962962963, "acc_stderr": 0.02784081149587192, "acc_norm": 0.2962962962962963, "acc_norm_stderr": 0.02784081149587192 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6092436974789915, "acc_stderr": 0.03169380235712996, "acc_norm": 0.6092436974789915, "acc_norm_stderr": 0.03169380235712996 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.31788079470198677, "acc_stderr": 0.038020397601079024, "acc_norm": 0.31788079470198677, "acc_norm_stderr": 0.038020397601079024 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.7651376146788991, "acc_stderr": 0.018175110510343564, "acc_norm": 0.7651376146788991, "acc_norm_stderr": 0.018175110510343564 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.44907407407407407, "acc_stderr": 0.03392238405321617, "acc_norm": 0.44907407407407407, "acc_norm_stderr": 0.03392238405321617 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.7598039215686274, "acc_stderr": 0.02998373305591361, "acc_norm": 0.7598039215686274, "acc_norm_stderr": 0.02998373305591361 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.759493670886076, "acc_stderr": 0.027820781981149685, "acc_norm": 0.759493670886076, "acc_norm_stderr": 0.027820781981149685 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.672645739910314, "acc_stderr": 0.03149384670994131, "acc_norm": 0.672645739910314, "acc_norm_stderr": 0.03149384670994131 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.6793893129770993, "acc_stderr": 0.04093329229834278, "acc_norm": 0.6793893129770993, "acc_norm_stderr": 0.04093329229834278 }, "harness|hendrycksTest-international_law|5": { "acc": 0.7768595041322314, "acc_stderr": 0.03800754475228732, "acc_norm": 0.7768595041322314, "acc_norm_stderr": 0.03800754475228732 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.75, "acc_stderr": 0.04186091791394607, "acc_norm": 0.75, "acc_norm_stderr": 0.04186091791394607 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.7055214723926381, "acc_stderr": 0.03581165790474082, "acc_norm": 0.7055214723926381, "acc_norm_stderr": 0.03581165790474082 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.39285714285714285, "acc_stderr": 0.04635550135609976, "acc_norm": 0.39285714285714285, "acc_norm_stderr": 0.04635550135609976 }, "harness|hendrycksTest-management|5": { "acc": 0.7669902912621359, "acc_stderr": 0.04185832598928315, "acc_norm": 0.7669902912621359, "acc_norm_stderr": 0.04185832598928315 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8290598290598291, "acc_stderr": 0.024662496845209807, "acc_norm": 0.8290598290598291, "acc_norm_stderr": 0.024662496845209807 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.67, "acc_stderr": 0.04725815626252609, "acc_norm": 0.67, "acc_norm_stderr": 0.04725815626252609 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.776500638569604, "acc_stderr": 0.01489723522945071, "acc_norm": 0.776500638569604, "acc_norm_stderr": 0.01489723522945071 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.653179190751445, "acc_stderr": 0.025624723994030454, "acc_norm": 0.653179190751445, "acc_norm_stderr": 0.025624723994030454 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.4324022346368715, "acc_stderr": 0.01656897123354861, "acc_norm": 0.4324022346368715, "acc_norm_stderr": 0.01656897123354861 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.630718954248366, "acc_stderr": 0.02763417668960266, "acc_norm": 0.630718954248366, "acc_norm_stderr": 0.02763417668960266 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.6527331189710611, "acc_stderr": 0.027040745502307336, "acc_norm": 0.6527331189710611, "acc_norm_stderr": 0.027040745502307336 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.6265432098765432, "acc_stderr": 0.026915003011380154, "acc_norm": 0.6265432098765432, "acc_norm_stderr": 0.026915003011380154 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.4219858156028369, "acc_stderr": 0.029462189233370593, "acc_norm": 0.4219858156028369, "acc_norm_stderr": 0.029462189233370593 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.4178617992177314, "acc_stderr": 0.012596744108998565, "acc_norm": 0.4178617992177314, "acc_norm_stderr": 0.012596744108998565 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.5330882352941176, "acc_stderr": 0.030306257722468304, "acc_norm": 0.5330882352941176, "acc_norm_stderr": 0.030306257722468304 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.5849673202614379, "acc_stderr": 0.01993362777685742, "acc_norm": 0.5849673202614379, "acc_norm_stderr": 0.01993362777685742 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.6363636363636364, "acc_stderr": 0.046075820907199756, "acc_norm": 0.6363636363636364, "acc_norm_stderr": 0.046075820907199756 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.636734693877551, "acc_stderr": 0.030789051139030806, "acc_norm": 0.636734693877551, "acc_norm_stderr": 0.030789051139030806 }, "harness|hendrycksTest-sociology|5": { "acc": 0.7263681592039801, "acc_stderr": 0.031524391865554, "acc_norm": 0.7263681592039801, "acc_norm_stderr": 0.031524391865554 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.83, "acc_stderr": 0.03775251680686371, "acc_norm": 0.83, "acc_norm_stderr": 0.03775251680686371 }, "harness|hendrycksTest-virology|5": { "acc": 0.4578313253012048, "acc_stderr": 0.038786267710023595, "acc_norm": 0.4578313253012048, "acc_norm_stderr": 0.038786267710023595 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.783625730994152, "acc_stderr": 0.03158149539338734, "acc_norm": 0.783625730994152, "acc_norm_stderr": 0.03158149539338734 }, "harness|truthfulqa:mc|0": { "mc1": 0.3598531211750306, "mc1_stderr": 0.016801860466677157, "mc2": 0.5015608436175973, "mc2_stderr": 0.015270015874980385 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

该数据集是在对模型 yeontaek/llama-2-13B-ensemble-v1 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 60 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 额外的 "results" 配置存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v1", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-08-24T06:11:39.305449 运行的最新结果：

python { "all": { "acc": 0.5771705795394726, "acc_stderr": 0.03407019951989153, "acc_norm": 0.5809029426034396, "acc_norm_stderr": 0.0340502352653076, "mc1": 0.3598531211750306, "mc1_stderr": 0.016801860466677157, "mc2": 0.5015608436175973, "mc2_stderr": 0.015270015874980385 }, "harness|arc:challenge|25": { "acc": 0.5998293515358362, "acc_stderr": 0.014317197787809169, "acc_norm": 0.6228668941979523, "acc_norm_stderr": 0.0141633668961926 }, "harness|hellaswag|10": { "acc": 0.6264688309101772, "acc_stderr": 0.004827526584889677, "acc_norm": 0.8236407090221072, "acc_norm_stderr": 0.0038034664560544717 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.32, "acc_stderr": 0.04688261722621504, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621504 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.45925925925925926, "acc_stderr": 0.04304979692464243, "acc_norm": 0.45925925925925926, "acc_norm_stderr": 0.04304979692464243 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5394736842105263, "acc_stderr": 0.04056242252249034, "acc_norm": 0.5394736842105263, "acc_norm_stderr": 0.04056242252249034 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.58, "acc_stderr": 0.049604496374885836, "acc_norm": 0.58, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6075471698113207, "acc_stderr": 0.03005258057955784, "acc_norm": 0.6075471698113207, "acc_norm_stderr": 0.03005258057955784 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6319444444444444, "acc_stderr": 0.040329990539607175, "acc_norm": 0.6319444444444444, "acc_norm_stderr": 0.040329990539607175 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.37, "acc_stderr": 0.048523658709391, "acc_norm": 0.37, "acc_norm_stderr": 0.048523658709391 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.34, "acc_stderr": 0.04760952285695236, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695236 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5491329479768786, "acc_stderr": 0.03794012674697031, "acc_norm": 0.5491329479768786, "acc_norm_stderr": 0.03794012674697031 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3235294117647059, "acc_stderr": 0.046550104113196177, "acc_norm": 0.3235294117647059, "acc_norm_stderr": 0.046550104113196177 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.46382978723404256, "acc_stderr": 0.032600385118357715, "acc_norm": 0.46382978723404256, "acc_norm_stderr": 0.032600385118357715 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.30701754385964913, "acc_stderr": 0.04339138322579861, "acc_norm": 0.30701754385964913, "acc_norm_stderr": 0.04339138322579861 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192118, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192118 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3306878306878307, "acc_stderr": 0.02422996529842507, "acc_norm": 0.3306878306878307, "acc_norm_stderr": 0.02422996529842507 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.35714285714285715, "acc_stderr": 0.042857142857142816, "acc_norm": 0.35714285714285715, "acc_norm_stderr": 0.042857142857142816 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6580645161290323, "acc_stderr": 0.026985289576552746, "acc_norm": 0.6580645161290323, "acc_norm_stderr": 0.026985289576552746 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.42857142857142855, "acc_stderr": 0.03481904844438803, "acc_norm": 0.42857142857142855, "acc_norm_stderr": 0.03481904844438803 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.59, "acc_stderr": 0.04943110704237102, "acc_norm": 0.59, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7090909090909091, "acc_stderr": 0.03546563019624336, "acc_norm": 0.7090909090909091, "acc_norm_stderr": 0.03546563019624336 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7424242424242424, "acc_stderr": 0.03115626951964683, "acc_norm": 0.7424242424242424, "acc_norm_stderr": 0.03115626951964683 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8238341968911918, "acc_stderr": 0.02749350424454806, "acc_norm": 0.8238341968911918, "acc_norm_stderr": 0.02749350424454806 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5692307692307692, "acc_stderr": 0.02510682066053975, "acc_norm": 0.5692307692307692, "acc_norm_stderr": 0.02510682066053975 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.2962962962962963, "acc_stderr": 0.02784081149587192, "acc_norm": 0.296296

搜集汇总

数据集介绍

构建方式

该数据集源自Open LLM Leaderboard对模型yeontaek/llama-2-13B-ensemble-v1的自动化评估流程。在评估过程中，系统针对60个不同的评测任务分别生成对应的配置项，每个配置项下存储了该任务在单次运行中的详细结果。数据集共包含一次运行记录，每次运行的时间戳被用作该配置下的数据分割标识，而“train”分割则始终指向最新一次的评估结果。此外，还额外设立了一个名为“results”的配置，用于汇总所有任务的聚合指标，这些指标正是排行榜上展示的最终性能数据。

特点

该数据集的一个显著特征是其结构的高度模块化与版本化。六十个配置项对应六十个独立的评测任务，覆盖了从常识推理（如ARC Challenge、HellaSwag）到多学科知识（如HendrycksTest涵盖的数十个学科）以及事实性问答（如TruthfulQA）等广泛领域。每个任务的结果以Parquet格式文件存储，并依据运行时间戳进行分割，确保了历史结果的可追溯性与最新结果的便捷访问。这种设计使得研究者能够轻松追踪模型在不同时间点、不同任务上的性能演变。

使用方法

使用者可通过HuggingFace的datasets库加载该数据集。例如，加载特定任务（如harness_truthfulqa_mc_0）的详细结果时，只需指定相应的配置名称和分割标识（如“train”或具体时间戳）。数据集返回的结果包含各类准确率指标及其标准误差，便于进行深入的性能分析。对于希望复现排行榜结果或进行细粒度模型评估的研究人员，该数据集提供了标准化的接口与详尽的元数据，极大地简化了工作流程。

背景与挑战

背景概述

随着大语言模型（LLM）技术的迅猛发展，如何客观、全面地评估模型性能成为学术界与工业界共同关注的焦点。在此背景下，Hugging Face团队于2023年推出了Open LLM Leaderboard，旨在为开源大语言模型提供标准化的评测平台。该数据集是yeontaek/llama-2-13B-ensemble-v1模型在Leaderboard上的评测记录，由Hugging Face团队于2023年8月创建，核心研究问题在于通过多任务、多维度基准测试，量化该集成模型在常识推理、学术知识、事实一致性等方面的综合能力。该数据集不仅为模型开发者提供了详尽的性能反馈，也推动了社区对LLM评估标准化与透明化的探索，对后续模型迭代与对比研究具有重要参考价值。

当前挑战

该数据集所反映的核心挑战之一在于领域问题的复杂性：大语言模型需同时应对ARC挑战集、HellaSwag、TruthfulQA等数十项任务，涵盖推理、知识、事实性等多个维度，这种多任务评估对模型的泛化能力提出了极高要求。构建过程中，数据集面临的技术挑战包括：如何确保60个配置项下评测结果的一致性，如何处理不同任务间指标（如acc、mc1、mc2）的标准化对齐，以及如何通过时间戳分割管理多轮运行结果，避免数据混淆与版本冲突。此外，评测结果中部分学科（如高等数学、经济学）准确率偏低，揭示了模型在专业领域知识深度上的不足，也凸显了当前评测体系在覆盖广度与任务难度平衡上的持续挑战。

常用场景

经典使用场景

在大型语言模型的评估体系中，open-llm-leaderboard-old/details_yeontaek__llama-2-13B-ensemble-v1 数据集扮演着关键角色。它专为记录和复现模型在 Open LLM Leaderboard 上的评测结果而设计，涵盖了 ARC-Challenge、HellaSwag、MMLU（涵盖 57 个学科）以及 TruthfulQA 等经典基准任务。研究者可借助该数据集，系统性地分析 llama-2-13B-ensemble-v1 模型在不同难度和知识领域的表现，从而深入理解其推理、常识理解和事实一致性等核心能力，为模型性能的横向对比与纵向追踪提供了标准化数据支撑。

实际应用

在实际应用中，该数据集为模型选型与部署提供了决策依据。开发者可通过对比不同模型在同一任务集上的表现，筛选出最适合特定场景（如教育问答、医疗咨询）的模型版本。例如，llama-2-13B-ensemble-v1 在高中政府与政治（82.38%）和市场营销（82.91%）任务上表现优异，提示其在社会科学领域的应用潜力。此外，数据集中的标准误信息有助于评估模型性能的稳定性，辅助工业界在风险敏感场景中做出更审慎的模型选择。

衍生相关工作

该数据集衍生了一系列关于模型集成与评测优化的经典工作。基于其提供的细粒度评测结果，研究者开展了对模型集成策略（如权重融合、投票机制）效果的深入分析，推动了 ensemble 方法在大语言模型中的系统化应用。同时，该数据集催生了针对 MMLU 学科难度分布的研究，以及 TruthfulQA 中事实一致性指标的改进工作。这些衍生研究共同构建了从模型评测到性能提升的闭环，为后续模型架构创新与训练策略优化奠定了方法论基础。