open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch

--- pretty_name: Evaluation run of TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch](https://huggingface.co/TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 64 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2023-10-22T17:26:29.355241](https://huggingface.co/datasets/open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch/blob/main/results_2023-10-22T17-26-29.355241.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.07340604026845637,\n\ \ \"em_stderr\": 0.002670854825275824,\n \"f1\": 0.17929844798657668,\n\ \ \"f1_stderr\": 0.003077076785957565,\n \"acc\": 0.3816157872344754,\n\ \ \"acc_stderr\": 0.008008666093235974\n },\n \"harness|drop|3\": {\n\ \ \"em\": 0.07340604026845637,\n \"em_stderr\": 0.002670854825275824,\n\ \ \"f1\": 0.17929844798657668,\n \"f1_stderr\": 0.003077076785957565\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.018953752843062926,\n \ \ \"acc_stderr\": 0.0037560783410314743\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.744277821625888,\n \"acc_stderr\": 0.012261253845440474\n\ \ }\n}\n```" repo_url: https://huggingface.co/TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|arc:challenge|25_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-08-30T11:14:15.390107.parquet' - config_name: harness_drop_3 data_files: - split: 2023_10_22T17_26_29.355241 path: - '**/details_harness|drop|3_2023-10-22T17-26-29.355241.parquet' - split: latest path: - '**/details_harness|drop|3_2023-10-22T17-26-29.355241.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2023_10_22T17_26_29.355241 path: - '**/details_harness|gsm8k|5_2023-10-22T17-26-29.355241.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2023-10-22T17-26-29.355241.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hellaswag|10_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-30T11:14:15.390107.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-management|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-virology|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-30T11:14:15.390107.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_08_30T11_14_15.390107 path: - '**/details_harness|truthfulqa:mc|0_2023-08-30T11:14:15.390107.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-08-30T11:14:15.390107.parquet' - config_name: harness_winogrande_5 data_files: - split: 2023_10_22T17_26_29.355241 path: - '**/details_harness|winogrande|5_2023-10-22T17-26-29.355241.parquet' - split: latest path: - '**/details_harness|winogrande|5_2023-10-22T17-26-29.355241.parquet' - config_name: results data_files: - split: 2023_08_30T11_14_15.390107 path: - results_2023-08-30T11:14:15.390107.parquet - split: 2023_10_22T17_26_29.355241 path: - results_2023-10-22T17-26-29.355241.parquet - split: latest path: - results_2023-10-22T17-26-29.355241.parquet --- # Dataset Card for Evaluation run of TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch](https://huggingface.co/TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2023-10-22T17:26:29.355241](https://huggingface.co/datasets/open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch/blob/main/results_2023-10-22T17-26-29.355241.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "em": 0.07340604026845637, "em_stderr": 0.002670854825275824, "f1": 0.17929844798657668, "f1_stderr": 0.003077076785957565, "acc": 0.3816157872344754, "acc_stderr": 0.008008666093235974 }, "harness|drop|3": { "em": 0.07340604026845637, "em_stderr": 0.002670854825275824, "f1": 0.17929844798657668, "f1_stderr": 0.003077076785957565 }, "harness|gsm8k|5": { "acc": 0.018953752843062926, "acc_stderr": 0.0037560783410314743 }, "harness|winogrande|5": { "acc": 0.744277821625888, "acc_stderr": 0.012261253845440474 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

数据集来源

该数据集是在对模型 TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch 进行评估运行期间自动创建的，评估运行在 Open LLM Leaderboard 上进行。

数据集结构

数据集由64个配置组成，每个配置对应一个评估任务。数据集从2次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

额外配置

一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_TFLai__Ensemble5-Platypus2-13B-QLora-0.80-epoch", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-22T17:26:29.355241 运行 的最新结果： python { "all": { "em": 0.07340604026845637, "em_stderr": 0.002670854825275824, "f1": 0.17929844798657668, "f1_stderr": 0.003077076785957565, "acc": 0.3816157872344754, "acc_stderr": 0.008008666093235974 }, "harness|drop|3": { "em": 0.07340604026845637, "em_stderr": 0.002670854825275824, "f1": 0.17929844798657668, "f1_stderr": 0.003077076785957565 }, "harness|gsm8k|5": { "acc": 0.018953752843062926, "acc_stderr": 0.0037560783410314743 }, "harness|winogrande|5": { "acc": 0.744277821625888, "acc_stderr": 0.012261253845440474 } }

配置详情

以下是数据集的配置详情：

• config_name: harness_arc_challenge_25

  • split: 2023_08_30T11_14_15.390107
    • path: **/details_harness|arc:challenge|25_2023-08-30T11:14:15.390107.parquet
  • split: latest
    • path: **/details_harness|arc:challenge|25_2023-08-30T11:14:15.390107.parquet

• config_name: harness_drop_3

  • split: 2023_10_22T17_26_29.355241
    • path: **/details_harness|drop|3_2023-10-22T17-26-29.355241.parquet
  • split: latest
    • path: **/details_harness|drop|3_2023-10-22T17-26-29.355241.parquet

• config_name: harness_gsm8k_5

  • split: 2023_10_22T17_26_29.355241
    • path: **/details_harness|gsm8k|5_2023-10-22T17-26-29.355241.parquet
  • split: latest
    • path: **/details_harness|gsm8k|5_2023-10-22T17-26-29.355241.parquet

• config_name: harness_hellaswag_10

  • split: 2023_08_30T11_14_15.390107
    • path: **/details_harness|hellaswag|10_2023-08-30T11:14:15.390107.parquet
  • split: latest
    • path: **/details_harness|hellaswag|10_2023-08-30T11:14:15.390107.parquet

• config_name: harness_hendrycksTest_5

  • split: 2023_08_30T11_14_15.390107
    • path:
      • **/details_harness|hendrycksTest-abstract_algebra|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-international_law|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-management|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-marketing|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2023-08-30T11:14:15.390107.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2023-08-30T11:14:15.390107.parquet

搜集汇总

数据集介绍

构建方式

在大规模语言模型评估的学术背景下，该数据集是围绕TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch模型在Open LLM Leaderboard上的评估过程自动生成的。数据集构建依托于HuggingFace的评估框架，通过两次独立运行，将每次运行的结果以时间戳为标识，分别存储为独立的split。每个split对应一个评估任务的配置，共计64个配置，覆盖了如ARC挑战赛、DROP、GSM8K、HellaSwag及HendrycksTest等多样化任务。此外，一个名为“results”的额外配置用于聚合所有运行的综合指标，为模型性能的量化分析提供结构化支撑。

特点

该数据集的核心特点在于其精细化的任务分解与时间维度追踪能力。它由64个独立配置组成，每个配置精确对应一个评估任务，如winogrande或gsm8k，确保了任务级别的细粒度数据访问。数据集的split设计独具匠心，每次评估运行都生成一个以时间戳命名的split，而“train”split则始终指向最新结果，实现了历史版本与最新状态的并行管理。这种架构不仅便于研究者回溯不同时间点的模型表现，还通过“results”配置提供了聚合性指标，如精确匹配率、F1分数和准确率及其标准误差，全面反映模型在多项基准上的综合能力。

使用方法

使用该数据集时，研究者可借助HuggingFace的datasets库进行高效加载。通过load_dataset函数，指定数据集名称和目标任务配置（如“harness_winogrande_5”），即可获取相应split的数据。split参数支持选择特定时间戳的运行结果或默认的“train”split以获取最新数据。数据以parquet格式存储，便于与pandas等工具无缝集成。对于需要分析模型在多任务上整体表现的场景，可直接访问“results”配置，其中包含了所有评估指标的聚合结果，适用于性能对比与趋势分析。

背景与挑战

背景概述

随着大语言模型（LLM）的飞速发展，如何系统、公正地评估其综合能力成为学界与工业界共同关注的核心议题。在此背景下，Hugging Face团队于2023年发起了Open LLM Leaderboard项目，旨在通过标准化的评测框架，对众多开源模型在多项自然语言理解与推理任务上的表现进行量化比较。该数据集正是为记录模型TFLai/Ensemble5-Platypus2-13B-QLora-0.80-epoch在Leaderboard上的评估结果而自动生成的，涵盖了从ARC挑战、DROP阅读理解、GSM8K数学推理到MMLU多学科知识等数十个任务的细粒度性能指标。作为Leaderboard生态的重要组成部分，该数据集不仅为研究者提供了模型间横向对比的可靠依据，更推动了开源LLM评估范式的透明化与可复现性。

当前挑战

该数据集所解决的领域挑战在于，大语言模型的能力评估长期缺乏统一、多维度且可扩展的基准体系。传统评估往往局限于单一任务或小规模测试集，难以全面反映模型在复杂推理、常识理解及专业领域知识等方面的真实水平。构建过程中，团队面临多重困难：首先，需整合来自不同来源、格式各异的评测任务（如HellaSwag、WinoGrande等），并确保每个任务的数据划分与评估协议的一致性；其次，为应对模型迭代与评测版本更新的需求，数据集设计需支持多时间戳分片存储，以追溯不同运行阶段的性能演化；此外，海量评测结果的高效存储与索引（如Parquet格式与分层配置）也对数据工程提出了严峻考验。

常用场景

经典使用场景

在大型语言模型（LLM）评估领域，该数据集作为Open LLM Leaderboard评测体系的核心组件，为模型性能的标准化度量提供了关键支撑。其经典使用场景在于通过多任务基准测试框架，系统性地评估模型在常识推理（如ARC挑战）、数学推理（GSM8K）、文本理解（DROP）及常识知识（Winogrande）等维度上的表现。研究者可依据数据集中64个精细化配置，分别加载各任务的详细评测结果，从而实现对模型能力的多维度剖析与横向对比。

解决学术问题

该数据集有效解决了LLM评估中缺乏统一、可复现基准的学术难题。通过提供标准化的评测流程与细粒度的结果记录，它使研究者能够精准定位模型在特定任务上的优势与短板，例如在Winogrande任务上准确率达74.43%而在GSM8K上仅1.90%的显著差异，揭示了当前模型在数学推理与常识理解间的能力鸿沟。这一数据资源为探索模型泛化瓶颈、指导训练策略优化提供了实证依据，推动了LLM评估体系的规范化与透明化进程。

衍生相关工作

该数据集衍生了一系列关于模型评测方法论与性能优化的重要工作。研究者基于其多任务结果，提出了针对特定能力短板（如数学推理）的微调策略，催生了Platypus系列等专注特定领域的改进模型。此外，数据集的标准化格式与开放接口，促进了评测工具链（如LM Evaluation Harness）的迭代发展，并为后续研究如模型能力可解释性分析、跨任务迁移学习等提供了数据基石，形成了以Open LLM Leaderboard为核心的评测生态。