open-llm-leaderboard-old/details_beomi__Yi-Ko-6B
收藏Hugging Face2023-12-04 更新2024-06-22 收录
下载链接：
https://hf-mirror.com/datasets/open-llm-leaderboard-old/details_beomi__Yi-Ko-6B
下载链接
链接失效反馈官方服务：
资源简介：
---
pretty_name: Evaluation run of beomi/Yi-Ko-6B
dataset_summary: "Dataset automatically created during the evaluation run of model\
\ [beomi/Yi-Ko-6B](https://huggingface.co/beomi/Yi-Ko-6B) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\
\nThe dataset is composed of 63 configuration, each one coresponding to one of the\
\ evaluated task.\n\nThe dataset has been created from 1 run(s). Each run can be\
\ found as a specific split in each configuration, the split being named using the\
\ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\
\nAn additional configuration \"results\" store all the aggregated results of the\
\ run (and is used to compute and display the aggregated metrics on the [Open LLM\
\ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\
\nTo load the details from a run, you can for instance do the following:\n```python\n\
from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_beomi__Yi-Ko-6B\"\
,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\
These are the [latest results from run 2023-12-04T19:08:16.844680](https://huggingface.co/datasets/open-llm-leaderboard/details_beomi__Yi-Ko-6B/blob/main/results_2023-12-04T19-08-16.844680.json)(note\
\ that their might be results for other tasks in the repos if successive evals didn't\
\ cover the same tasks. You find each in the results and the \"latest\" split for\
\ each eval):\n\n```python\n{\n \"all\": {\n \"acc\": 0.5511585458954897,\n\
\ \"acc_stderr\": 0.03372606124688932,\n \"acc_norm\": 0.5592593520510756,\n\
\ \"acc_norm_stderr\": 0.034493739566800206,\n \"mc1\": 0.24479804161566707,\n\
\ \"mc1_stderr\": 0.015051869486715014,\n \"mc2\": 0.37094583310302975,\n\
\ \"mc2_stderr\": 0.013820249952756731\n },\n \"harness|arc:challenge|25\"\
: {\n \"acc\": 0.454778156996587,\n \"acc_stderr\": 0.014551507060836355,\n\
\ \"acc_norm\": 0.48890784982935154,\n \"acc_norm_stderr\": 0.01460779491401305\n\
\ },\n \"harness|hellaswag|10\": {\n \"acc\": 0.5488946425014938,\n\
\ \"acc_stderr\": 0.004965866098318173,\n \"acc_norm\": 0.7447719577773352,\n\
\ \"acc_norm_stderr\": 0.004350982826580606\n },\n \"harness|hendrycksTest-abstract_algebra|5\"\
: {\n \"acc\": 0.3,\n \"acc_stderr\": 0.046056618647183814,\n \
\ \"acc_norm\": 0.3,\n \"acc_norm_stderr\": 0.046056618647183814\n \
\ },\n \"harness|hendrycksTest-anatomy|5\": {\n \"acc\": 0.5185185185185185,\n\
\ \"acc_stderr\": 0.04316378599511324,\n \"acc_norm\": 0.5185185185185185,\n\
\ \"acc_norm_stderr\": 0.04316378599511324\n },\n \"harness|hendrycksTest-astronomy|5\"\
: {\n \"acc\": 0.5789473684210527,\n \"acc_stderr\": 0.04017901275981749,\n\
\ \"acc_norm\": 0.5789473684210527,\n \"acc_norm_stderr\": 0.04017901275981749\n\
\ },\n \"harness|hendrycksTest-business_ethics|5\": {\n \"acc\": 0.57,\n\
\ \"acc_stderr\": 0.04975698519562428,\n \"acc_norm\": 0.57,\n \
\ \"acc_norm_stderr\": 0.04975698519562428\n },\n \"harness|hendrycksTest-clinical_knowledge|5\"\
: {\n \"acc\": 0.5849056603773585,\n \"acc_stderr\": 0.03032594578928611,\n\
\ \"acc_norm\": 0.5849056603773585,\n \"acc_norm_stderr\": 0.03032594578928611\n\
\ },\n \"harness|hendrycksTest-college_biology|5\": {\n \"acc\": 0.6180555555555556,\n\
\ \"acc_stderr\": 0.040629907841466674,\n \"acc_norm\": 0.6180555555555556,\n\
\ \"acc_norm_stderr\": 0.040629907841466674\n },\n \"harness|hendrycksTest-college_chemistry|5\"\
: {\n \"acc\": 0.39,\n \"acc_stderr\": 0.049020713000019756,\n \
\ \"acc_norm\": 0.39,\n \"acc_norm_stderr\": 0.049020713000019756\n \
\ },\n \"harness|hendrycksTest-college_computer_science|5\": {\n \"\
acc\": 0.4,\n \"acc_stderr\": 0.049236596391733084,\n \"acc_norm\"\
: 0.4,\n \"acc_norm_stderr\": 0.049236596391733084\n },\n \"harness|hendrycksTest-college_mathematics|5\"\
: {\n \"acc\": 0.33,\n \"acc_stderr\": 0.047258156262526045,\n \
\ \"acc_norm\": 0.33,\n \"acc_norm_stderr\": 0.047258156262526045\n \
\ },\n \"harness|hendrycksTest-college_medicine|5\": {\n \"acc\": 0.5433526011560693,\n\
\ \"acc_stderr\": 0.03798106566014498,\n \"acc_norm\": 0.5433526011560693,\n\
\ \"acc_norm_stderr\": 0.03798106566014498\n },\n \"harness|hendrycksTest-college_physics|5\"\
: {\n \"acc\": 0.3137254901960784,\n \"acc_stderr\": 0.04617034827006718,\n\
\ \"acc_norm\": 0.3137254901960784,\n \"acc_norm_stderr\": 0.04617034827006718\n\
\ },\n \"harness|hendrycksTest-computer_security|5\": {\n \"acc\":\
\ 0.72,\n \"acc_stderr\": 0.04512608598542128,\n \"acc_norm\": 0.72,\n\
\ \"acc_norm_stderr\": 0.04512608598542128\n },\n \"harness|hendrycksTest-conceptual_physics|5\"\
: {\n \"acc\": 0.48936170212765956,\n \"acc_stderr\": 0.03267862331014063,\n\
\ \"acc_norm\": 0.48936170212765956,\n \"acc_norm_stderr\": 0.03267862331014063\n\
\ },\n \"harness|hendrycksTest-econometrics|5\": {\n \"acc\": 0.3333333333333333,\n\
\ \"acc_stderr\": 0.044346007015849245,\n \"acc_norm\": 0.3333333333333333,\n\
\ \"acc_norm_stderr\": 0.044346007015849245\n },\n \"harness|hendrycksTest-electrical_engineering|5\"\
: {\n \"acc\": 0.5517241379310345,\n \"acc_stderr\": 0.04144311810878152,\n\
\ \"acc_norm\": 0.5517241379310345,\n \"acc_norm_stderr\": 0.04144311810878152\n\
\ },\n \"harness|hendrycksTest-elementary_mathematics|5\": {\n \"acc\"\
: 0.35714285714285715,\n \"acc_stderr\": 0.024677862841332783,\n \"\
acc_norm\": 0.35714285714285715,\n \"acc_norm_stderr\": 0.024677862841332783\n\
\ },\n \"harness|hendrycksTest-formal_logic|5\": {\n \"acc\": 0.25396825396825395,\n\
\ \"acc_stderr\": 0.03893259610604674,\n \"acc_norm\": 0.25396825396825395,\n\
\ \"acc_norm_stderr\": 0.03893259610604674\n },\n \"harness|hendrycksTest-global_facts|5\"\
: {\n \"acc\": 0.32,\n \"acc_stderr\": 0.04688261722621503,\n \
\ \"acc_norm\": 0.32,\n \"acc_norm_stderr\": 0.04688261722621503\n \
\ },\n \"harness|hendrycksTest-high_school_biology|5\": {\n \"acc\": 0.635483870967742,\n\
\ \"acc_stderr\": 0.02737987122994325,\n \"acc_norm\": 0.635483870967742,\n\
\ \"acc_norm_stderr\": 0.02737987122994325\n },\n \"harness|hendrycksTest-high_school_chemistry|5\"\
: {\n \"acc\": 0.4729064039408867,\n \"acc_stderr\": 0.03512819077876106,\n\
\ \"acc_norm\": 0.4729064039408867,\n \"acc_norm_stderr\": 0.03512819077876106\n\
\ },\n \"harness|hendrycksTest-high_school_computer_science|5\": {\n \
\ \"acc\": 0.55,\n \"acc_stderr\": 0.04999999999999999,\n \"acc_norm\"\
: 0.55,\n \"acc_norm_stderr\": 0.04999999999999999\n },\n \"harness|hendrycksTest-high_school_european_history|5\"\
: {\n \"acc\": 0.6545454545454545,\n \"acc_stderr\": 0.037131580674819135,\n\
\ \"acc_norm\": 0.6545454545454545,\n \"acc_norm_stderr\": 0.037131580674819135\n\
\ },\n \"harness|hendrycksTest-high_school_geography|5\": {\n \"acc\"\
: 0.702020202020202,\n \"acc_stderr\": 0.03258630383836556,\n \"acc_norm\"\
: 0.702020202020202,\n \"acc_norm_stderr\": 0.03258630383836556\n },\n\
\ \"harness|hendrycksTest-high_school_government_and_politics|5\": {\n \
\ \"acc\": 0.8134715025906736,\n \"acc_stderr\": 0.028112091210117467,\n\
\ \"acc_norm\": 0.8134715025906736,\n \"acc_norm_stderr\": 0.028112091210117467\n\
\ },\n \"harness|hendrycksTest-high_school_macroeconomics|5\": {\n \
\ \"acc\": 0.5897435897435898,\n \"acc_stderr\": 0.02493931390694079,\n \
\ \"acc_norm\": 0.5897435897435898,\n \"acc_norm_stderr\": 0.02493931390694079\n\
\ },\n \"harness|hendrycksTest-high_school_mathematics|5\": {\n \"\
acc\": 0.3037037037037037,\n \"acc_stderr\": 0.028037929969114986,\n \
\ \"acc_norm\": 0.3037037037037037,\n \"acc_norm_stderr\": 0.028037929969114986\n\
\ },\n \"harness|hendrycksTest-high_school_microeconomics|5\": {\n \
\ \"acc\": 0.5966386554621849,\n \"acc_stderr\": 0.031866081214088314,\n\
\ \"acc_norm\": 0.5966386554621849,\n \"acc_norm_stderr\": 0.031866081214088314\n\
\ },\n \"harness|hendrycksTest-high_school_physics|5\": {\n \"acc\"\
: 0.36423841059602646,\n \"acc_stderr\": 0.03929111781242741,\n \"\
acc_norm\": 0.36423841059602646,\n \"acc_norm_stderr\": 0.03929111781242741\n\
\ },\n \"harness|hendrycksTest-high_school_psychology|5\": {\n \"acc\"\
: 0.7779816513761468,\n \"acc_stderr\": 0.017818849564796648,\n \"\
acc_norm\": 0.7779816513761468,\n \"acc_norm_stderr\": 0.017818849564796648\n\
\ },\n \"harness|hendrycksTest-high_school_statistics|5\": {\n \"acc\"\
: 0.4444444444444444,\n \"acc_stderr\": 0.03388857118502326,\n \"\
acc_norm\": 0.4444444444444444,\n \"acc_norm_stderr\": 0.03388857118502326\n\
\ },\n \"harness|hendrycksTest-high_school_us_history|5\": {\n \"acc\"\
: 0.7205882352941176,\n \"acc_stderr\": 0.031493281045079556,\n \"\
acc_norm\": 0.7205882352941176,\n \"acc_norm_stderr\": 0.031493281045079556\n\
\ },\n \"harness|hendrycksTest-high_school_world_history|5\": {\n \"\
acc\": 0.7172995780590717,\n \"acc_stderr\": 0.029312814153955924,\n \
\ \"acc_norm\": 0.7172995780590717,\n \"acc_norm_stderr\": 0.029312814153955924\n\
\ },\n \"harness|hendrycksTest-human_aging|5\": {\n \"acc\": 0.5964125560538116,\n\
\ \"acc_stderr\": 0.03292802819330314,\n \"acc_norm\": 0.5964125560538116,\n\
\ \"acc_norm_stderr\": 0.03292802819330314\n },\n \"harness|hendrycksTest-human_sexuality|5\"\
: {\n \"acc\": 0.6641221374045801,\n \"acc_stderr\": 0.041423137719966634,\n\
\ \"acc_norm\": 0.6641221374045801,\n \"acc_norm_stderr\": 0.041423137719966634\n\
\ },\n \"harness|hendrycksTest-international_law|5\": {\n \"acc\":\
\ 0.743801652892562,\n \"acc_stderr\": 0.03984979653302872,\n \"acc_norm\"\
: 0.743801652892562,\n \"acc_norm_stderr\": 0.03984979653302872\n },\n\
\ \"harness|hendrycksTest-jurisprudence|5\": {\n \"acc\": 0.6574074074074074,\n\
\ \"acc_stderr\": 0.045879047413018105,\n \"acc_norm\": 0.6574074074074074,\n\
\ \"acc_norm_stderr\": 0.045879047413018105\n },\n \"harness|hendrycksTest-logical_fallacies|5\"\
: {\n \"acc\": 0.6134969325153374,\n \"acc_stderr\": 0.03825825548848607,\n\
\ \"acc_norm\": 0.6134969325153374,\n \"acc_norm_stderr\": 0.03825825548848607\n\
\ },\n \"harness|hendrycksTest-machine_learning|5\": {\n \"acc\": 0.3482142857142857,\n\
\ \"acc_stderr\": 0.04521829902833587,\n \"acc_norm\": 0.3482142857142857,\n\
\ \"acc_norm_stderr\": 0.04521829902833587\n },\n \"harness|hendrycksTest-management|5\"\
: {\n \"acc\": 0.7281553398058253,\n \"acc_stderr\": 0.044052680241409216,\n\
\ \"acc_norm\": 0.7281553398058253,\n \"acc_norm_stderr\": 0.044052680241409216\n\
\ },\n \"harness|hendrycksTest-marketing|5\": {\n \"acc\": 0.782051282051282,\n\
\ \"acc_stderr\": 0.027046857630716667,\n \"acc_norm\": 0.782051282051282,\n\
\ \"acc_norm_stderr\": 0.027046857630716667\n },\n \"harness|hendrycksTest-medical_genetics|5\"\
: {\n \"acc\": 0.64,\n \"acc_stderr\": 0.04824181513244218,\n \
\ \"acc_norm\": 0.64,\n \"acc_norm_stderr\": 0.04824181513244218\n \
\ },\n \"harness|hendrycksTest-miscellaneous|5\": {\n \"acc\": 0.7177522349936143,\n\
\ \"acc_stderr\": 0.016095302969878537,\n \"acc_norm\": 0.7177522349936143,\n\
\ \"acc_norm_stderr\": 0.016095302969878537\n },\n \"harness|hendrycksTest-moral_disputes|5\"\
: {\n \"acc\": 0.6358381502890174,\n \"acc_stderr\": 0.025906632631016117,\n\
\ \"acc_norm\": 0.6358381502890174,\n \"acc_norm_stderr\": 0.025906632631016117\n\
\ },\n \"harness|hendrycksTest-moral_scenarios|5\": {\n \"acc\": 0.27262569832402234,\n\
\ \"acc_stderr\": 0.014893391735249624,\n \"acc_norm\": 0.27262569832402234,\n\
\ \"acc_norm_stderr\": 0.014893391735249624\n },\n \"harness|hendrycksTest-nutrition|5\"\
: {\n \"acc\": 0.5980392156862745,\n \"acc_stderr\": 0.02807415894760065,\n\
\ \"acc_norm\": 0.5980392156862745,\n \"acc_norm_stderr\": 0.02807415894760065\n\
\ },\n \"harness|hendrycksTest-philosophy|5\": {\n \"acc\": 0.6655948553054662,\n\
\ \"acc_stderr\": 0.026795422327893934,\n \"acc_norm\": 0.6655948553054662,\n\
\ \"acc_norm_stderr\": 0.026795422327893934\n },\n \"harness|hendrycksTest-prehistory|5\"\
: {\n \"acc\": 0.5864197530864198,\n \"acc_stderr\": 0.027402042040269966,\n\
\ \"acc_norm\": 0.5864197530864198,\n \"acc_norm_stderr\": 0.027402042040269966\n\
\ },\n \"harness|hendrycksTest-professional_accounting|5\": {\n \"\
acc\": 0.48226950354609927,\n \"acc_stderr\": 0.02980873964223777,\n \
\ \"acc_norm\": 0.48226950354609927,\n \"acc_norm_stderr\": 0.02980873964223777\n\
\ },\n \"harness|hendrycksTest-professional_law|5\": {\n \"acc\": 0.41395045632333766,\n\
\ \"acc_stderr\": 0.012579699631289265,\n \"acc_norm\": 0.41395045632333766,\n\
\ \"acc_norm_stderr\": 0.012579699631289265\n },\n \"harness|hendrycksTest-professional_medicine|5\"\
: {\n \"acc\": 0.5661764705882353,\n \"acc_stderr\": 0.030105636570016636,\n\
\ \"acc_norm\": 0.5661764705882353,\n \"acc_norm_stderr\": 0.030105636570016636\n\
\ },\n \"harness|hendrycksTest-professional_psychology|5\": {\n \"\
acc\": 0.5375816993464052,\n \"acc_stderr\": 0.020170614974969768,\n \
\ \"acc_norm\": 0.5375816993464052,\n \"acc_norm_stderr\": 0.020170614974969768\n\
\ },\n \"harness|hendrycksTest-public_relations|5\": {\n \"acc\": 0.6636363636363637,\n\
\ \"acc_stderr\": 0.04525393596302505,\n \"acc_norm\": 0.6636363636363637,\n\
\ \"acc_norm_stderr\": 0.04525393596302505\n },\n \"harness|hendrycksTest-security_studies|5\"\
: {\n \"acc\": 0.6285714285714286,\n \"acc_stderr\": 0.03093285879278985,\n\
\ \"acc_norm\": 0.6285714285714286,\n \"acc_norm_stderr\": 0.03093285879278985\n\
\ },\n \"harness|hendrycksTest-sociology|5\": {\n \"acc\": 0.7611940298507462,\n\
\ \"acc_stderr\": 0.030147775935409217,\n \"acc_norm\": 0.7611940298507462,\n\
\ \"acc_norm_stderr\": 0.030147775935409217\n },\n \"harness|hendrycksTest-us_foreign_policy|5\"\
: {\n \"acc\": 0.79,\n \"acc_stderr\": 0.040936018074033256,\n \
\ \"acc_norm\": 0.79,\n \"acc_norm_stderr\": 0.040936018074033256\n \
\ },\n \"harness|hendrycksTest-virology|5\": {\n \"acc\": 0.4397590361445783,\n\
\ \"acc_stderr\": 0.03864139923699121,\n \"acc_norm\": 0.4397590361445783,\n\
\ \"acc_norm_stderr\": 0.03864139923699121\n },\n \"harness|hendrycksTest-world_religions|5\"\
: {\n \"acc\": 0.7134502923976608,\n \"acc_stderr\": 0.03467826685703826,\n\
\ \"acc_norm\": 0.7134502923976608,\n \"acc_norm_stderr\": 0.03467826685703826\n\
\ },\n \"harness|truthfulqa:mc|0\": {\n \"mc1\": 0.24479804161566707,\n\
\ \"mc1_stderr\": 0.015051869486715014,\n \"mc2\": 0.37094583310302975,\n\
\ \"mc2_stderr\": 0.013820249952756731\n },\n \"harness|winogrande|5\"\
: {\n \"acc\": 0.7292817679558011,\n \"acc_stderr\": 0.012487904760626304\n\
\ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.12509476876421532,\n \
\ \"acc_stderr\": 0.009112601439849625\n }\n}\n```"
repo_url: https://huggingface.co/beomi/Yi-Ko-6B
leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
point_of_contact: clementine@hf.co
configs:
- config_name: harness_arc_challenge_25
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|arc:challenge|25_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|arc:challenge|25_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_gsm8k_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|gsm8k|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|gsm8k|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hellaswag_10
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hellaswag|10_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hellaswag|10_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-management|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-virology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-management|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-virology|5_2023-12-04T19-08-16.844680.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_abstract_algebra_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_anatomy_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-anatomy|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-anatomy|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_astronomy_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-astronomy|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-astronomy|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_business_ethics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_clinical_knowledge_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_biology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_biology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_biology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_chemistry_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_computer_science_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_mathematics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_medicine_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_college_physics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-college_physics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_physics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_computer_security_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-computer_security|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-computer_security|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_conceptual_physics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_econometrics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-econometrics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-econometrics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_electrical_engineering_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_elementary_mathematics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_formal_logic_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_global_facts_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-global_facts|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-global_facts|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_biology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_chemistry_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_computer_science_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_european_history_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_geography_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_government_and_politics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_macroeconomics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_mathematics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_microeconomics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_physics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_psychology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_statistics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_us_history_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_high_school_world_history_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_human_aging_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-human_aging|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_aging|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_human_sexuality_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_international_law_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-international_law|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-international_law|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_jurisprudence_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_logical_fallacies_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_machine_learning_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_management_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-management|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-management|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_marketing_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-marketing|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-marketing|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_medical_genetics_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_miscellaneous_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_moral_disputes_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_moral_scenarios_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_nutrition_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-nutrition|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-nutrition|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_philosophy_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-philosophy|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-philosophy|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_prehistory_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-prehistory|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-prehistory|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_professional_accounting_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_professional_law_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-professional_law|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_law|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_professional_medicine_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_professional_psychology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_public_relations_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-public_relations|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-public_relations|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_security_studies_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-security_studies|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-security_studies|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_sociology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-sociology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-sociology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_us_foreign_policy_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_virology_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-virology|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-virology|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_hendrycksTest_world_religions_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|hendrycksTest-world_religions|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-world_religions|5_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_truthfulqa_mc_0
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|truthfulqa:mc|0_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|truthfulqa:mc|0_2023-12-04T19-08-16.844680.parquet'
- config_name: harness_winogrande_5
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- '**/details_harness|winogrande|5_2023-12-04T19-08-16.844680.parquet'
- split: latest
path:
- '**/details_harness|winogrande|5_2023-12-04T19-08-16.844680.parquet'
- config_name: results
data_files:
- split: 2023_12_04T19_08_16.844680
path:
- results_2023-12-04T19-08-16.844680.parquet
- split: latest
path:
- results_2023-12-04T19-08-16.844680.parquet
---
# Dataset Card for Evaluation run of beomi/Yi-Ko-6B
## Dataset Description
- **Homepage:**
- **Repository:** https://huggingface.co/beomi/Yi-Ko-6B
- **Paper:**
- **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
- **Point of Contact:** clementine@hf.co
### Dataset Summary
Dataset automatically created during the evaluation run of model [beomi/Yi-Ko-6B](https://huggingface.co/beomi/Yi-Ko-6B) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the aggregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).
To load the details from a run, you can for instance do the following:
```python
from datasets import load_dataset
data = load_dataset("open-llm-leaderboard/details_beomi__Yi-Ko-6B",
"harness_winogrande_5",
split="train")
```
## Latest results
These are the [latest results from run 2023-12-04T19:08:16.844680](https://huggingface.co/datasets/open-llm-leaderboard/details_beomi__Yi-Ko-6B/blob/main/results_2023-12-04T19-08-16.844680.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval):
```python
{
"all": {
"acc": 0.5511585458954897,
"acc_stderr": 0.03372606124688932,
"acc_norm": 0.5592593520510756,
"acc_norm_stderr": 0.034493739566800206,
"mc1": 0.24479804161566707,
"mc1_stderr": 0.015051869486715014,
"mc2": 0.37094583310302975,
"mc2_stderr": 0.013820249952756731
},
"harness|arc:challenge|25": {
"acc": 0.454778156996587,
"acc_stderr": 0.014551507060836355,
"acc_norm": 0.48890784982935154,
"acc_norm_stderr": 0.01460779491401305
},
"harness|hellaswag|10": {
"acc": 0.5488946425014938,
"acc_stderr": 0.004965866098318173,
"acc_norm": 0.7447719577773352,
"acc_norm_stderr": 0.004350982826580606
},
"harness|hendrycksTest-abstract_algebra|5": {
"acc": 0.3,
"acc_stderr": 0.046056618647183814,
"acc_norm": 0.3,
"acc_norm_stderr": 0.046056618647183814
},
"harness|hendrycksTest-anatomy|5": {
"acc": 0.5185185185185185,
"acc_stderr": 0.04316378599511324,
"acc_norm": 0.5185185185185185,
"acc_norm_stderr": 0.04316378599511324
},
"harness|hendrycksTest-astronomy|5": {
"acc": 0.5789473684210527,
"acc_stderr": 0.04017901275981749,
"acc_norm": 0.5789473684210527,
"acc_norm_stderr": 0.04017901275981749
},
"harness|hendrycksTest-business_ethics|5": {
"acc": 0.57,
"acc_stderr": 0.04975698519562428,
"acc_norm": 0.57,
"acc_norm_stderr": 0.04975698519562428
},
"harness|hendrycksTest-clinical_knowledge|5": {
"acc": 0.5849056603773585,
"acc_stderr": 0.03032594578928611,
"acc_norm": 0.5849056603773585,
"acc_norm_stderr": 0.03032594578928611
},
"harness|hendrycksTest-college_biology|5": {
"acc": 0.6180555555555556,
"acc_stderr": 0.040629907841466674,
"acc_norm": 0.6180555555555556,
"acc_norm_stderr": 0.040629907841466674
},
"harness|hendrycksTest-college_chemistry|5": {
"acc": 0.39,
"acc_stderr": 0.049020713000019756,
"acc_norm": 0.39,
"acc_norm_stderr": 0.049020713000019756
},
"harness|hendrycksTest-college_computer_science|5": {
"acc": 0.4,
"acc_stderr": 0.049236596391733084,
"acc_norm": 0.4,
"acc_norm_stderr": 0.049236596391733084
},
"harness|hendrycksTest-college_mathematics|5": {
"acc": 0.33,
"acc_stderr": 0.047258156262526045,
"acc_norm": 0.33,
"acc_norm_stderr": 0.047258156262526045
},
"harness|hendrycksTest-college_medicine|5": {
"acc": 0.5433526011560693,
"acc_stderr": 0.03798106566014498,
"acc_norm": 0.5433526011560693,
"acc_norm_stderr": 0.03798106566014498
},
"harness|hendrycksTest-college_physics|5": {
"acc": 0.3137254901960784,
"acc_stderr": 0.04617034827006718,
"acc_norm": 0.3137254901960784,
"acc_norm_stderr": 0.04617034827006718
},
"harness|hendrycksTest-computer_security|5": {
"acc": 0.72,
"acc_stderr": 0.04512608598542128,
"acc_norm": 0.72,
"acc_norm_stderr": 0.04512608598542128
},
"harness|hendrycksTest-conceptual_physics|5": {
"acc": 0.48936170212765956,
"acc_stderr": 0.03267862331014063,
"acc_norm": 0.48936170212765956,
"acc_norm_stderr": 0.03267862331014063
},
"harness|hendrycksTest-econometrics|5": {
"acc": 0.3333333333333333,
"acc_stderr": 0.044346007015849245,
"acc_norm": 0.3333333333333333,
"acc_norm_stderr": 0.044346007015849245
},
"harness|hendrycksTest-electrical_engineering|5": {
"acc": 0.5517241379310345,
"acc_stderr": 0.04144311810878152,
"acc_norm": 0.5517241379310345,
"acc_norm_stderr": 0.04144311810878152
},
"harness|hendrycksTest-elementary_mathematics|5": {
"acc": 0.35714285714285715,
"acc_stderr": 0.024677862841332783,
"acc_norm": 0.35714285714285715,
"acc_norm_stderr": 0.024677862841332783
},
"harness|hendrycksTest-formal_logic|5": {
"acc": 0.25396825396825395,
"acc_stderr": 0.03893259610604674,
"acc_norm": 0.25396825396825395,
"acc_norm_stderr": 0.03893259610604674
},
"harness|hendrycksTest-global_facts|5": {
"acc": 0.32,
"acc_stderr": 0.04688261722621503,
"acc_norm": 0.32,
"acc_norm_stderr": 0.04688261722621503
},
"harness|hendrycksTest-high_school_biology|5": {
"acc": 0.635483870967742,
"acc_stderr": 0.02737987122994325,
"acc_norm": 0.635483870967742,
"acc_norm_stderr": 0.02737987122994325
},
"harness|hendrycksTest-high_school_chemistry|5": {
"acc": 0.4729064039408867,
"acc_stderr": 0.03512819077876106,
"acc_norm": 0.4729064039408867,
"acc_norm_stderr": 0.03512819077876106
},
"harness|hendrycksTest-high_school_computer_science|5": {
"acc": 0.55,
"acc_stderr": 0.04999999999999999,
"acc_norm": 0.55,
"acc_norm_stderr": 0.04999999999999999
},
"harness|hendrycksTest-high_school_european_history|5": {
"acc": 0.6545454545454545,
"acc_stderr": 0.037131580674819135,
"acc_norm": 0.6545454545454545,
"acc_norm_stderr": 0.037131580674819135
},
"harness|hendrycksTest-high_school_geography|5": {
"acc": 0.702020202020202,
"acc_stderr": 0.03258630383836556,
"acc_norm": 0.702020202020202,
"acc_norm_stderr": 0.03258630383836556
},
"harness|hendrycksTest-high_school_government_and_politics|5": {
"acc": 0.8134715025906736,
"acc_stderr": 0.028112091210117467,
"acc_norm": 0.8134715025906736,
"acc_norm_stderr": 0.028112091210117467
},
"harness|hendrycksTest-high_school_macroeconomics|5": {
"acc": 0.5897435897435898,
"acc_stderr": 0.02493931390694079,
"acc_norm": 0.5897435897435898,
"acc_norm_stderr": 0.02493931390694079
},
"harness|hendrycksTest-high_school_mathematics|5": {
"acc": 0.3037037037037037,
"acc_stderr": 0.028037929969114986,
"acc_norm": 0.3037037037037037,
"acc_norm_stderr": 0.028037929969114986
},
"harness|hendrycksTest-high_school_microeconomics|5": {
"acc": 0.5966386554621849,
"acc_stderr": 0.031866081214088314,
"acc_norm": 0.5966386554621849,
"acc_norm_stderr": 0.031866081214088314
},
"harness|hendrycksTest-high_school_physics|5": {
"acc": 0.36423841059602646,
"acc_stderr": 0.03929111781242741,
"acc_norm": 0.36423841059602646,
"acc_norm_stderr": 0.03929111781242741
},
"harness|hendrycksTest-high_school_psychology|5": {
"acc": 0.7779816513761468,
"acc_stderr": 0.017818849564796648,
"acc_norm": 0.7779816513761468,
"acc_norm_stderr": 0.017818849564796648
},
"harness|hendrycksTest-high_school_statistics|5": {
"acc": 0.4444444444444444,
"acc_stderr": 0.03388857118502326,
"acc_norm": 0.4444444444444444,
"acc_norm_stderr": 0.03388857118502326
},
"harness|hendrycksTest-high_school_us_history|5": {
"acc": 0.7205882352941176,
"acc_stderr": 0.031493281045079556,
"acc_norm": 0.7205882352941176,
"acc_norm_stderr": 0.031493281045079556
},
"harness|hendrycksTest-high_school_world_history|5": {
"acc": 0.7172995780590717,
"acc_stderr": 0.029312814153955924,
"acc_norm": 0.7172995780590717,
"acc_norm_stderr": 0.029312814153955924
},
"harness|hendrycksTest-human_aging|5": {
"acc": 0.5964125560538116,
"acc_stderr": 0.03292802819330314,
"acc_norm": 0.5964125560538116,
"acc_norm_stderr": 0.03292802819330314
},
"harness|hendrycksTest-human_sexuality|5": {
"acc": 0.6641221374045801,
"acc_stderr": 0.041423137719966634,
"acc_norm": 0.6641221374045801,
"acc_norm_stderr": 0.041423137719966634
},
"harness|hendrycksTest-international_law|5": {
"acc": 0.743801652892562,
"acc_stderr": 0.03984979653302872,
"acc_norm": 0.743801652892562,
"acc_norm_stderr": 0.03984979653302872
},
"harness|hendrycksTest-jurisprudence|5": {
"acc": 0.6574074074074074,
"acc_stderr": 0.045879047413018105,
"acc_norm": 0.6574074074074074,
"acc_norm_stderr": 0.045879047413018105
},
"harness|hendrycksTest-logical_fallacies|5": {
"acc": 0.6134969325153374,
"acc_stderr": 0.03825825548848607,
"acc_norm": 0.6134969325153374,
"acc_norm_stderr": 0.03825825548848607
},
"harness|hendrycksTest-machine_learning|5": {
"acc": 0.3482142857142857,
"acc_stderr": 0.04521829902833587,
"acc_norm": 0.3482142857142857,
"acc_norm_stderr": 0.04521829902833587
},
"harness|hendrycksTest-management|5": {
"acc": 0.7281553398058253,
"acc_stderr": 0.044052680241409216,
"acc_norm": 0.7281553398058253,
"acc_norm_stderr": 0.044052680241409216
},
"harness|hendrycksTest-marketing|5": {
"acc": 0.782051282051282,
"acc_stderr": 0.027046857630716667,
"acc_norm": 0.782051282051282,
"acc_norm_stderr": 0.027046857630716667
},
"harness|hendrycksTest-medical_genetics|5": {
"acc": 0.64,
"acc_stderr": 0.04824181513244218,
"acc_norm": 0.64,
"acc_norm_stderr": 0.04824181513244218
},
"harness|hendrycksTest-miscellaneous|5": {
"acc": 0.7177522349936143,
"acc_stderr": 0.016095302969878537,
"acc_norm": 0.7177522349936143,
"acc_norm_stderr": 0.016095302969878537
},
"harness|hendrycksTest-moral_disputes|5": {
"acc": 0.6358381502890174,
"acc_stderr": 0.025906632631016117,
"acc_norm": 0.6358381502890174,
"acc_norm_stderr": 0.025906632631016117
},
"harness|hendrycksTest-moral_scenarios|5": {
"acc": 0.27262569832402234,
"acc_stderr": 0.014893391735249624,
"acc_norm": 0.27262569832402234,
"acc_norm_stderr": 0.014893391735249624
},
"harness|hendrycksTest-nutrition|5": {
"acc": 0.5980392156862745,
"acc_stderr": 0.02807415894760065,
"acc_norm": 0.5980392156862745,
"acc_norm_stderr": 0.02807415894760065
},
"harness|hendrycksTest-philosophy|5": {
"acc": 0.6655948553054662,
"acc_stderr": 0.026795422327893934,
"acc_norm": 0.6655948553054662,
"acc_norm_stderr": 0.026795422327893934
},
"harness|hendrycksTest-prehistory|5": {
"acc": 0.5864197530864198,
"acc_stderr": 0.027402042040269966,
"acc_norm": 0.5864197530864198,
"acc_norm_stderr": 0.027402042040269966
},
"harness|hendrycksTest-professional_accounting|5": {
"acc": 0.48226950354609927,
"acc_stderr": 0.02980873964223777,
"acc_norm": 0.48226950354609927,
"acc_norm_stderr": 0.02980873964223777
},
"harness|hendrycksTest-professional_law|5": {
"acc": 0.41395045632333766,
"acc_stderr": 0.012579699631289265,
"acc_norm": 0.41395045632333766,
"acc_norm_stderr": 0.012579699631289265
},
"harness|hendrycksTest-professional_medicine|5": {
"acc": 0.5661764705882353,
"acc_stderr": 0.030105636570016636,
"acc_norm": 0.5661764705882353,
"acc_norm_stderr": 0.030105636570016636
},
"harness|hendrycksTest-professional_psychology|5": {
"acc": 0.5375816993464052,
"acc_stderr": 0.020170614974969768,
"acc_norm": 0.5375816993464052,
"acc_norm_stderr": 0.020170614974969768
},
"harness|hendrycksTest-public_relations|5": {
"acc": 0.6636363636363637,
"acc_stderr": 0.04525393596302505,
"acc_norm": 0.6636363636363637,
"acc_norm_stderr": 0.04525393596302505
},
"harness|hendrycksTest-security_studies|5": {
"acc": 0.6285714285714286,
"acc_stderr": 0.03093285879278985,
"acc_norm": 0.6285714285714286,
"acc_norm_stderr": 0.03093285879278985
},
"harness|hendrycksTest-sociology|5": {
"acc": 0.7611940298507462,
"acc_stderr": 0.030147775935409217,
"acc_norm": 0.7611940298507462,
"acc_norm_stderr": 0.030147775935409217
},
"harness|hendrycksTest-us_foreign_policy|5": {
"acc": 0.79,
"acc_stderr": 0.040936018074033256,
"acc_norm": 0.79,
"acc_norm_stderr": 0.040936018074033256
},
"harness|hendrycksTest-virology|5": {
"acc": 0.4397590361445783,
"acc_stderr": 0.03864139923699121,
"acc_norm": 0.4397590361445783,
"acc_norm_stderr": 0.03864139923699121
},
"harness|hendrycksTest-world_religions|5": {
"acc": 0.7134502923976608,
"acc_stderr": 0.03467826685703826,
"acc_norm": 0.7134502923976608,
"acc_norm_stderr": 0.03467826685703826
},
"harness|truthfulqa:mc|0": {
"mc1": 0.24479804161566707,
"mc1_stderr": 0.015051869486715014,
"mc2": 0.37094583310302975,
"mc2_stderr": 0.013820249952756731
},
"harness|winogrande|5": {
"acc": 0.7292817679558011,
"acc_stderr": 0.012487904760626304
},
"harness|gsm8k|5": {
"acc": 0.12509476876421532,
"acc_stderr": 0.009112601439849625
}
}
```
### Supported Tasks and Leaderboards
[More Information Needed]
### Languages
[More Information Needed]
## Dataset Structure
### Data Instances
[More Information Needed]
### Data Fields
[More Information Needed]
### Data Splits
[More Information Needed]
## Dataset Creation
### Curation Rationale
[More Information Needed]
### Source Data
#### Initial Data Collection and Normalization
[More Information Needed]
#### Who are the source language producers?
[More Information Needed]
### Annotations
#### Annotation process
[More Information Needed]
#### Who are the annotators?
[More Information Needed]
### Personal and Sensitive Information
[More Information Needed]
## Considerations for Using the Data
### Social Impact of Dataset
[More Information Needed]
### Discussion of Biases
[More Information Needed]
### Other Known Limitations
[More Information Needed]
## Additional Information
### Dataset Curators
[More Information Needed]
### Licensing Information
[More Information Needed]
### Citation Information
[More Information Needed]
### Contributions
[More Information Needed]
原始信息汇总
数据集概述
数据集简介
该数据集是在对模型 beomi/Yi-Ko-6B 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。
数据集结构
- 配置数量：63个配置，每个配置对应一个评估任务。
- 创建来源：从1次运行中创建。每个运行在每个配置中作为一个特定的分割存在，分割名称使用运行的时间戳。
- 最新结果："train" 分割始终指向最新结果。
- 结果汇总：一个额外的配置 "results" 存储所有运行的汇总结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。
数据加载示例
python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_beomi__Yi-Ko-6B", "harness_winogrande_5", split="train")
最新结果
以下是 2023-12-04T19:08:16.844680 运行的最新结果：
python { "all": { "acc": 0.5511585458954897, "acc_stderr": 0.03372606124688932, "acc_norm": 0.5592593520510756, "acc_norm_stderr": 0.034493739566800206, "mc1": 0.24479804161566707, "mc1_stderr": 0.015051869486715014, "mc2": 0.37094583310302975, "mc2_stderr": 0.013820249952756731 }, "harness|arc:challenge|25": { "acc": 0.454778156996587, "acc_stderr": 0.014551507060836355, "acc_norm": 0.48890784982935154, "acc_norm_stderr": 0.01460779491401305 }, "harness|hellaswag|10": { "acc": 0.5488946425014938, "acc_stderr": 0.004965866098318173, "acc_norm": 0.7447719577773352, "acc_norm_stderr": 0.004350982826580606 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5185185185185185, "acc_stderr": 0.04316378599511324, "acc_norm": 0.5185185185185185, "acc_norm_stderr": 0.04316378599511324 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5789473684210527, "acc_stderr": 0.04017901275981749, "acc_norm": 0.5789473684210527, "acc_norm_stderr": 0.04017901275981749 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.57, "acc_stderr": 0.04975698519562428, "acc_norm": 0.57, "acc_norm_stderr": 0.04975698519562428 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5849056603773585, "acc_stderr": 0.03032594578928611, "acc_norm": 0.5849056603773585, "acc_norm_stderr": 0.03032594578928611 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6180555555555556, "acc_stderr": 0.040629907841466674, "acc_norm": 0.6180555555555556, "acc_norm_stderr": 0.040629907841466674 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.39, "acc_stderr": 0.049020713000019756, "acc_norm": 0.39, "acc_norm_stderr": 0.049020713000019756 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.33, "acc_stderr": 0.047258156262526045, "acc_norm": 0.33, "acc_norm_stderr": 0.047258156262526045 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5433526011560693, "acc_stderr": 0.03798106566014498, "acc_norm": 0.5433526011560693, "acc_norm_stderr": 0.03798106566014498 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3137254901960784, "acc_stderr": 0.04617034827006718, "acc_norm": 0.3137254901960784, "acc_norm_stderr": 0.04617034827006718 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.72, "acc_stderr": 0.04512608598542128, "acc_norm": 0.72, "acc_norm_stderr": 0.04512608598542128 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.48936170212765956, "acc_stderr": 0.03267862331014063, "acc_norm": 0.48936170212765956, "acc_norm_stderr": 0.03267862331014063 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.3333333333333333, "acc_stderr": 0.044346007015849245, "acc_norm": 0.3333333333333333, "acc_norm_stderr": 0.044346007015849245 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5517241379310345, "acc_stderr": 0.04144311810878152, "acc_norm": 0.5517241379310345, "acc_norm_stderr": 0.04144311810878152 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.35714285714285715, "acc_stderr": 0.024677862841332783, "acc_norm": 0.35714285714285715, "acc_norm_stderr": 0.024677862841332783 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.25396825396825395, "acc_stderr": 0.03893259610604674, "acc_norm": 0.25396825396825395, "acc_norm_stderr": 0.03893259610604674 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.32, "acc_stderr": 0.04688261722621503, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621503 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.635483870967742, "acc_stderr": 0.02737987122994325, "acc_norm": 0.635483870967742, "acc_norm_stderr": 0.02737987122994325 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.4729064039408867, "acc_stderr": 0.03512819077876106, "acc_norm": 0.4729064039408867, "acc_norm_stderr": 0.03512819077876106 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.55, "acc_stderr": 0.04999999999999999, "acc_norm": 0.55, "acc_norm_stderr": 0.04999999999999999 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.6545454545454545, "acc_stderr": 0.037131580674819135, "acc_norm": 0.6545454545454545, "acc_norm_stderr": 0.037131580674819135 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.702020202020202, "acc_stderr": 0.03258630383836556, "acc_norm": 0.702020202020202, "acc_norm_stderr": 0.03258630383836556 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8134715025906736, "acc_stderr": 0.028112091210117467, "acc_norm": 0.8134715025906736, "acc_norm_stderr": 0.028112091210117467 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5897435897435898, "acc_stderr": 0.02493931390694079, "acc_norm": 0.5897435897435898, "acc_norm_stderr": 0.02493931390694079 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3037037037037037, "acc_stderr": 0.028037929969114986, "acc_norm": 0.3037037037037037, "acc_norm_stderr": 0.0
搜集汇总
数据集介绍
构建方式
在大型语言模型评测领域，Open LLM Leaderboard 作为一个权威的基准平台，为模型性能的横向比较提供了标准化的评估框架。该数据集正是针对韩语模型 beomi/Yi-Ko-6B 在 Leaderboard 上的自动评估过程而生成的副产品。数据集构建方式如下：通过运行一次完整的评测流程，收集了模型在 63 个不同评测任务上的详细结果。每个任务对应一个独立的配置（configuration），而每次运行的结果则被存储为特定时间戳命名的 split 分片。所有配置下的 'train' split 统一指向最新一次运行的结果，而额外的 'results' 配置则汇总了全部聚合指标，用于 Leaderboard 上的最终展示与计算。
特点
该数据集展现出多维度的结构特性，其核心在于精细化的任务划分与版本管理。63 个配置分别对应 ARC-Challenge、HellaSwag、GSM8K、Winogrande、TruthfulQA 以及涵盖 57 个学科的 MMLU（HendrycksTest）等多样化评测基准。每个配置内部，数据以 Parquet 格式存储，按运行时间戳进行分片，确保了结果的可追溯性与历史版本的可复现性。这种设计不仅允许研究者回溯任意一次评测的原始细节，还通过 'latest' split 提供了对最新结果的便捷访问，从而在动态更新的 Leaderboard 生态中维持了数据的一致性与时效性。
使用方法
研究者可通过 Hugging Face Datasets 库便捷地加载该数据集。例如，欲获取模型在 Winogrande 任务上的最新评测详情，只需调用 `load_dataset("open-llm-leaderboard/details_beomi__Yi-Ko-6B", "harness_winogrande_5", split="train")` 即可。若需回溯历史版本，可将 split 参数替换为具体的运行时间戳，如 `2023_12_04T19_08_16.844680`。这种灵活的接口设计使得对模型性能的深度分析、跨时间维度的比较以及评测结果的复现工作变得高效而直观，极大便利了自然语言处理社区对模型能力的系统性评估。
背景与挑战
背景概述
随着大语言模型技术的迅猛发展，如何系统性地评估模型在多样化任务上的表现成为学术界与工业界共同关注的焦点。Open LLM Leaderboard作为HuggingFace社区推出的重要基准平台，旨在通过标准化评测流程衡量开源语言模型的综合能力。该数据集于2023年12月由HuggingFace团队主导创建，专门用于记录模型beomi/Yi-Ko-6B在63个配置任务上的评测结果，涵盖常识推理、数学计算、多领域知识问答等维度。作为韩国社区针对Yi-6B模型进行韩语优化的产物，Yi-Ko-6B的评测数据为研究多语言大模型的能力迁移与领域适配提供了关键参考，推动了非英语语言模型评估体系的完善。
当前挑战
该数据集面临的挑战主要体现在两个层面。在领域问题层面，大语言模型评估长期受困于任务多样性不足与评测指标单一化的困境，现有基准难以全面反映模型在复杂推理、事实性知识获取及鲁棒性等维度的真实能力。例如在MMLU等学科测试中，模型在形式逻辑与初等数学等任务上的准确率不足35%，暴露出推理深度与知识覆盖的显著短板。在构建过程中，数据集需处理63个异构任务的评测结果整合问题，包括不同任务格式的标准化、跨时间戳运行结果的版本管理，以及如何通过单一结果配置文件（如results_2023-12-04T19-08-16.844680.json）高效存储多维度指标，这对数据架构的扩展性与可复现性提出了严苛要求。
常用场景
经典使用场景
在大型语言模型评估领域，Open LLM Leaderboard 的评估运行数据集为模型性能的量化分析提供了标准化基准。该数据集涵盖了从常识推理（如 HellaSwag、Winogrande）到数学问题求解（如 GSM8K）以及多领域知识问答（如 MMLU 中的 57 个学科任务）等 63 个配置项，每个配置对应一个独立评估任务。研究者可通过加载特定任务的详细结果，深入分析模型在推理、知识记忆和逻辑一致性等维度的表现，从而对模型能力形成全景式认知。该数据集最经典的使用场景是作为模型间横向比较的桥梁，通过统一的任务难度和评估协议，揭示不同架构、不同训练策略下模型的相对优劣。
衍生相关工作
该数据集衍生出了多个推动评估范式发展的经典工作。例如，基于该数据集的聚合结果，研究者提出了更稳健的模型排名方法，通过引入置信区间和误差分析来应对单次评估的随机波动。此外，该数据集的细粒度任务配置启发了针对特定能力（如多语言理解、代码生成）的扩展评估套件，如将 MMLU 的学科分类迁移至非英语语境。在模型分析层面，有工作利用该数据集中的错误案例分布，揭示了模型在低资源领域（如形式逻辑、大学数学）的系统性缺陷，进而催生了针对性数据增强与训练策略的改进研究。这些衍生工作共同构建了以标准化评估为核心的大模型研究生态。
数据集最近研究
最新研究方向
该数据集聚焦于韩语大语言模型Yi-Ko-6B在Open LLM Leaderboard上的多任务评估，覆盖了ARC挑战、HellaSwag、GSM8K、MMLU及TruthfulQA等63项基准测试。前沿研究方向集中在利用标准化评测框架系统度量模型在常识推理、数学解题、多领域知识掌握及事实一致性方面的表现。近期热点包括通过细粒度结果分析（如Winogrande反事实推理和HellaSwag场景理解）来揭示模型在零样本泛化中的优势与短板，尤其关注其在韩语文化语境下的知识迁移能力。该数据集的系统化评测流程为多语言大模型的公平比较提供了可复现的范式，对推动非英语语言模型的透明化与可控性发展具有深远意义。
以上内容由遇见数据集搜集并总结生成
