open-llm-leaderboard-old/details_Weyaxi__Einstein-v5-v0.2-7B

该数据集是在模型 Weyaxi/Einstein-v5-v0.2-7B 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个被评估的任务。数据集包含一次运行的结果，每次运行在每个配置中表示为特定的分割，train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 Python 中的 datasets 库加载运行中的详细信息的示例。README 中还包含了特定运行的最新结果，显示了不同任务的各种准确性指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型Weyaxi/Einstein-v5-v0.2-7B在Open LLM Leaderboard上的运行过程中自动创建的。

数据集组成

数据集由63个配置组成，每个配置对应一个评估任务。数据集是从1次运行中创建的，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train"分割始终指向最新的结果。

额外配置

一个额外的配置"results"存储了所有运行的聚合结果，用于计算并在Open LLM Leaderboard上显示聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_Weyaxi__Einstein-v5-v0.2-7B", "harness_winogrande_5", split="train")

最新结果

以下是2024-03-27T21:09:37.228677运行的最新结果：

python { "all": { "acc": 0.612286564752706, "acc_stderr": 0.032839983165383065, "acc_norm": 0.6135779860343825, "acc_norm_stderr": 0.03350956178751591, "mc1": 0.3537331701346389, "mc1_stderr": 0.016737814358846147, "mc2": 0.5259333753586267, "mc2_stderr": 0.015070357329952046 }, "harness|arc:challenge|25": { "acc": 0.5691126279863481, "acc_stderr": 0.014471133392642463, "acc_norm": 0.6092150170648464, "acc_norm_stderr": 0.01425856388051378 }, "harness|hellaswag|10": { "acc": 0.6148177653853814, "acc_stderr": 0.004856437955719861, "acc_norm": 0.8098984266082454, "acc_norm_stderr": 0.003915792315457802 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5481481481481482, "acc_stderr": 0.042992689054808644, "acc_norm": 0.5481481481481482, "acc_norm_stderr": 0.042992689054808644 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6644736842105263, "acc_stderr": 0.03842498559395269, "acc_norm": 0.6644736842105263, "acc_norm_stderr": 0.03842498559395269 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.54, "acc_stderr": 0.05009082659620333, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.660377358490566, "acc_stderr": 0.029146904747798328, "acc_norm": 0.660377358490566, "acc_norm_stderr": 0.029146904747798328 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7083333333333334, "acc_stderr": 0.038009680605548594, "acc_norm": 0.7083333333333334, "acc_norm_stderr": 0.038009680605548594 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.47, "acc_stderr": 0.050161355804659205, "acc_norm": 0.47, "acc_norm_stderr": 0.050161355804659205 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.49, "acc_stderr": 0.05024183937956912, "acc_norm": 0.49, "acc_norm_stderr": 0.05024183937956912 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.39, "acc_stderr": 0.04902071300001975, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001975 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5606936416184971, "acc_stderr": 0.037842719328874674, "acc_norm": 0.5606936416184971, "acc_norm_stderr": 0.037842719328874674 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3235294117647059, "acc_stderr": 0.04655010411319619, "acc_norm": 0.3235294117647059, "acc_norm_stderr": 0.04655010411319619 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.76, "acc_stderr": 0.042923469599092816, "acc_norm": 0.76, "acc_norm_stderr": 0.042923469599092816 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5319148936170213, "acc_stderr": 0.03261936918467383, "acc_norm": 0.5319148936170213, "acc_norm_stderr": 0.03261936918467383 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4649122807017544, "acc_stderr": 0.046920083813689104, "acc_norm": 0.4649122807017544, "acc_norm_stderr": 0.046920083813689104 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5103448275862069, "acc_stderr": 0.04165774775728763, "acc_norm": 0.5103448275862069, "acc_norm_stderr": 0.04165774775728763 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.41534391534391535, "acc_stderr": 0.025379524910778394, "acc_norm": 0.41534391534391535, "acc_norm_stderr": 0.025379524910778394 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.42857142857142855, "acc_stderr": 0.04426266681379909, "acc_norm": 0.42857142857142855, "acc_norm_stderr": 0.04426266681379909 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.635483870967742, "acc_stderr": 0.027379871229943245, "acc_norm": 0.635483870967742, "acc_norm_stderr": 0.027379871229943245 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.4729064039408867, "acc_stderr": 0.03512819077876106, "acc_norm": 0.4729064039408867, "acc_norm_stderr": 0.03512819077876106 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.67, "acc_stderr": 0.047258156262526066, "acc_norm": 0.67, "acc_norm_stderr": 0.047258156262526066 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7757575757575758, "acc_stderr": 0.032568666616811015, "acc_norm": 0.7757575757575758, "acc_norm_stderr": 0.032568666616811015 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7626262626262627, "acc_stderr": 0.03031371053819889, "acc_norm": 0.7626262626262627, "acc_norm_stderr": 0.03031371053819889 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8549222797927462, "acc_stderr": 0.025416343096306422, "acc_norm": 0.8549222797927462, "acc_norm_stderr": 0.025416343096306422 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5974358974358974, "acc_stderr": 0.02486499515976775, "acc_norm": 0.5974358974358974, "acc_norm_stderr": 0.02486499515976775 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3, "acc_stderr": 0.027940457136228416, "acc_norm": 0.3, "acc_norm_stderr": 0.027940457136228416 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6302521008403361, "acc_stderr": 0.03135709599613591, "acc_norm": 0.6302521008403361, "acc_norm_stderr": 0.03135709599613591 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.31125827814569534, "acc_stderr": 0.03780445850526732, "acc_norm": 0.31125827814569534, "acc_norm_stderr": 0.03780445850526732 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.7944954128440367, "acc_stderr": 0.017324352325016015, "acc_norm": 0.7944954128440367, "acc_norm_stderr": 0.017324352325016015 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.44907407407407407, "acc_stderr": 0.03392238405321616, "acc_norm": 0.44907407407407407, "acc_norm_stderr": 0.03392238405321616 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.8235294117647058, "acc_stderr": 0.026756401538078966, "acc_norm": 0.8235294117647058, "acc_norm_stderr": 0.026756401538078966 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.7848101265822784, "acc_stderr": 0.026750826994676187, "acc_norm": 0.7848101265822784, "acc_norm_stderr": 0.026750826994676187 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.6547085201793722, "acc_stderr": 0.03191100192835794, "acc_norm": 0.6547085201793722, "acc_norm_stderr": 0.03191100192835794 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.7251908396946565, "acc_stderr": 0.03915345408847836, "acc_norm": 0.7251908396946565, "acc_norm_stderr": 0.03915345408847836 }, "harness|hendrycksTest-international_law|5": { "acc": 0.7603305785123967, "acc_stderr": 0.03896878985070416, "acc_norm": 0.7603305785123967, "acc_norm_stderr": 0.03896878985070416 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.7870370370370371, "acc_stderr": 0.0395783547198098, "acc_norm": 0.7870370370370371, "acc_norm_stderr": 0.0395783547198098 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.7239263803680982, "acc_stderr": 0.03512385283705048, "acc_norm": 0.7239263803680982, "acc_norm_stderr": 0.03512385283705048 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.4642857142857143, "acc_stderr": 0.04733667890053756, "acc_norm": 0.4642857142857143, "acc_norm_stderr": 0.04733667890053756 }, "harness|hendrycksTest-management|5": { "acc": 0.7864077669902912, "acc_stderr": 0.040580420156460344, "acc_norm": 0.7864077669902912, "acc_norm_stderr": 0.040580420156460344 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8547008547008547, "acc_stderr": 0.02308663508684141, "acc_norm": 0.8547008547008547, "acc_norm_stderr": 0.02308663508684141 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.7, "acc_stderr": 0.046056618647183814, "acc_norm": 0.7, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.7931034482758621, "acc_stderr": 0.014485656041669178, "acc_norm": 0.7931034482758621, "acc_norm_stderr": 0.014485656041669178 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.6965317919075145, "acc_stderr": 0.02475241196091721, "acc_norm": 0.6965317919075145, "acc_norm_stderr": 0.02475241196091721 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.2759776536312849, "acc_stderr": 0.014950103002475358, "acc_norm": 0.2759776536312849, "acc_norm_stderr": 0.014950103002475358 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.7254901960784313, "acc_stderr": 0.025553169991826517, "acc_norm": 0.7254901960784313, "acc_norm_stderr": 0.025553169991826517 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.6913183279742765, "acc_stderr": 0.026236965881153266, "acc_norm": 0.6913183279742765, "acc_norm_stderr": 0.026236965881153266 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.7129629629629629, "acc_stderr": 0.02517104191530968, "acc_norm": 0.7129629629629629, "acc_norm_stderr": 0.02517104191530968 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.44680851063829785, "acc_stderr": 0.029658235097666907, "acc_norm": 0.44680851063829785, "acc_norm_stderr": 0.029658235097666907 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.4621903520208605, "acc_stderr": 0.012733671880342507, "acc_norm": 0.4621903520208605, "acc_norm_stderr": 0.012733671880342507 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.5845588235294118, "acc_stderr": 0.029935342707877746, "acc_norm": 0.5845588235294118, "acc_norm_stderr": 0.029935342707877746 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.6568627450980392, "acc_stderr": 0.019206606848825362, "acc_norm": 0.6568627450980392, "acc_norm_stderr": 0.019206606848825362 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.6363636363636364, "acc_stderr": 0.04607582090719976, "acc_norm": 0.6363636363636364, "acc_norm_stderr": 0.04607582090719976 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.7346938775510204, "acc_stderr": 0.028263889943784593, "acc_norm": 0.7346938775510204, "acc_norm_stderr": 0.028263889943784593 }, "harness|hendrycksTest-sociology|5": { "acc": 0.7860696517412935, "acc_stderr": 0.028996909693328913, "acc_norm": 0.7860696517412935, "acc_norm_stderr": 0.028996909693328913 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.85, "acc_stderr": 0.03588702812826369, "acc_norm": 0.85, "acc_norm_stderr": 0.03588702812826369 }, "harness|hendrycksTest-virology|5": { "acc": 0.4879518072289157, "acc_stderr": 0.03891364495835821, "acc_norm": 0.4879518072289157, "acc_norm_stderr": 0.03891364495835821 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.7953216374269005, "acc_stderr": 0.030944459778533207, "acc_norm": 0.7953216374269005, "acc_norm_stderr": 0.030944459778533207 }, "harness|truthfulqa:mc|0": { "mc1": 0.3537331701346389, "mc1_stderr": 0.016737814358846147, "mc2": 0.5259333753586267, "mc2_stderr": 0.015070357329952046 }, "harness|winogrande|5": { "acc": 0.7868981846882399, "acc_stderr": 0.011508957690722769 }, "harness|gsm8k|5": { "acc": 0.5966641394996209, "acc_stderr": 0.01351265478181471 } }

配置详情

• config_name: harness_arc_challenge_25

  • 分割: 2024_03_27T21_09_37.228677
    • 路径: **/details_harness|arc:challenge|25_2024-03-27T21-09-37.228677.parquet
  • 分割: latest
    • 路径: **/details_harness|arc:challenge|25_2024-03-27T21-09-37.228677.parquet

• config_name: harness_gsm8k_5

  • 分割: 2024_03_27T21_09_37.228677
    • 路径: **/details_harness|gsm8k|5_2024-03-27T21-09-37.228677.parquet
  • 分割: latest
    • 路径: **/details_harness|gsm8k|5_2024-03-27T21-09-37.228677.parquet

• config_name: harness_hellaswag_10

  • 分割: 2024_03_27T21_09_37.228677
    • 路径: **/details_harness|hellaswag|10_2024-03-27T21-09-37.228677.parquet
  • 分割: latest
    • 路径: **/details_harness|hellaswag|10_2024-03-27T21-09-37.228677.parquet

• config_name: harness_hendrycksTest_5

  • 分割: 2024_03_27T21_09_37.228677
    • 路径:
      • **/details_harness|hendrycksTest-abstract_algebra|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-international_law|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-management|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-marketing|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-professional_law|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-professional_medicine|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-professional_psychology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-public_relations|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-security_studies|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-sociology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-us_foreign_policy|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-virology|5_2024-03-27T21-09-37.228677.parquet
      • **/details_harness|hendrycksTest-world_religions|5_2024-03-27T21-09-37.228677.parquet

搜集汇总

数据集介绍

构建方式

该数据集是在Open LLM Leaderboard框架下，针对模型Weyaxi/Einstein-v5-v0.2-7B进行自动化评估过程中生成的。构建方式基于运行评估任务时产生的详尽记录，涵盖了63个独立的配置，每个配置对应一个具体的评估任务。数据集由一次完整的运行产生，每次运行的结果以时间戳命名的分割形式存储，而'train'分割则始终指向最新一次评估的结果。此外，还包含一个名为'results'的配置，用于汇总所有评估任务的聚合指标，这些指标被用于计算和展示排行榜上的综合表现。

特点

该数据集的结构化特征鲜明，体现在其多任务、多分割的组织形式上。每个评估任务（如ARC挑战、HellaSwag、GSM8K等）均作为独立配置存在，便于研究者按需访问特定领域的性能细节。数据集的动态更新机制使得最新结果始终可通过'train'分割获取，而历史结果则被保留在时间戳命名的分割中，支持对模型性能演变的追溯分析。聚合结果配置则提供了跨任务的统一视角，涵盖了准确率、标准误差等关键统计量，为模型能力的全面评估提供了坚实的数据基础。

使用方法

使用该数据集时，研究者可通过HuggingFace的datasets库进行加载。具体而言，调用load_dataset函数并指定数据集名称、目标配置名称（如'harness_winogrande_5'）以及所需的分割（如'train'）即可获取特定任务的详细评估数据。对于需要分析多次运行结果的场景，可以通过选择不同时间戳命名的分割来访问历史数据。此外，'results'配置提供了所有任务的聚合指标，便于进行全局性的性能比较与综合评估。

背景与挑战

背景概述

随着大规模语言模型（LLM）在自然语言处理领域的广泛应用，如何系统且公正地评估其多维能力成为研究焦点。Open LLM Leaderboard由Hugging Face团队于2023年发起，旨在为开源社区提供一个标准化、透明化的模型性能竞技平台。该数据集作为对Weyaxi/Einstein-v5-v0.2-7B模型的一次完整评估记录，涵盖了从常识推理到数学求解、从多学科知识到伦理判断等63项任务的详细结果。其核心研究问题在于：如何通过一组精心设计的基准任务，客观衡量模型在零样本与少样本场景下的真实泛化能力。该数据集不仅为模型开发者提供了细粒度的性能反馈，也为后续模型优化与对比研究奠定了数据基础，在开源LLM评估领域具有重要影响力。

当前挑战

当前数据集面临的主要挑战源于评估体系的广度与深度之间的平衡。其一，所覆盖的63项任务虽涉及ARC、HellaSwag、GSM8K、MMLU等多个经典基准，但任务间难度差异悬殊，例如高中美国历史（acc达0.82）与抽象代数（acc仅0.30）之间的差距，暗示模型在知识型与推理型任务上的能力分布不均衡，如何设计更具区分度的任务集成为难题。其二，构建过程中需处理不同任务格式的异构性，如多项选择、生成式问答与常识推理等，统一评估管线（lm-evaluation-harness）的适配与结果对齐存在技术挑战。此外，单次运行结果可能受随机性影响，而数据集的版本管理（如每次评估仅保留最新分割）也增加了长期追踪模型性能演变的复杂性。

常用场景

经典使用场景

在大型语言模型评估领域，open-llm-leaderboard-old/details_Weyaxi__Einstein-v5-v0.2-7B 数据集被广泛用于基准测试与能力诊断。该数据集由 63 个任务配置组成，涵盖 ARC-Challenge、HellaSwag、GSM8K、TruthfulQA 以及涵盖 57 个学科的 MMLU 等经典评测项目。研究者可通过加载各任务的详细评估记录，对模型在常识推理、数学解题、知识问答与事实一致性等维度的表现进行细粒度分析。这一标准化评估流程为模型间横向对比提供了可靠依据，成为衡量开源大语言模型综合性能的重要工具。

衍生相关工作

该数据集衍生了一系列关于大语言模型评估方法论与能力分析的经典工作。基于其结构化结果，研究者提出了多维能力雷达图与任务难度分层等可视化分析方法，深化了对模型认知架构的理解。相关工作还包括利用该数据集的细粒度指标构建模型性能预测模型，以及探索不同评测任务之间的相关性网络。此外，该数据集催生了针对特定领域（如医学、法律）的专项评估基准，推动了大语言模型在垂直领域的适应性研究。这些衍生工作共同构建了从评测到优化的完整研究闭环，促进了开源大语言模型的持续进步。

数据集最近研究

最新研究方向

在大型语言模型（LLM）性能评估的前沿领域，基于开放排行榜的细粒度评测数据集正成为研究热点。该数据集记录了Weyaxi/Einstein-v5-v0.2-7B模型在Open LLM Leaderboard上的完整评估结果，涵盖ARC、HellaSwag、MMLU、TruthfulQA、Winogrande及GSM8K等63项多样化任务，其标准化评测框架为模型能力的横向对比提供了坚实基准。当前研究方向聚焦于利用此类结构化评测数据揭示模型的推理、常识与知识掌握短板，并推动领域内对模型泛化性与可靠性的深入探讨。该数据集的持续更新与公开共享，不仅加速了LLM性能优化迭代，更促进了社区对模型安全性与伦理边界的审视，对构建可信人工智能具有深远意义。