open-llm-leaderboard/details_Severian__ANIMA-Phi-Neptune-Mistral-7B-v3

该数据集是在评估模型Severian/ANIMA-Phi-Neptune-Mistral-7B-v3时自动创建的，用于在Open LLM Leaderboard上进行评估。数据集包含61个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果存储为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置，存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集创建背景

该数据集是在对模型 Severian/ANIMA-Phi-Neptune-Mistral-7B-v3 进行评估运行期间自动创建的，评估结果展示在 Open LLM Leaderboard 上。

数据集结构

• 配置数量：数据集包含 61 个配置，每个配置对应一个评估任务。
• 运行次数：数据集来自 1 次运行。每个运行结果作为一个特定的分割（split）存储在每个配置中，分割名称使用运行的时间戳。
• 最新结果："train" 分割始终指向最新的结果。
• 汇总结果：一个额外的配置 "results" 存储所有运行的汇总结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_Severian__ANIMA-Phi-Neptune-Mistral-7B-v3", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-10-11T01:16:32.937269 运行 的最新结果：

python { "all": { "acc": 0.5392786633208031, "acc_stderr": 0.03494779312823446, "acc_norm": 0.5431264898387953, "acc_norm_stderr": 0.03493217150757376, "mc1": 0.412484700122399, "mc1_stderr": 0.01723329939957122, "mc2": 0.5940288949043588, "mc2_stderr": 0.015208554054531144 }, "harness|arc:challenge|25": { "acc": 0.5401023890784983, "acc_stderr": 0.01456431885692485, "acc_norm": 0.568259385665529, "acc_norm_stderr": 0.014474591427196202 }, "harness|hellaswag|10": { "acc": 0.5893248356901015, "acc_stderr": 0.004909509538525159, "acc_norm": 0.7881896036646087, "acc_norm_stderr": 0.004077561349272391 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.29, "acc_stderr": 0.045604802157206845, "acc_norm": 0.29, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4666666666666667, "acc_stderr": 0.043097329010363554, "acc_norm": 0.4666666666666667, "acc_norm_stderr": 0.043097329010363554 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5263157894736842, "acc_stderr": 0.04063302731486671, "acc_norm": 0.5263157894736842, "acc_norm_stderr": 0.04063302731486671 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.04999999999999999, "acc_norm": 0.55, "acc_norm_stderr": 0.04999999999999999 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6377358490566037, "acc_stderr": 0.0295822451283843, "acc_norm": 0.6377358490566037, "acc_norm_stderr": 0.0295822451283843 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5347222222222222, "acc_stderr": 0.04171115858181618, "acc_norm": 0.5347222222222222, "acc_norm_stderr": 0.04171115858181618 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.39, "acc_stderr": 0.04902071300001975, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001975 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.54, "acc_stderr": 0.05009082659620333, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.33, "acc_stderr": 0.047258156262526045, "acc_norm": 0.33, "acc_norm_stderr": 0.047258156262526045 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5722543352601156, "acc_stderr": 0.037724468575180276, "acc_norm": 0.5722543352601156, "acc_norm_stderr": 0.037724468575180276 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.30392156862745096, "acc_stderr": 0.045766654032077615, "acc_norm": 0.30392156862745096, "acc_norm_stderr": 0.045766654032077615 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.65, "acc_stderr": 0.047937248544110196, "acc_norm": 0.65, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.4425531914893617, "acc_stderr": 0.03246956919789958, "acc_norm": 0.4425531914893617, "acc_norm_stderr": 0.03246956919789958 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.38596491228070173, "acc_stderr": 0.04579639422070435, "acc_norm": 0.38596491228070173, "acc_norm_stderr": 0.04579639422070435 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192117, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192117 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3439153439153439, "acc_stderr": 0.024464426625596433, "acc_norm": 0.3439153439153439, "acc_norm_stderr": 0.024464426625596433 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.3888888888888889, "acc_stderr": 0.0436031486007746, "acc_norm": 0.3888888888888889, "acc_norm_stderr": 0.0436031486007746 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6161290322580645, "acc_stderr": 0.02766618207553965, "acc_norm": 0.6161290322580645, "acc_norm_stderr": 0.02766618207553965 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.37438423645320196, "acc_stderr": 0.03405155380561952, "acc_norm": 0.37438423645320196, "acc_norm_stderr": 0.03405155380561952 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.56, "acc_stderr": 0.04988876515698589, "acc_norm": 0.56, "acc_norm_stderr": 0.04988876515698589 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.6545454545454545, "acc_stderr": 0.03713158067481913, "acc_norm": 0.6545454545454545, "acc_norm_stderr": 0.03713158067481913 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.6565656565656566, "acc_stderr": 0.03383201223244441, "acc_norm": 0.6565656565656566, "acc_norm_stderr": 0.03383201223244441 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7357512953367875, "acc_stderr": 0.03182155050916645, "acc_norm": 0.7357512953367875, "acc_norm_stderr": 0.03182155050916645 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.47692307692307695, "acc_stderr": 0.025323990861736118, "acc_norm": 0.47692307692307695, "acc_norm_stderr": 0.025323990861736118 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.2962962962962963, "acc_stderr": 0.027840811495871937, "acc_norm": 0.2962962962962963, "acc_norm_stderr": 0.027840811495871937 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.4957983193277311, "acc_stderr": 0.0324773433444811, "acc_norm": 0.4957983193277311, "acc_norm_stderr": 0.0324773433444811 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.2781456953642384, "acc_stderr": 0.03658603262763743, "acc_norm": 0.2781456953642384, "acc_norm_stderr": 0.03658603262763743 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.7229357798165138, "acc_stderr": 0.01918848259016953, "acc_norm": 0.7229357798165138, "acc_norm_stderr": 0.01918848259016953 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.39814814814814814, "acc_stderr": 0.033384734032074016, "acc_norm": 0.39814814814814814, "acc_norm_stderr": 0.033384734032074016 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.6519607843137255, "acc_stderr": 0.03343311240488418, "acc_norm": 0.6519607843137255, "acc_norm_stderr": 0.03343311240488418 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.7088607594936709, "acc_stderr": 0.02957160106575337, "acc_norm": 0.7088607594936709, "acc_norm_stderr": 0.02957160106575337 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.6098654708520179, "acc_stderr": 0.03273766725459157, "acc_norm": 0.6098654708520179, "acc_norm_stderr": 0.03273766725459157 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.6183206106870229, "acc_stderr": 0.0426073515764456, "acc_norm": 0.6183206106870229, "acc_norm_stderr": 0.0426073515764456 }, "harness|hendrycksTest-international_law|5": { "acc": 0.6446280991735537, "acc_stderr": 0.0436923632657398, "acc_norm": 0.6446280991735537, "acc_norm_stderr": 0.0436923632657398 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.6111111111111112, "acc_stderr": 0.04712821257426769, "acc_norm": 0.6111111111111112, "acc_norm_stderr": 0.04712821257426769 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.6196319018404908, "acc_stderr": 0.038142698932618374, "acc_norm": 0.6196319018404908, "acc_norm_stderr": 0.038142698932618374 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.39285714285714285, "acc_stderr": 0.04635550135609976, "acc_norm": 0.39285714285714285, "acc_norm_stderr": 0.04635550135609976 }, "harness|hendrycksTest-management|5": { "acc": 0.7184466019417476, "acc_stderr": 0.04453254836326468, "acc_norm": 0.7184466019417476, "acc_norm_stderr": 0.04453254836326468 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8290598290598291, "acc_stderr": 0.024662496845209818, "acc_norm": 0.8290598290598291, "acc_norm_stderr": 0.024662496845209818 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.62, "acc_stderr": 0.04878317312145633, "acc_norm": 0.62, "acc_norm_stderr": 0.04878317312145633 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.7088122605363985, "acc_stderr": 0.016246087069701407, "acc_norm": 0.7088122605363985, "acc_norm_stderr": 0.016246087069701407 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.5664739884393064, "acc_stderr": 0.02668013476167922, "acc_norm": 0.5664739884393064, "acc_norm_stderr": 0.02668013476167922 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.34972067039106147, "acc_stderr": 0.015949308790233638, "acc_norm": 0.34972067039106147, "acc_norm_stderr": 0.015949308790233638 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.5686274509803921, "acc_stderr": 0.028358956313423545, "acc_norm": 0.5686274509803921, "acc_norm_stderr": 0.028358956313423545 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.6302250803858521, "acc_stderr": 0.027417996705630998, "acc_norm": 0.6302250803858521, "acc_norm_stderr": 0.027417996705630998 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.6018518518518519, "acc_stderr": 0.027237415094592474, "acc_norm": 0.6018518518518519, "acc_norm_stderr": 0.027237415094592474 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.3723404255319149, "acc_stderr": 0.028838921471251455, "acc_norm": 0.3723404255319149, "acc_norm_stderr": 0.028838921471251455 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.38722294654498046, "acc_stderr": 0.012441155326854927, "acc_norm": 0.38722294654498046, "acc_norm_stderr": 0.012441155326854927 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.4852941176470588, "acc_stderr": 0.03035969707904611, "acc_norm": 0.4852941176470588, "acc_norm_stderr": 0.03035969707904611 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.5359477124183006, "acc_stderr": 0.02017548876548405, "acc_norm": 0.5359477124183006, "acc_norm_stderr": 0.02017548876548405 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.5909090909090909, "acc_stderr": 0.04709306978661895, "acc_norm": 0.5909090909090909, "acc_norm_stderr": 0.04709306978661895 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.6530612244897959, "acc_stderr": 0.030472526026726492, "acc_norm": 0.6530612244897959, "acc_norm_stderr": 0.030472526026726492 }, "harness|hendrycksTest-sociology|5": { "acc": 0.7164179104477612, "acc_stderr": 0.03187187537919796, "acc_norm": 0.7164179104477612, "acc_norm_stderr": 0.03187187537919796 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.79, "acc_stderr": 0.040936018074033256, "acc_norm": 0.79, "acc_norm_stderr": 0.040936018074033256 }, "harness|hendrycksTest-virology|5": { "acc": 0.4578313253012048, "acc_stderr": 0.0387862677100236, "acc_norm": 0.4578313253012048, "acc_norm_stderr": 0.0387862677100236 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.7251461988304093, "acc_stderr": 0.034240429246915824, "acc_norm": 0.7251461988304093, "acc_norm_stderr": 0.034240429246915824 }, "harness|truthfulqa:mc|0": { "mc1": 0.412484700122399, "mc1_stderr": 0.01723329939957122, "mc2": 0.5940288949043588, "mc2_stderr": 0.015208554054531144 } }

搜集汇总

数据集介绍

构建方式

该数据集是在Open LLM Leaderboard评估框架下，对Severian/ANIMA-Phi-Neptune-Mistral-7B-v3模型进行自动化评测过程中生成的。数据集由61个配置构成，每个配置对应一项被评估的任务。评估共执行一次，每次运行的结果以时间戳命名作为独立的分割（split），而“train”分割始终指向最新一次运行的结果。此外，一个名为“results”的额外配置存储了所有运行的聚合指标，用于在排行榜上计算和展示综合性能。

特点

数据集的核心特点在于其结构与评估流程的紧密耦合。每个配置涵盖一个特定任务（如ARC挑战、HellaSwag、MMLU等），并包含详细的评估指标（如准确率及其标准误）。数据以Parquet格式存储，支持高效加载。通过保留历史运行记录，数据集能够追踪模型性能的演变，同时“latest”分割确保了最新结果的便捷访问。这种设计不仅提供了细粒度的任务级分析，还实现了跨时间维度的性能对比。

使用方法

用户可通过HuggingFace的datasets库轻松加载数据。例如，使用`load_dataset("open-llm-leaderboard/details_Severian__ANIMA-Phi-Neptune-Mistral-7B-v3", "harness_truthfulqa_mc_0", split="train")`即可获取特定任务的最新细节。每个配置的名称与评估任务一一对应，用户可根据需求选择所需任务。加载后的数据可直接用于分析模型在各基准上的表现，或与其他模型的结果进行对比研究。

背景与挑战

背景概述

该数据集源于HuggingFace社区主导的Open LLM Leaderboard评估项目，创建于2023年10月，由HuggingFace团队（主要联系人Clementine）维护，旨在为大型语言模型提供标准化、可复现的性能基准。数据集记录了模型Severian/ANIMA-Phi-Neptune-Mistral-7B-v3在61个任务配置上的评估结果，覆盖ARC挑战集、HellaSwag常识推理、TruthfulQA真实性测试以及涵盖57个学科的MMLU基准。作为开放评估体系的核心组件，它推动了语言模型评测的透明化与社区协作，使研究者能够横向对比模型在多维度能力上的表现，对理解7B参数级别模型的推理、知识掌握与真实性具有重要参考价值。

当前挑战

该数据集面临的核心挑战包括：1) 领域问题层面，大型语言模型评测需应对多任务泛化性不足的困境，模型在ARC挑战集（准确率约54%）与MMLU数学子集（约30%）等任务上表现差异显著，凸显了单一模型在复杂推理与学科知识覆盖上的局限性。2) 构建过程中，数据集需从原始评估日志自动生成61个独立配置，并维护时间戳分片以追踪多次运行结果，这要求高效的数据管道设计来确保parquet文件与JSON结果的一致性，同时处理不同任务（如多项选择与生成式评测）的异构指标聚合，避免因版本迭代导致的兼容性问题。

常用场景

经典使用场景

该数据集作为Open LLM Leaderboard评估流程的自动化产物，核心用途在于系统性地记录模型Severian/ANIMA-Phi-Neptune-Mistral-7B-v3在61项任务上的细粒度表现。研究者可借助该数据集复现模型在ARC挑战、HellaSwag常识推理、TruthfulQA真实性检测以及涵盖57个学科的MMLU基准上的性能指标，为大型语言模型的横向对比提供标准化、可追溯的评测依据。

衍生相关工作

该数据集衍生的工作主要集中在评估方法论创新与模型诊断工具的开发。例如，研究者基于其细粒度结果构建了任务难度聚类分析，揭示了模型在形式逻辑与高等数学等任务上的系统性弱点；另有工作利用该数据集的多次运行记录，提出了基于标准误差的置信区间评估框架，以更严谨地比较不同模型的性能差异。这些成果进一步丰富了开放评估生态的技术纵深。

数据集最近研究

最新研究方向

近年来，大语言模型（LLM）的评估与基准测试成为自然语言处理领域的研究热点，Open LLM Leaderboard作为衡量模型综合能力的重要平台，催生了大量围绕模型性能评测的方法论创新。针对Severian/ANIMA-Phi-Neptune-Mistral-7B-v3模型，该数据集自动记录了其在61项任务上的多维度评估结果，涵盖推理、常识、专业知识及伦理判断等前沿方向。这一研究方向不仅聚焦于模型在ARC挑战赛、HellaSwag等经典基准上的表现，更延伸至MMLU等大规模多任务测试，揭示了模型在抽象代数、医学遗传学等细分领域的潜力与局限。数据集的结构化设计支持细粒度结果复现与分析，为模型迭代提供了关键反馈。该工作与当前LLM可解释性、公平性及鲁棒性研究紧密相连，推动了从单一指标到全景式评估范式的转变，对构建更可靠、更透明的AI系统具有深远意义。