pbevan11/EQ-Bench
收藏资源简介：
--- license: mit language: - en pretty_name: EQ-Bench size_categories: - n<1K --- # EQ-Bench This is the EQ-Bench v2 English dataset, all credit to Samuel J. Paech. --- Title: `EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models` Abstract: https://arxiv.org/abs/2312.06281 EQ-Bench is a benchmark for language models designed to assess emotional intelligence. Why emotional intelligence? One reason is that it represents a subset of abilities that are important for the user experience, and which isn't explicitly tested by other benchmarks. Another reason is that it's not trivial to improve scores by fine tuning for the benchmark, which makes it harder to "game" the leaderboard. EQ-Bench is a little different from traditional psychometric tests. It uses a specific question format, in which the subject has to read a dialogue then rate the intensity of possible emotional responses of one of the characters. Every question is interpretative and assesses the ability to predict the magnitude of the 4 presented emotions. The test is graded without the need for a judge (so there is no length bias). It's cheap to run (only 171 questions), and produces results that correlate strongly with human preference (Arena ELO) and multi-domain benchmarks like MMLU. Homepage: https://eqbench.com/ ### Citation ```bibtex @misc{paech2023eqbench, title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models}, author={Samuel J. Paech}, year={2023}, eprint={2312.06281}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```
许可证：MIT协议 语言：英语 美观名称：EQ-Bench 样本规模类别：少于1000条 --- # EQ-Bench 本数据集为EQ-Bench v2英语版数据集，所有荣誉归于Samuel J. Paech。 --- ### 论文标题：《EQ-Bench：面向大语言模型（Large Language Model, LLM）的情感智能基准测试》 摘要链接：https://arxiv.org/abs/2312.06281 EQ-Bench是一款专为评估大语言模型情感智能而设计的基准测试套件。 为何选择情感智能？其一，该能力属于对用户体验至关重要的能力子集，且未被其他基准测试明确覆盖；其二，通过针对该基准进行微调以提升测试分数并非易事，这使得测试排行榜更难被“刷分”。 EQ-Bench与传统心理测量测试略有不同。它采用特定的提问格式：受试者需先阅读一段对话，随后对其中某一角色可能产生的情绪反应强度进行评级。每道题目均为阐释性问题，旨在评估模型预测给定4种情绪强度的能力。该测试无需人工评委即可完成评分（因此不存在长度偏差），运行成本低廉（仅包含171道题目），且生成的结果与人类偏好评分（Arena ELO）及多领域基准测试（如MMLU）具有强相关性。 官方主页：https://eqbench.com/ ### 引用 bibtex @misc{paech2023eqbench, title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models}, author={Samuel J. Paech}, year={2023}, eprint={2312.06281}, archivePrefix={arXiv}, primaryClass={cs.CL} }
EQ-Bench
概述
EQ-Bench是一个用于评估大型语言模型情感智能的基准测试。该数据集由Samuel J. Paech创建，旨在测试语言模型在情感智能方面的能力。
特点
- 情感智能测试：EQ-Bench专注于评估语言模型的情感智能，这是用户体验中重要的一部分，且在其他基准测试中未被明确测试。
- 难以通过微调提高分数：该基准测试设计使得通过微调来提高分数变得困难，从而难以“操纵”排行榜。
- 独特的测试格式：使用特定的问答格式，要求模型阅读对话并评估其中一个角色的情感反应强度。每个问题都是解释性的，评估模型预测四种情感强度的能力。
- 无评委评分：测试无需评委评分，避免了长度偏差。
- 成本效益高：测试包含171个问题，成本低廉。
- 结果相关性强：测试结果与人类偏好（Arena ELO）和多领域基准测试（如MMLU）有很强的相关性。
引用
bibtex @misc{paech2023eqbench, title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models}, author={Samuel J. Paech}, year={2023}, eprint={2312.06281}, archivePrefix={arXiv}, primaryClass={cs.CL} }
