OpenToM

OpenToM是由伦敦国王学院和华为伦敦研究中心共同创建的数据集，旨在评估大型语言模型在理解他人心理状态（Theory-of-Mind）方面的能力。该数据集包含596个叙事故事，每个故事都设计有明确的性格特征和由角色意图触发的行动，以及针对物理和心理世界中角色心理状态的多样化问题。OpenToM通过更长、更清晰的叙事故事，以及包含角色个性和偏好的设计，挑战语言模型在模拟角色心理状态方面的能力，特别是在心理世界中的表现。数据集的应用领域主要集中在提升人工智能的社会智能，特别是在理解和模拟人类心理状态方面，以解决现有模型在处理复杂社会互动时的不足。OpenToM is a dataset co-created by King's College London and Huawei London Research Center, aiming to evaluate the performance of large language models (LLMs) in theory-of-mind tasks, i.e., understanding others' mental states. The dataset comprises 596 narrative stories, each crafted with explicit personality traits, actions motivated by characters' intentions, and diverse questions probing the characters' mental states across both physical and psychological domains. OpenToM challenges language models' capacity to simulate characters' mental states, particularly their performance in the psychological domain, via longer, more coherent narrative stories and designs integrating character personalities and preferences. The primary application focus of OpenToM is to enhance the social intelligence of artificial intelligence, especially in understanding and simulating human mental states, so as to address the shortcomings of existing AI models in handling complex social interactions.

搜集汇总

背景与挑战

背景概述

OpenToM是一个由伦敦国王学院和华为伦敦研究中心创建的数据集，包含596个叙事故事，旨在评估大型语言模型在理解他人心理状态方面的能力。该数据集通过设计明确的性格特征、角色意图和多样化问题，挑战模型在模拟角色心理状态，特别是在心理世界中的表现，以提升人工智能的社会智能。