RoomSpace

This dataset, associated with the IJCAI-24 paper "Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning", aims to enhance spatial reasoning evaluations in language models. Our benchmark encompasses a broad spectrum of qualitative spatial relationships, including topological, directional, and distance relations. These are presented with different viewing points, varied granularities, and density of relation constraints to mimic real-world complexities, promoting more accurate evaluation of language models' capabilities in spatial reasoning tasks.

本数据集与IJCAI-24论文《重构大语言模型（Large Language Model，LLM）的空间推理（Spatial Reasoning）评估：面向定性推理（Qualitative Reasoning）的真实世界仿真基准》相关，旨在优化大语言模型空间推理能力的评估流程。本基准涵盖了丰富的定性空间关系类型，包括拓扑关系、方向关系与距离关系。此类关系以差异化的观测视角、多样化的粒度层级以及疏密不一的关系约束形式呈现，用以模拟真实世界的复杂场景，进而更精准地评估大语言模型在空间推理任务中的实际能力。

搜集汇总

背景与挑战

背景概述

RoomSpace数据集是一个用于增强语言模型空间推理评估的基准，与IJCAI-24论文相关。它涵盖拓扑、方向和距离等多种定性空间关系，并通过不同视角、粒度和约束密度来模拟真实世界复杂性，以更准确地评估语言模型在空间推理任务中的能力。