PISTOL
收藏资源简介:
PISTOL是一个专为大型语言模型(LLMs)结构化不学习评估设计的数据集编译管道。该管道由剑桥大学计算机科学与技术系和UCL人工智能中心联合开发,旨在通过合成知识图类型数据集来模拟真实世界的数据结构,从而支持多种场景下的结构化LLM不学习评估。PISTOL不依赖预训练模型生成数据,确保了数据独立性,适用于评估不同不学习方法在处理高度互联数据、批量数据或特定领域数据时的有效性和鲁棒性。该数据集的应用领域包括但不限于隐私保护、数据安全更新和商业需求,旨在解决因数据修改或删除带来的高成本和复杂性问题。
PISTOL is a dataset compilation pipeline specifically designed for structured non-learning evaluation of Large Language Models (LLMs). This pipeline was co-developed by the Department of Computer Science and Technology, University of Cambridge, and the AI Centre at University College London (UCL). It aims to simulate real-world data structures via synthetic knowledge graph-style datasets, thus supporting structured non-learning evaluation of LLMs across diverse scenarios. PISTOL eliminates reliance on pre-trained models for data generation, ensuring full data independence, and is applicable to evaluating the effectiveness and robustness of various non-learning methods when handling highly interconnected data, batch data, or domain-specific data. Its application scope includes but is not limited to privacy preservation, data security updates, and business requirements, with the goal of addressing the high costs and complexity arising from data modification or deletion.




