The Impact of Evidence Order on LLMs in Fact Verification

This repository contains the data to replicate the results of the paper title: "The Impact of Evidence Order on LLMs in Fact Verification". Contents The repository includes the following files: filler.txt: Contains a dummy passage about llamas sourced from Wikipedia that was used as unrelated text for testing evidence placement within different context lengths (as referenced in Section 3.3 of the paper) hover_dataset.pqt: Contains the HOVER dataset that includes evidence from multiple Wikipedia articles per claim, creating multi-hop verification scenarios that mimic real-world fact-checking complexities feverous_dev.pqt: Contains the FEVEROUS dataset which expands upon the FEVER dataset by incorporating structured data and more complex claims requiring verification Related Code Repository The code used to analyze this data and reproduce the results presented in the paper can be found in a separate repository: https://anonymous.4open.science/r/Needle_In_Factchecking/README.md本仓库收录了可复现论文《证据顺序对大语言模型（LLM）事实核查任务的影响》研究结果的全部数据集。 ### 数据集内容 本仓库包含以下文件： filler.txt：收录一段源自维基百科的关于美洲驼（llamas）的模拟文本，该文本被用作无关测试语料，用于测试不同上下文长度下的证据放置效果（详见论文第3.3节） hover_dataset.pqt：收录HOVER数据集，该数据集为每条事实主张配备多篇维基百科文章的证据，构建多跳核查场景，模拟真实世界事实核查的复杂特性 feverous_dev.pqt：收录FEVEROUS数据集，该数据集在FEVER数据集的基础上进行扩展，纳入结构化数据与更复杂的待核查主张 ### 相关代码仓库 用于分析本数据集并复现论文所述研究结果的代码，可在以下独立仓库获取： https://anonymous.4open.science/r/Needle_In_Factchecking/README.md