TweetNERD - End to End Entity Linking Benchmark for Tweets

TweetNERD - End to End Entity Linking Benchmark for Tweets Paper - Video - Neurips Page This is the dataset described in the paper <strong>TweetNERD - End to End Entity Linking Benchmark for Tweets</strong> (accepted to Thirty-sixth Conference on Neural Information Processing Systems (Neurips) Datasets and Benchmarks Track). Named Entity Recognition and Disambiguation (NERD) systems are foundational for information retrieval, question answering, event detection, and other natural language processing (NLP) applications. We introduce TweetNERD, a dataset of 340K+ Tweets across 2010-2021, for benchmarking NERD systems on Tweets. This is the largest and most temporally diverse open sourced dataset benchmark for NERD on Tweets and can be used to facilitate research in this area. TweetNERD dataset is released under Creative Commons Attribution 4.0 International (CC BY 4.0) LICENSE. The license only applies to the data files present in this dataset. See <strong>Data usage policy</strong> below. Check out more details at https://github.com/twitter-research/TweetNERD <strong>Usage</strong> We provide the dataset split across the following tab seperated files: <strong>OOD.public.tsv</strong>: OOD split of the data in the paper. <strong>Academic.public.tsv</strong>: Academic split of the data described in the paper. <code>part_*.public.tsv</code>: Remaining data split into parts in no particular order. Each file is tab separated and has has the following format: tweet_id phrase start end entityId score 22 twttr 20 25 Q918 3 21 twttr 20 25 Q918 3 1457198399032287235 Diwali 30 38 Q10244 3 1232456079247736833 NO_PHRASE -1 -1 NO_ENTITY -1 For tweets which don't have any entity, their column values for <code>phrase, start, end, entityId, score</code> are set <code>NO_PHRASE, -1, -1, NO_ENTITY, -1</code> respectively. Description of file columns is as follows: Column Type Missing Value Description tweet_id string ID of the Tweet phrase string NO_PHRASE entity phrase start int -1 start offset of the phrase in text using <code>UTF-16BE</code> encoding end int -1 end offset of the phrase in the text using <code>UTF-16BE</code> encoding entityId string NO_ENTITY Entity ID. If not missing can be NOT FOUND, AMBIGUOUS, or Wikidata ID of format Q{numbers}, e.g. Q918 score int -1 Number of annotators who agreed on the phrase, start, end, entityId information In order to use the dataset you need to utilize the <code>tweet_id</code> column and get the Tweet text using the Twitter API (See <strong>Data usage policy</strong> section below). Data stats Split Number of Rows Number unique tweets OOD 34102 25000 Academic 51685 30119 part_0 11830 10000 part_1 35681 25799 part_2 34256 25000 part_3 36478 25000 part_4 37518 24999 part_5 36626 25000 part_6 34001 24984 part_7 34125 24981 part_8 32556 25000 part_9 32657 25000 part_10 32442 25000 part_11 32033 24972 Data usage policy Use of this dataset is subject to you obtaining lawful access to the Twitter API, which requires you to agree to the Developer Terms Policies and Agreements. Please cite the following if you use TweetNERD in your paper: <pre>@dataset{TweetNERD_Zenodo_2022_6617192, author = {Mishra, Shubhanshu and Saini, Aman and Makki, Raheleh and Mehta, Sneha and Haghighi, Aria and Mollahosseini, Ali}, title = {{TweetNERD - End to End Entity Linking Benchmark for Tweets}}, month = jun, year = 2022, note = {{Data usage policy Use of this dataset is subject to you obtaining lawful access to the [Twitter API](https://developer.twitter.com/en/docs /twitter-api), which requires you to agree to the [Developer Terms Policies and Agreements](https://developer.twitter.com/en /developer-terms/).}}, publisher = {Zenodo}, version = {0.0.0}, doi = {10.5281/zenodo.6617192}, url = {https://doi.org/10.5281/zenodo.6617192} } @inproceedings{TweetNERDNeurips2022, author = {Mishra, Shubhanshu and Saini, Aman and Makki, Raheleh and Mehta, Sneha and Haghighi, Aria and Mollahosseini, Ali}, booktitle = {Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks}, pages = {}, title = {TweetNERD - End to End Entity Linking Benchmark for Tweets}, volume = {2}, year = {2022}, eprint = {arXiv:2210.08129}, doi = {10.48550/arXiv.2210.08129} } </pre>

**TweetNERD：面向推文的端到端实体链接基准数据集** 论文 - 演示视频 - NeurIPS 页面
本数据集对应论文《TweetNERD：面向推文的端到端实体链接基准数据集》，该论文已被第36届神经信息处理系统大会（NeurIPS）数据集与基准赛道收录。

命名实体识别与消歧（Named Entity Recognition and Disambiguation, NERD）系统是信息检索、问答、事件检测及其他自然语言处理（NLP）应用的基础组件。本工作推出TweetNERD数据集，该数据集包含2010年至2021年间的34万余条推文，用于在推文场景下对NERD系统进行基准测试。本数据集是目前规模最大、时间跨度最广的开源推文NERD基准数据集，可推动该领域的相关研究。

TweetNERD数据集采用知识共享署名4.0国际许可协议（Creative Commons Attribution 4.0 International, CC BY 4.0）发布，该许可仅适用于本数据集内的数据文件。详见下文**数据使用政策**。更多细节可访问：https://github.com/twitter-research/TweetNERD

---

### 使用方式
本数据集按以下制表符分隔文件划分：
- **OOD.public.tsv**：论文中所述的分布外（OOD）划分数据集
- **Academic.public.tsv**：论文中所述的学术划分数据集
- `part_*.public.tsv`：剩余数据按无特定顺序拆分为多个分片文件

每个文件均为制表符分隔格式，示例结构如下：

tweet_id phrase start end entityId score
22 twttr 20 25 Q918 3
21 twttr 20 25 Q918 3
1457198399032287235 Diwali 30 38 Q10244 3
1232456079247736833 NO_PHRASE -1 -1 NO_ENTITY -1

对于不含任何实体的推文，其`phrase`、`start`、`end`、`entityId`、`score`列的值分别设为`NO_PHRASE`、`-1`、`-1`、`NO_ENTITY`、`-1`。

各列的详细说明如下：
| 列名 | 数据类型 | 缺失值标记 | 说明 |
|------|----------|------------|------|
| tweet_id | 字符串 | 无 | 推文的唯一标识符 |
| phrase | 字符串 | `NO_PHRASE` | 实体短语 |
| start | 整数 | `-1` | 实体短语在推文中的起始偏移量，采用`UTF-16BE`编码 |
| end | 整数 | `-1` | 实体短语在推文中的结束偏移量，采用`UTF-16BE`编码 |
| entityId | 字符串 | `NO_ENTITY` | 实体标识符。若不为缺失值，可能为`NOT FOUND`、`AMBIGUOUS`或格式为`Q{数字}`的维基数据（Wikidata）ID，例如`Q918` |
| score | 整数 | `-1` | 同意该短语、起始位置、结束位置及实体ID标注的标注者人数 |

若要使用本数据集，需通过`tweet_id`列调用Twitter API获取推文原文（详见下文**数据使用政策**部分）。

---

### 数据集统计信息
| 数据集划分 | 总行数 | 唯一推文数 |
|------------|--------|------------|
| OOD | 34102 | 25000 |
| Academic | 51685 | 30119 |
| part_0 | 11830 | 10000 |
| part_1 | 35681 | 25799 |
| part_2 | 34256 | 25000 |
| part_3 | 36478 | 25000 |
| part_4 | 37518 | 24999 |
| part_5 | 36626 | 25000 |
| part_6 | 34001 | 24984 |
| part_7 | 34125 | 24981 |
| part_8 | 32556 | 25000 |
| part_9 | 32657 | 25000 |
| part_10 | 32442 | 25000 |
| part_11 | 32033 | 24972 |

---

### 数据使用政策
使用本数据集需先合法获取Twitter API的访问权限，且需同意其开发者条款政策与协议。若您在论文中使用TweetNERD数据集，请引用以下文献：
bibtex
@dataset{TweetNERD_Zenodo_2022_6617192,
author = {Mishra, Shubhanshu and Saini, Aman and Makki, Raheleh and Mehta, Sneha and Haghighi, Aria and Mollahosseini, Ali},
title = {{TweetNERD - End to End Entity Linking Benchmark for Tweets}},
month = jun,
year = {2022},
note = {{Data usage policy Use of this dataset is subject to you obtaining lawful access to the [Twitter API](https://developer.twitter.com/en/docs /twitter-api), which requires you to agree to the [Developer Terms Policies and Agreements](https://developer.twitter.com/en /developer-terms/).}},
publisher = {Zenodo},
version = {0.0.0},
doi = {10.5281/zenodo.6617192},
url = {https://doi.org/10.5281/zenodo.6617192}
}
@inproceedings{TweetNERDNeurips2022,
author = {Mishra, Shubhanshu and Saini, Aman and Makki, Raheleh and Mehta, Sneha and Haghighi, Aria and Mollahosseini, Ali},
booktitle = {Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks},
pages = {},
title = {TweetNERD - End to End Entity Linking Benchmark for Tweets},
volume = {2},
year = {2022},
eprint = {arXiv:2210.08129},
doi = {10.48550/arXiv.2210.08129}
}