Data for Cyrillic Reference Parsing

We provide a <strong>synthetic reference data set</strong> covering over 100,000 labeled references (mostly Russian language) and a manually annotated set of real references (771 in number) gathered from multidisciplinary <strong>Cyrillic script publications</strong>. <strong>Background:</strong> Extracting structured data from bibliographic references is a crucial task for the creation of scholarly databases. While approaches, tools, and evaluation data sets for the task exist, there is a distinct lack of support for languages other than English and scripts other than the Latin alphabet. A significant portion of the scientific literature that is thereby excluded consists of publications written in Cyrillic script languages. To address this problem, we introduce a new multilingual and multidisciplinary data set of over 100,000 labeled reference strings. The data set covers multiple Cyrillic languages and contains over 700 manually labeled references, while the remaining are generated synthetically. With random samples of varying size of this data, we train multiple well-performing sequence labeling BERT models and thus show the usability of our proposed data set. To this end, we showcase an implementation of a multilingual BERT model trained on the synthetic data and evaluated on the manually labeled references. Our model achieves an F1 score of 0.93 and thereby significantly outperforms a state-of-the-art model we retrain and evaluate on our data. The code for generating the data set is available at https://github.com/igor261/Sequence-Labeling-for-Citation-Field-Extraction-from-Cyrillic-Script-References When using the data set, please cite the following paper: <em>Igor Shapiro, Tarek Saier, Michael Färber: "Sequence Labeling for Citation Field Extraction from Cyrillic Script References". In Proceedings of the AAAI-22 Workshop on Scientific Document Understanding (SDU@AAAI'22), 2022.</em>

本研究提供**合成参考文献数据集（synthetic reference data set）**，涵盖10万余条带标注参考文献（以俄语为主），以及从多学科**西里尔字母（Cyrillic script）出版物**中收集的771条人工标注真实参考文献。**背景**：从**参考文献（bibliographic references）**中提取结构化数据是构建学术数据库的核心任务。目前虽已有针对该任务的方法、工具与评估数据集，但针对非英语语言及非拉丁字母脚本的支持存在明显缺失。因此被排除的大量科学文献均为西里尔字母语言撰写的出版物。为解决这一问题，我们推出全新的多语言、多学科标注参考文献字符串数据集，规模超10万条。本数据集涵盖多种西里尔语言，包含700余条人工标注参考文献，其余数据均为合成生成。我们利用该数据集不同规模的随机样本，训练了多款性能优异的**序列标注BERT模型（sequence labeling BERT models）**，以此验证本数据集的可用性。为此，我们展示了基于合成数据训练、并在人工标注参考文献上评估的多语言BERT模型实现方案。我们的模型**F1值（F1 score）**达到0.93，显著优于我们在本数据集上重新训练并评估的**当前最优模型（state-of-the-art model）**。本数据集的生成代码可在以下网址获取：https://github.com/igor261/Sequence-Labeling-for-Citation-Field-Extraction-from-Cyrillic-Script-References 若使用本数据集，请引用以下论文：伊戈尔·夏皮罗（Igor Shapiro）、塔雷克·扎伊尔（Tarek Saier）、迈克尔·费伯（Michael Färber）：《基于西里尔字母参考文献的引文字段提取序列标注》，收录于2022年AAAI-22科学文献理解研讨会（SDU@AAAI'22）论文集。