clarin-pl/PUGG
收藏资源简介：
--- annotations_creators: - expert-generated language_creators: [] language: - pl license: - cc-by-sa-4.0 multilinguality: - monolingual size_categories: - 1K<n<10K - 10K<n<100K source_datasets: - original task_categories: - question-answering - text-retrieval task_ids: - extractive-qa - document-retrieval pretty_name: 'PUGG: KBQA, MRC, IR dataset for Polish' tags: - knowledge graph - KBQA - wikipedia - wikidata configs: - config_name: kbqa_all data_files: - split: train path: kbqa/*/train.jsonl - split: test path: kbqa/*/test.jsonl - config_name: kbqa_natural data_files: - split: train path: kbqa/natural/train.jsonl - split: test path: kbqa/natural/test.jsonl - config_name: kbqa_template-based data_files: - split: train path: kbqa/template-based/train.jsonl - split: test path: kbqa/template-based/test.jsonl - config_name: mrc data_files: - split: train path: mrc/train.jsonl - split: test path: mrc/test.jsonl - config_name: ir_corpus data_files: - split: test path: ir/corpus.jsonl - config_name: ir_queries data_files: - split: test path: ir/queries.jsonl - config_name: ir_qrels data_files: - split: test path: ir/qrels/test.jsonl --- # PUGG: KBQA, MRC, IR Dataset for Polish ## Description This repository contains the PUGG dataset designed for three NLP tasks in the Polish language: - KBQA (Knowledge Base Question Answering) - MRC (Machine Reading Comprehension) - IR (Information Retrieval) ## Paper For more detailed information, please refer to our research paper titled: **"Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction"** Authored by: * Albert Sawczyn * Katsiaryna Viarenich * Konrad Wojtasik * Aleksandra Domogała * Marcin Oleksy * Maciej Piasecki * Tomasz Kajdanowicz **The paper was accepted for ACL 2024 (findings).** ## Repositories The dataset is available in the following repositories: * [General](https://huggingface.co/datasets/clarin-pl/PUGG) **(this repository)** - contains all tasks (KBQA, MRC, IR*) For more straightforward usage, the tasks are also available in separate repositories: * [KBQA](https://huggingface.co/datasets/clarin-pl/PUGG_KBQA) * [MRC](https://huggingface.co/datasets/clarin-pl/PUGG_MRC) * [IR](https://huggingface.co/datasets/clarin-pl/PUGG_IR) The knowledge graph for KBQA task is available in the following repository: * [Knowledge Graph](https://huggingface.co/datasets/clarin-pl/PUGG_KG) Note: If you want to utilize the IR task in the BEIR format (`qrels` in `.tsv` format), please download the [IR](https://huggingface.co/datasets/clarin-pl/PUGG_IR) repository. ## Links * Code: * [Github](https://github.com/CLARIN-PL/PUGG) * Paper: * ACL - TBA * [Arxiv](https://arxiv.org/abs/2408.02337) ## Citation ```bibtex @misc{sawczyn2024developingpuggpolishmodern, title={Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction}, author={Albert Sawczyn and Katsiaryna Viarenich and Konrad Wojtasik and Aleksandra Domogała and Marcin Oleksy and Maciej Piasecki and Tomasz Kajdanowicz}, year={2024}, eprint={2408.02337}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2408.02337}, } ``` ## Contact albert.sawczyn@pwr.edu.pl ## Usage ```python from datasets import load_dataset # loading KBQA (all) dataset = load_dataset("clarin-pl/PUGG", "kbqa_all") print(dataset) # loading KBQA (natural) dataset = load_dataset("clarin-pl/PUGG", "kbqa_natural") print(dataset) # loading KBQA (template-based) dataset = load_dataset("clarin-pl/PUGG", "kbqa_template-based") print(dataset) # loading MRC dataset = load_dataset("clarin-pl/PUGG", "mrc") print(dataset) # loading IR ## corpus dataset = load_dataset("clarin-pl/PUGG", "ir_corpus") print(dataset) ## queries dataset = load_dataset("clarin-pl/PUGG", "ir_queries") print(dataset) ## qrels dataset = load_dataset("clarin-pl/PUGG", "ir_qrels") print(dataset) ```
annotations_creators: - 专家生成 language_creators: [] language: - 波兰语（pl） license: - CC BY-SA 4.0（知识共享署名-相同方式共享4.0协议） multilinguality: - 单语言 size_categories: - 1千<样本量<1万 - 1万<样本量<10万 source_datasets: - 原创数据集 task_categories: - 问答 - 文本检索 task_ids: - 抽取式问答 - 文档检索 pretty_name: 'PUGG：面向波兰语的KBQA、MRC、IR数据集' tags: - 知识图谱（knowledge graph） - KBQA - 维基百科（wikipedia） - 维基数据（wikidata） configs: - config_name: kbqa_all data_files: - split: 训练集 path: kbqa/*/train.jsonl - split: 测试集 path: kbqa/*/test.jsonl - config_name: kbqa_natural data_files: - split: 训练集 path: kbqa/natural/train.jsonl - split: 测试集 path: kbqa/natural/test.jsonl - config_name: kbqa_template-based data_files: - split: 训练集 path: kbqa/template-based/train.jsonl - split: 测试集 path: kbqa/template-based/test.jsonl - config_name: mrc data_files: - split: 训练集 path: mrc/train.jsonl - split: 测试集 path: mrc/test.jsonl - config_name: ir_corpus data_files: - split: 测试集 path: ir/corpus.jsonl - config_name: ir_queries data_files: - split: 测试集 path: ir/queries.jsonl - config_name: ir_qrels data_files: - split: 测试集 path: ir/qrels/test.jsonl # PUGG：面向波兰语的KBQA、MRC、IR数据集 ## 数据集概述 本仓库包含专为波兰语自然语言处理三大任务设计的PUGG数据集： - KBQA（知识库问答，Knowledge Base Question Answering） - MRC（机器阅读理解，Machine Reading Comprehension） - IR（信息检索，Information Retrieval） ## 相关论文 如需获取更详尽的信息，请参阅我们的研究论文： **《面向波兰语的PUGG：KBQA、MRC与IR数据集的现代化构建方法》** 作者： * Albert Sawczyn * Katsiaryna Viarenich * Konrad Wojtasik * Aleksandra Domogała * Marcin Oleksy * Maciej Piasecki * Tomasz Kajdanowicz 本论文已被ACL 2024会议发现版块收录。 ## 仓库分布 本数据集可通过以下仓库获取： * [通用仓库](https://huggingface.co/datasets/clarin-pl/PUGG)（本仓库）—— 涵盖全部三大任务（KBQA、MRC、IR*） 为便于更直接地使用，各任务已拆分至独立仓库： * [KBQA任务仓库](https://huggingface.co/datasets/clarin-pl/PUGG_KBQA) * [MRC任务仓库](https://huggingface.co/datasets/clarin-pl/PUGG_MRC) * [IR任务仓库](https://huggingface.co/datasets/clarin-pl/PUGG_IR) KBQA任务所需的知识图谱可通过以下仓库获取： * [知识图谱仓库](https://huggingface.co/datasets/clarin-pl/PUGG_KG) 注意：若需使用BEIR格式的IR任务（即`.tsv`格式的`qrels`文件），请下载[IR任务仓库](https://huggingface.co/datasets/clarin-pl/PUGG_IR)。 ## 相关链接 * 代码： * [GitHub仓库](https://github.com/CLARIN-PL/PUGG) * 论文： * ACL会议——待公布 * [Arxiv预印本](https://arxiv.org/abs/2408.02337) ## 引用格式 bibtex @misc{sawczyn2024developingpuggpolishmodern, title={Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction}, author={Albert Sawczyn and Katsiaryna Viarenich and Konrad Wojtasik and Aleksandra Domogała and Marcin Oleksy and Maciej Piasecki and Tomasz Kajdanowicz}, year={2024}, eprint={2408.02337}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2408.02337}, } ## 联系方式 albert.sawczyn@pwr.edu.pl ## 使用示例 python from datasets import load_dataset # 加载全量KBQA数据集 dataset = load_dataset("clarin-pl/PUGG", "kbqa_all") print(dataset) # 加载自然语言KBQA数据集 dataset = load_dataset("clarin-pl/PUGG", "kbqa_natural") print(dataset) # 加载基于模板的KBQA数据集 dataset = load_dataset("clarin-pl/PUGG", "kbqa_template-based") print(dataset) # 加载MRC数据集 dataset = load_dataset("clarin-pl/PUGG", "mrc") print(dataset) # 加载IR数据集 ## 语料库 dataset = load_dataset("clarin-pl/PUGG", "ir_corpus") print(dataset) ## 查询集 dataset = load_dataset("clarin-pl/PUGG", "ir_queries") print(dataset) ## 相关性标注文件 dataset = load_dataset("clarin-pl/PUGG", "ir_qrels") print(dataset)
PUGG: KBQA, MRC, IR 数据集概述
数据集描述
本数据集名为PUGG，是根据研究论文《Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction》创建的，该论文由以下作者撰写：
- Albert Sawczyn
- Katsiaryna Viarenich
- Konrad Wojtasik
- Aleksandra Domogała
- Marcin Oleksy
- Maciej Piasecki
- Tomasz Kajdanowicz
该论文已被ACL 2024（Findings）接受。
数据集用途
PUGG数据集旨在支持波兰语的KBQA（知识库问答）、MRC（机器阅读理解）和IR（信息检索）研究。
