microsoft/msr_zhen_translation_parity

Name: microsoft/msr_zhen_translation_parity
Creator: microsoft
Published: 2024-01-18 11:09:33
License: 暂无描述

Hugging Face2024-01-18 更新2024-05-25 收录

下载链接：

https://hf-mirror.com/datasets/microsoft/msr_zhen_translation_parity

下载链接

链接失效反馈

官方服务：

资源简介：

--- annotations_creators: - no-annotation language_creators: - expert-generated - machine-generated language: - en license: - ms-pl multilinguality: - monolingual - translation size_categories: - 1K<n<10K source_datasets: - extended|other-newstest2017 task_categories: - translation task_ids: [] paperswithcode_id: null pretty_name: MsrZhenTranslationParity dataset_info: features: - name: Reference-HT dtype: string - name: Reference-PE dtype: string - name: Combo-4 dtype: string - name: Combo-5 dtype: string - name: Combo-6 dtype: string - name: Online-A-1710 dtype: string splits: - name: train num_bytes: 1797033 num_examples: 2001 download_size: 0 dataset_size: 1797033 --- # Dataset Card for msr_zhen_translation_parity ## Table of Contents - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards) - [Languages](#languages) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-fields) - [Data Splits](#data-splits) - [Dataset Creation](#dataset-creation) - [Curation Rationale](#curation-rationale) - [Source Data](#source-data) - [Annotations](#annotations) - [Personal and Sensitive Information](#personal-and-sensitive-information) - [Considerations for Using the Data](#considerations-for-using-the-data) - [Social Impact of Dataset](#social-impact-of-dataset) - [Discussion of Biases](#discussion-of-biases) - [Other Known Limitations](#other-known-limitations) - [Additional Information](#additional-information) - [Dataset Curators](#dataset-curators) - [Licensing Information](#licensing-information) - [Citation Information](#citation-information) - [Contributions](#contributions) ## Dataset Description - **Homepage:** [Translator Human Parity Data](https://msropendata.com/datasets/93f9aa87-9491-45ac-81c1-6498b6be0d0b) - **Repository:** - **Paper:** [Achieving Human Parity on Automatic Chinese to English News Translation](https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation/) - **Leaderboard:** - **Point of Contact:** ### Dataset Summary > Human evaluation results and translation output for the Translator Human Parity Data release, > as described in https://blogs.microsoft.com/ai/machine-translation-news-test-set-human-parity/ > The Translator Human Parity Data release contains all human evaluation results and translations > related to our paper "Achieving Human Parity on Automatic Chinese to English News Translation", > published on March 14, 2018. We have released this data to > 1) allow external validation of our claim of having achieved human parity > 2) to foster future research by releasing two additional human references > for the Reference-WMT test set. > The dataset includes: 1) two new references for Chinese-English language pair of WMT17, one based on human translation from scratch (Reference-HT), the other based on human post-editing (Reference-PE); 2) human parity translations generated by our research systems Combo-4, Combo-5, and Combo-6, as well as translation output from online machine translation service Online-A-1710, collected on October 16, 2017; The data package provided with the study also includes (but not parsed and provided as workable features of this dataset) all data points collected in human evaluation campaigns. ### Supported Tasks and Leaderboards [More Information Needed] ### Languages This dataset contains 6 extra English translations to Chinese-English language pair of WMT17. ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields As mentioned in the summary, this dataset provides 6 extra English translations of Chinese-English language pair of WMT17. Data fields are named exactly like the associated paper for easier cross-referenceing. - `Reference-HT`: human translation from scrach. - `Reference-PE`: human post-editing. - `Combo-4`, `Combo-5`, `Combo-6`: three translations by research systems. - `Online-A-1710`: a translation from an anonymous online machine translation service. All data fields of a record are translations for the same Chinese source sentence. ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information Citation information is available at this link [Achieving Human Parity on Automatic Chinese to English News Translation](https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation/) ### Contributions Thanks to [@leoxzhao](https://github.com/leoxzhao) for adding this dataset.

annotations_creators: - 无注释（no-annotation） language_creators: - 专家生成 - 机器生成 language: - 英语（en） license: - ms-pl multilinguality: - 单语 - 翻译 size_categories: - 1000<n<10000 source_datasets: - extended|other-newstest2017 task_categories: - 翻译 task_ids: [] paperswithcode_id: 无 pretty_name: MsrZhenTranslationParity dataset_info: features: - 名称: 参考-HT（Reference-HT）数据类型: 字符串 - 名称: 参考-PE（Reference-PE）数据类型: 字符串 - 名称: Combo-4 数据类型: 字符串 - 名称: Combo-5 数据类型: 字符串 - 名称: Combo-6 数据类型: 字符串 - 名称: Online-A-1710 数据类型: 字符串 splits: - 名称: train 字节数: 1797033 样本数: 2001 下载大小: 0 数据集大小: 1797033 # msr_zhen_translation_parity 数据集卡片 ## 目录 - [数据集描述](#dataset-description) - [数据集摘要](#dataset-summary) - [支持任务与排行榜](#supported-tasks-and-leaderboards) - [语言](#languages) - [数据集结构](#dataset-structure) - [数据实例](#data-instances) - [数据字段](#data-fields) - [数据划分](#data-splits) - [数据集构建](#dataset-creation) - [构建原理](#curation-rationale) - [源数据](#source-data) - [注释](#annotations) - [个人与敏感信息](#personal-and-sensitive-information) - [数据使用注意事项](#considerations-for-using-the-data) - [数据集的社会影响](#social-impact-of-dataset) - [偏差讨论](#discussion-of-biases) - [其他已知局限性](#other-known-limitations) - [附加信息](#additional-information) - [数据集维护者](#dataset-curators) - [许可证信息](#licensing-information) - [引用信息](#citation-information) - [贡献](#contributions) ## 数据集描述 - **主页**： [翻译者人类对齐数据集](https://msropendata.com/datasets/93f9aa87-9491-45ac-81c1-6498b6be0d0b) - **代码仓库**： - **论文**： [实现汉英新闻机器翻译的人类对齐性能](https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation/) - **排行榜**： - **联络人**： ### 数据集摘要 > 本内容对应《翻译者人类对齐数据集》发布版本的人类评估结果与翻译输出，详细说明参见 https://blogs.microsoft.com/ai/machine-translation-news-test-set-human-parity/ > > 《翻译者人类对齐数据集》发布版本包含与我们2018年3月14日发表的论文《实现汉英新闻机器翻译的人类对齐性能》相关的全部人类评估结果与翻译内容。我们发布此数据以达成两个目标： > 1) 允许外部验证我们达成人类对齐性能的研究声明； > 2) 为WMT17测试集发布两份额外的人工参考译文，以推动后续相关研究。本数据集包含： 1) WMT17汉英语言对的两份新参考译文：一份为从头人工翻译参考译文（Reference-HT），另一份为人工后编辑参考译文（Reference-PE）； 2) 本研究团队开发的Combo-4、Combo-5、Combo-6三款研究系统生成的人类对齐级翻译结果，以及2017年10月16日采集的在线机器翻译服务输出Online-A-1710；本研究配套的数据包还包含人类评估活动中收集的全部数据点，但本数据集未对其进行解析并作为可用特征提供。 ### 支持任务与排行榜 [需要更多信息] ### 语言本数据集为WMT17汉英语言对提供了6份额外的英语译文。 ## 数据集结构 ### 数据实例 [需要更多信息] ### 数据字段如摘要所述，本数据集为WMT17汉英语言对提供了6份额外的英语译文。数据字段名称与相关论文完全一致，便于交叉引用。 - `Reference-HT`：从头人工翻译参考译文。 - `Reference-PE`：人工后编辑参考译文。 - `Combo-4`、`Combo-5`、`Combo-6`：三款研究系统生成的翻译结果。 - `Online-A-1710`：某匿名在线机器翻译服务生成的翻译结果。单条记录的所有数据字段均对应同一条中文源句。 ### 数据划分 [需要更多信息] ## 数据集构建 ### 构建原理 [需要更多信息] ### 源数据 #### 初始数据收集与归一化 [需要更多信息] #### 源语言生成者是谁？ [需要更多信息] ### 注释 #### 注释流程 [需要更多信息] #### 注释者是谁？ [需要更多信息] ### 个人与敏感信息 [需要更多信息] ## 数据使用注意事项 ### 数据集的社会影响 [需要更多信息] ### 偏差讨论 [需要更多信息] ### 其他已知局限性 [需要更多信息] ## 附加信息 ### 数据集维护者 [需要更多信息] ### 许可证信息 [需要更多信息] ### 引用信息引用信息可通过以下链接获取：[实现汉英新闻机器翻译的人类对齐性能](https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation/) ### 贡献感谢[@leoxzhao](https://github.com/leoxzhao) 添加本数据集。

提供机构：

microsoft

原始信息汇总

数据集概述

数据集描述

数据集摘要

数据集名称: MsrZhenTranslationParity
数据集内容: 包含人类评估结果和翻译输出，用于验证“Achieving Human Parity on Automatic Chinese to English News Translation”论文中的人类对等性声明。数据集提供两种新的参考翻译：基于从头开始的人工翻译（Reference-HT）和基于人工后编辑的翻译（Reference-PE），以及其他由研究系统生成的翻译（Combo-4, Combo-5, Combo-6）和在线机器翻译服务输出（Online-A-1710）。