mediabiasgroup/mbib-base

--- license: cc-by-nc-nd-4.0 task_categories: - text-classification language: - en tags: - media - mediabias - media-bias - media bias size_categories: - 1M<n<10M dataset_info: config_name: plain_text splits: - name: cognitive_bias - name: fake_news - name: gender_bias - name: hate_speech - name: linguistic_bias - name: political_bias - name: racial_bias - name: text_level_bias configs: - config_name: default data_files: - split: cognitive_bias path: mbib-aggregated/cognitive-bias.csv - split: fake_news path: mbib-aggregated/fake-news.csv - split: gender_bias path: mbib-aggregated/gender-bias.csv - split: hate_speech path: mbib-aggregated/hate-speech.csv - split: linguistic_bias path: mbib-aggregated/linguistic-bias.csv - split: political_bias path: mbib-aggregated/political-bias.csv - split: racial_bias path: mbib-aggregated/racial-bias.csv - split: text_level_bias path: mbib-aggregated/text-level-bias.csv --- # Dataset Card for Media-Bias-Identification-Benchmark ## Table of Contents - [Dataset Card for Media-Bias-Identification-Benchmark](#dataset-card-for-mbib) - [Table of Contents](#table-of-contents) - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Tasks and Information](#tasks-and-information) - [Baseline](#baseline) - [Languages](#languages) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [cognitive-bias](#cognitive-bias) - [Data Fields](#data-fields) - [Considerations for Using the Data](#considerations-for-using-the-data) - [Social Impact of Dataset](#social-impact-of-dataset) - [Discussion of Biases](#discussion-of-biases) - [Other Known Limitations](#other-known-limitations) - [Citation Information](#citation-information) - [Contributions](#contributions) ## Dataset Description - **Homepage:** https://github.com/Media-Bias-Group/Media-Bias-Identification-Benchmark - **Repository:** https://github.com/Media-Bias-Group/Media-Bias-Identification-Benchmark - **Paper:** https://doi.org/10.1145/3539618.3591882 - **Point of Contact:** [Martin Wessel](mailto:[e-mail]) ### Baseline <table> <tr><td><b>Task</b></td><td><b>Model</b></td><td><b>Micro F1</b></td><td><b>Macro F1</b></td></tr> <td>cognitive-bias</td> <td> ConvBERT/ConvBERT</td> <td>0.7126</td> <td> 0.7664</td></tr> <td>fake-news</td> <td>Bart/RoBERTa-T</td> <td>0.6811</td> <td> 0.7533</td> </tr> <td>gender-bias</td> <td> RoBERTa-T/ELECTRA</td> <td>0.8334</td> <td>0.8211</td> </tr> <td>hate-speech</td> <td>RoBERTA-T/Bart</td> <td>0.8897</td> <td> 0.7310</td> </tr> <td>linguistic-bias</td> <td> ConvBERT/Bart </td> <td> 0.7044 </td> <td> 0.4995 </td> </tr> <td>political-bias</td> <td> ConvBERT/ConvBERT </td> <td> 0.7041 </td> <td> 0.7110 </td> </tr> <td>racial-bias</td> <td> ConvBERT/ELECTRA </td> <td> 0.8772 </td> <td> 0.6170 </td> </tr> <td>text-leve-bias</td> <td> ConvBERT/ConvBERT </td> <td> 0.7697</td> <td> 0.7532 </td> </tr> </table> ### Languages All datasets are in English ## Dataset Structure ### Data Instances #### cognitive-bias An example of one training instance looks as follows. ```json { "text": "A defense bill includes language that would require military hospitals to provide abortions on demand", "label": 1 } ``` ### Data Fields - `text`: a sentence from various sources (eg., news articles, twitter, other social media). - `label`: binary indicator of bias (0 = unbiased, 1 = biased) ## Considerations for Using the Data ### Social Impact of Dataset We believe that MBIB offers a new common ground for research in the domain, especially given the rising amount of (research) attention directed toward media bias ### Citation Information ``` @inproceedings{ title = {Introducing MBIB - the first Media Bias Identification Benchmark Task and Dataset Collection}, author = {Wessel, Martin and Spinde, Timo and Horych, Tomáš and Ruas, Terry and Aizawa, Akiko and Gipp, Bela}, year = {2023}, note = {[in review]} } ```

原始信息汇总

数据集概述

数据集名称

Media-Bias-Identification-Benchmark (MBIB)

数据集许可证

cc-by-nc-nd-4.0

任务类别

• text-classification

语言

• en

标签

• media
• mediabias
• media-bias
• media bias

大小分类

• 1M<n<10M

数据集配置

• config_name: plain_text

数据集分割

• cognitive_bias
• fake_news
• gender_bias
• hate_speech
• linguistic_bias
• political_bias
• racial_bias
• text_level_bias

数据集结构

• 数据实例

  • cognitive-bias json { "text": "A defense bill includes language that would require military hospitals to provide abortions on demand", "label": 1 }

• 数据字段

  • text: 来自各种来源的句子（例如，新闻文章、Twitter、其他社交媒体）。
  • label: 偏见的二元指示符（0 = 无偏见，1 = 有偏见）

基准模型性能

• cognitive-bias
  • Model: ConvBERT/ConvBERT
  • Micro F1: 0.7126
  • Macro F1: 0.7664
• fake-news
  • Model: Bart/RoBERTa-T
  • Micro F1: 0.6811
  • Macro F1: 0.7533
• gender-bias
  • Model: RoBERTa-T/ELECTRA
  • Micro F1: 0.8334
  • Macro F1: 0.8211
• hate-speech
  • Model: RoBERTA-T/Bart
  • Micro F1: 0.8897
  • Macro F1: 0.7310
• linguistic-bias
  • Model: ConvBERT/Bart
  • Micro F1: 0.7044
  • Macro F1: 0.4995
• political-bias
  • Model: ConvBERT/ConvBERT
  • Micro F1: 0.7041
  • Macro F1: 0.7110
• racial-bias
  • Model: ConvBERT/ELECTRA
  • Micro F1: 0.8772
  • Macro F1: 0.6170
• text-level-bias
  • Model: ConvBERT/ConvBERT
  • Micro F1: 0.7697
  • Macro F1: 0.7532