<p>Performance of Bangla Bert base.</p>
收藏NIAID Data Ecosystem2026-05-10 收录
下载链接:
https://figshare.com/articles/dataset/_p_Performance_of_Bangla_Bert_base_p_/31415041
下载链接
链接失效反馈官方服务:
资源简介:
Named Entity Recognition (NER) in regional dialects is a critical yet underexplored area in Natural Language Processing (NLP), especially for low-resource languages like Bangla. While NER systems for Standard Bangla have made progress, no existing resources or models specifically address the challenge of regional dialects such as Barishal, Chittagong, Mymensingh, Noakhali, and Sylhet, which exhibit unique linguistic features that existing models fail to handle effectively. To fill this gap, we introduce ANCHOLIK-NER, the first benchmark dataset for NER in Bangla regional dialects, comprising 17,405 sentences and 101,817 words annotated with 10 entity tags across 5 regions. The dataset was sourced from publicly available resources and supplemented with manual translations, ensuring alignment of named entities across dialects. We evaluate three transformer-based models—Bangla BERT, Bangla Bert Base, and BERT Base Multilingual Cased—on this dataset. Bangla BERT achieved the highest performance overall, with F1-scores of 82.27% (Mymensingh), 81.48% (Barishal), 78.75% (Sylhet), 78.50% (Noakhali), and 75.31% (Chittagong). These results highlight strong recognition capability in Mymensingh and Barishal, while dialectal variation in Chittagong remains challenging. As no prior NER resources exist for Bangla regional dialects, this work provides a foundational dataset and baseline benchmarks to facilitate future research. Future work will focus on dialect-aware model adaptation and expanding coverage to additional regions.
命名实体识别(Named Entity Recognition, NER)在自然语言处理(Natural Language Processing, NLP)领域是一个至关重要却尚未得到充分探索的研究方向,对于孟加拉语(Bangla)这类低资源语言而言尤为如此。尽管标准孟加拉语的NER系统已取得一定进展,但目前尚无任何现有资源或模型能够专门应对巴里萨尔(Barishal)、吉大港(Chittagong)、迈门辛(Mymensingh)、诺阿卡利(Noakhali)以及锡尔赫特(Sylhet)等孟加拉语地区方言带来的挑战——这些方言拥有独特的语言特征,现有模型无法对其进行有效处理。为填补这一研究空白,我们推出了ANCHOLIK-NER,首个面向孟加拉语地区方言的命名实体识别基准数据集,该数据集涵盖5个方言区域的17,405个句子与101,817个词,共标注了10类实体标签。该数据集的素材源自公开可用资源,并通过人工翻译进行补充,确保不同方言间的命名实体对齐一致。我们在该数据集上评估了3种基于Transformer的模型:孟加拉语BERT(Bangla BERT)、孟加拉语基础BERT(Bangla Bert Base)以及多语言大小写敏感BERT(BERT Base Multilingual Cased)。其中孟加拉语BERT整体性能最优,在迈门辛方言上的F1值为82.27%,巴里萨尔方言为81.48%,锡尔赫特方言为78.75%,诺阿卡利方言为78.50%,吉大港方言为75.31%。该结果表明,模型在迈门辛与巴里萨尔方言上具备出色的识别能力,但吉大港方言的方言差异仍对模型构成较大挑战。由于目前尚无针对孟加拉语地区方言的NER相关资源,本研究提供了首个基准数据集与基线基准,以推动后续相关研究的开展。未来的研究将聚焦于方言感知的模型适配,并将数据集覆盖范围拓展至更多方言区域。
创建时间:
2026-02-25



