dipta007/Barta

Barta（孟加拉语意为“新闻”）是一个大规模的孟加拉语新闻语料库，包含来自多个孟加拉国新闻源的170万篇文章，涵盖政治、体育、娱乐、技术等多个类别。关键特征包括：170万篇文章、多样化的类别（如国内、国际、体育、娱乐、技术、生活方式等）、结构化字段（包括标题、文章正文、来源和类别）以及通过内容哈希进行去重和清理。数据集适用于孟加拉语语言模型的预训练、标题生成训练和评估、文本摘要、新闻分类和主题建模以及孟加拉语NLP研究和基准测试。

Barta (meaning news in Bengali) is a large-scale Bengali news corpus containing 1.7M articles from multiple Bangladeshi news sources, spanning diverse categories including politics, sports, entertainment, technology, and more. Key features include: 1.7M articles, diverse categories (e.g., national, international, sports, entertainment, technology, lifestyle), structured fields (including headline, article body, source, and category), and deduplication and cleaning via content hashing. The dataset is intended for pretraining Bengali language models, headline generation training and evaluation, text summarization for Bengali, news categorization and topic modeling, and Bengali NLP research and benchmarking.