fineweb2-telugu-safety

该数据集是一个多语言文本数据集，包含多个配置，每个配置具有不同的特征和数据规模。主要特征包括文本内容、ID、来源URL、日期、文件路径、语言信息（语言、语言分数、语言脚本）、安全标签（安全类别、严重性、置信度、原因、关键词等）以及其他元数据。数据集分为四个配置：默认配置、拉丁字母转写合并配置、泰卢固语拉丁字母转写配置和泰卢固语原文配置。每个配置的训练集大小和样本数量不同，最大配置包含70,000个样本。数据集适用于多语言文本处理、安全内容检测和语言识别等任务。

This is a multilingual text dataset consisting of multiple configurations, each with unique characteristics and data scales. Core features of the dataset include text content, unique ID, source URL, date, file path, language-related information (language type, language score, language script), security labels (including security category, severity level, confidence value, cause, keywords, etc.), and additional metadata. The dataset is split into four configurations: the default configuration, the Latin transliteration merged configuration, the Telugu Latin transliteration configuration, and the original Telugu text configuration. Each configuration differs in training set size and sample count, with the largest configuration encompassing 70,000 samples. This dataset is suitable for tasks including multilingual text processing, security content detection, and language identification.