shivaniachary123/pii-masking-300k

Ai4Privacy PII 300k数据集是世界上最大的开放隐私掩码数据集，专门用于训练和评估模型以从文本中移除个人可识别信息（PII）和敏感信息，特别是在人工智能助手和大语言模型的背景下。该数据集包含约220,000个文本条目，覆盖3040万文本令牌和760万PII令牌，支持6种语言（英语、法语、德语、意大利语、荷兰语、西班牙语）并在8个司法管辖区进行本地化。数据集分为两个子集：OpenPII-220k包含27个PII类别，针对教育、健康和心理学领域的749个讨论主题/用例；FinPII-80k则包含约20个针对保险和金融的额外PII类型。数据集采用79%-21%的训练/验证分割，通过专有算法生成合成数据，确保无隐私侵犯，并经过人工循环验证，令牌标签准确率约98.3%。数据集格式包括源文本、掩码后文本、隐私掩码标签、跨度标签、多语言BERT令牌等字段，适用于令牌分类、文本生成等多种机器学习任务。

The Ai4Privacy PII 300k Dataset is the worlds largest open dataset for privacy masking, designed to train and evaluate models for removing personally identifiable information (PII) and sensitive data from text, particularly in the context of AI assistants and LLMs. It contains approximately 220,000 text entries with 30.4 million text tokens and 7.6 million PII tokens, supporting 6 languages (English, French, German, Italian, Dutch, Spanish) with strong localization across 8 jurisdictions. The dataset is split into two subsets: OpenPII-220k includes 27 PII classes targeting 749 discussion subjects/use cases across education, health, and psychology; FinPII-80k adds approximately 20 additional types tailored to insurance and finance. It features a 79%-21% training/validation split, uses synthetically generated data via proprietary algorithms without privacy violations, and is human-in-the-loop validated with ~98.3% token label accuracy. The dataset includes fields such as source text, target text, privacy mask labels, span labels, and multilingual BERT tokens, and is compatible with machine learning tasks like token classification and text generation.