laion/reference-voices-enhanced

---
license: cc-by-4.0
task_categories:
- audio-classification
- text-to-speech
tags:
- voice
- speaker-embedding
- deduplicated
- emotion
- quality-filtered
- speech-enhancement
- clearervoice
pretty_name: Reference Voices Enhanced
size_categories:
- 1K<n<10K
---

# Reference Voices Enhanced

2,004 AI voice samples enhanced with [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio) MossFormer2_SE_48K speech enhancement, annotated with [Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus) (59 quality + emotion scores).

## Dataset Summary

- **Source**: [laion/ai-voices-deduplicated](https://huggingface.co/datasets/laion/ai-voices-deduplicated) (2,004 speaker-deduplicated, quality-filtered AI voice samples)
- **Speech Enhancement**: ClearerVoice MossFormer2_SE_48K — background noise removal and speech clarity improvement
- **Output Format**: Enhanced WAV files at 48kHz (replacing original MP3s)
- **Annotations**: Full Empathic Insight Voice Plus scores (59 dimensions: 55 emotion scores + 4 quality scores)
- **Metadata**: Updated JSON sidecar per sample with all annotation scores
- **Packaging**: Single tar file

## Enhancement Pipeline

1. **Source data**: 2,004 samples from `laion/ai-voices-deduplicated`, organized by gender (male/female/androgynous) and age category (child/teenager/young_adult/adult/elderly)
2. **Speech enhancement**: Each audio sample processed through [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio) `MossFormer2_SE_48K` model for noise suppression and speech clarity improvement at 48kHz
3. **Format conversion**: Original MP3 files replaced with enhanced WAV files at 48kHz sample rate
4. **Emotion annotation**: All enhanced samples annotated with [Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus), providing 59 scores per sample
5. **Metadata update**: JSON sidecar files updated with all annotation scores

## Structure

```
reference-voices-enhanced.tar
├── male/
│ ├── 02_child/ (1 sample)
│ ├── 03_teenager/ (5 samples)
│ ├── 04_young_adult/ (217 samples)
│ ├── 05_adult/ (813 samples)
│ └── 08_elderly/ (1 sample)
├── female/
│ ├── 02_child/ (16 samples)
│ ├── 03_teenager/ (3 samples)
│ ├── 04_young_adult/ (376 samples)
│ ├── 05_adult/ (514 samples)
│ └── 08_elderly/ (1 sample)
└── androgynous/
├── 02_child/ (13 samples)
├── 04_young_adult/ (19 samples)
└── 05_adult/ (25 samples)
```

Each sample consists of:
- **`.wav`** — Enhanced audio file (48kHz, WAV format)
- **`.json`** — Metadata sidecar with caption, emotion scores, and quality scores

## Gender Distribution

| Gender | Count |
|---|---|
| Male | 1,037 |
| Female | 910 |
| Androgynous | 57 |
| **Total** | **2,004** |

## Annotations

### Quality Scores (4 dimensions)

All samples were quality-filtered in the source dataset with:
- `score_background_quality >= 3.5` (DNS MOS background quality)
- `score_content_enjoyment >= 5.0` (content enjoyment rating)

The full set of quality scores in each JSON sidecar:

| Score | Description |
|---|---|
| `score_background_quality` | DNS MOS background quality rating |
| `score_content_enjoyment` | Content enjoyment rating |
| `score_overall_quality` | Overall audio quality rating |
| `score_speech_quality` | Speech-specific quality rating |

### Emotion Scores (55 dimensions)

Each JSON sidecar contains 55 emotion and vocal characteristic scores from Empathic Insight Voice Plus, including:

- **Core emotions**: Anger, Disgust, Fear, Sadness, Joy/Happiness, Contentment, Amusement, Affection, Awe
- **Complex emotions**: Contempt, Confusion, Distress, Disappointment, Bitterness, Nostalgia, Guilt/Shame, Envy/Jealousy
- **Social/cognitive**: Concentration, Contemplation, Determination, Pride, Relief, Sarcasm/Irony, Triumph
- **Surprise variants**: Astonishment/Surprise, Excitement
- **Vocal characteristics**: Age, Arousal, Valence, Dominance, Authenticity, Monotone vs. Expressive, Confident vs. Hesitant, Formal vs. Casual, Fast vs. Slow, Loud vs. Soft, Staccato vs. Legato, Tense vs. Relaxed, Nasal, Breathy/Whisper, Creaky/Vocal Fry, Trembling/Shaky, Lisp/Speech Impediment, Accent Strength
- **Additional**: Background Noise, Music/Singing, Laughter, Non-speech Sounds, Reverberation, Multiple Speakers

## Speech Enhancement Details

The [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio) MossFormer2_SE_48K model is a state-of-the-art speech enhancement model that:

- Removes background noise while preserving speech quality
- Operates at 48kHz for high-fidelity output
- Uses the MossFormer2 architecture optimized for speech enhancement (SE)
- Produces clean, broadcast-quality speech suitable for TTS reference voices

## Source Dataset Lineage

```
laion/reference_ai_voices_with_timbre_annotations (17 tar files, ~32,000 samples)
│
├── Quality filtering (score_background_quality >= 3.5, score_content_enjoyment >= 5.0)
│ → 11,473 samples
│
├── Speaker deduplication (Orange/Speaker-wavLM-tbr embeddings, agglomerative clustering)
│ → 2,004 unique speakers
│
└── laion/ai-voices-deduplicated (2,004 samples, MP3)
│
├── ClearerVoice MossFormer2_SE_48K speech enhancement
├── Format conversion to 48kHz WAV
├── Empathic Insight Voice Plus annotation (59 scores)
│
└── laion/reference-voices-enhanced (2,004 samples, WAV @ 48kHz) ← this dataset
```

## Usage

```python
from huggingface_hub import hf_hub_download
import tarfile

# Download the tar file
path = hf_hub_download(
"laion/reference-voices-enhanced",
"reference-voices-enhanced.tar",
repo_type="dataset"
)

# Extract
with tarfile.open(path) as tar:
tar.extractall("./reference-voices-enhanced")
```

## Citation

If you use this dataset, please cite the source dataset and the tools used:

- [LAION AI Voices Deduplicated](https://huggingface.co/datasets/laion/ai-voices-deduplicated)
- [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)
- [Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus)
- [Orange Speaker-wavLM-tbr](https://huggingface.co/Orange/Speaker-wavLM-tbr)

## License

CC-BY-4.0

许可证：CC-BY-4.0
任务类别：
- 音频分类
- 文本转语音
标签：
- 语音
- 说话人嵌入
- 去重
- 情感
- 质量过滤
- 语音增强
- ClearerVoice
数据集名称：增强型参考语音
样本量范围：1000~10000

# 增强型参考语音数据集
本数据集包含2004条经[ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)的MossFormer2_SE_48K语音增强模型处理的AI语音样本，且已通过[Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus)完成标注（涵盖59项质量与情感评分）。

## 数据集概览
- **数据源**：[laion/ai-voices-deduplicated](https://huggingface.co/datasets/laion/ai-voices-deduplicated)（包含2004条经过说话人去重与质量过滤的AI语音样本）
- **语音增强**：采用ClearerVoice MossFormer2_SE_48K模型完成背景噪声去除与语音清晰度提升
- **输出格式**：将原始MP3文件替换为48kHz采样率的增强版WAV音频文件
- **标注信息**：完整的Empathic Insight Voice Plus标注评分（共59个维度，其中55项为情感评分，4项为质量评分）
- **元数据**：为每个样本添加附带所有标注评分的JSON附属文件
- **打包方式**：单个tar归档文件

## 增强流程
1. **源数据**：取自`laion/ai-voices-deduplicated`的2004条样本，按性别（男性/女性/中性）与年龄组别（儿童/青少年/青年/成人/老年人）进行分类组织
2. **语音增强**：将每条音频样本通过[ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)的`MossFormer2_SE_48K`模型处理，实现48kHz采样率下的噪声抑制与语音清晰度提升
3. **格式转换**：将原始MP3文件替换为48kHz采样率的增强版WAV文件
4. **情感标注**：所有增强后的样本均通过[Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus)完成标注，每条样本包含59项评分
5. **元数据更新**：为每个样本生成包含所有标注评分的JSON附属文件

## 数据集结构

reference-voices-enhanced.tar
├── male/
│ ├── 02_child/ (1 条样本)
│ ├── 03_teenager/ (5 条样本)
│ ├── 04_young_adult/ (217 条样本)
│ ├── 05_adult/ (813 条样本)
│ └── 08_elderly/ (1 条样本)
├── female/
│ ├── 02_child/ (16 条样本)
│ ├── 03_teenager/ (3 条样本)
│ ├── 04_young_adult/ (376 条样本)
│ ├── 05_adult/ (514 条样本)
│ └── 08_elderly/ (1 条样本)
└── androgynous/
├── 02_child/ (13 条样本)
├── 04_young_adult/ (19 条样本)
└── 05_adult/ (25 条样本)

每条样本包含以下文件：
- **`.wav`**：增强后的音频文件（48kHz采样率，WAV格式）
- **`.json`**：附带元数据的附属文件，包含文本说明、情感评分与质量评分

## 性别分布
| 性别 | 样本数量 |
|---|---|
| 男性 | 1037 |
| 女性 | 910 |
| 中性 | 57 |
| **总计** | **2004** |

## 标注信息
### 质量评分（4个维度）
源数据集中已对所有样本完成质量过滤，过滤条件为：
- `score_background_quality ≥ 3.5`（DNS MOS背景质量评分）
- `score_content_enjoyment ≥ 5.0`（内容愉悦度评分）

每个JSON附属文件中包含的完整质量评分项如下：
| 评分项 | 描述 |
|---|---|
| `score_background_quality` | DNS MOS背景质量评分 |
| `score_content_enjoyment` | 内容愉悦度评分 |
| `score_overall_quality` | 整体音频质量评分 |
| `score_speech_quality` | 语音专项质量评分 |

### 情感评分（55个维度）
每个JSON附属文件中包含Empathic Insight Voice Plus生成的55项情感与语音特征评分，涵盖以下类别：
- **核心情感**：愤怒、厌恶、恐惧、悲伤、喜悦/快乐、满足、愉悦、喜爱、敬畏
- **复杂情感**：轻蔑、困惑、苦恼、失望、苦涩、怀旧、内疚/羞耻、嫉妒/妒忌
- **社会/认知类**：专注、沉思、坚定、自豪、释然、讽刺/反语、胜利感
- **惊讶变体**：惊愕/惊讶、兴奋
- **语音特征**：年龄、唤醒度、效价、支配度、真实性、单调/富有表现力、自信/犹豫、正式/随意、快速/缓慢、响亮/轻柔、断奏/连奏、紧张/放松、鼻音、气息声/耳语声、沙声/喉音、颤抖/不稳、口吃/言语障碍、口音强度
- **附加项**：背景噪声、音乐/歌唱、笑声、非语音声音、混响、多说话人

## 语音增强细节
[ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)的MossFormer2_SE_48K模型是一款前沿语音增强模型，具备以下特性：
- 去除背景噪声的同时保留语音质量
- 支持48kHz采样率，实现高保真输出
- 采用专为语音增强（SE）优化的MossFormer2架构
- 生成干净的广播级语音，适用于文本转语音（TTS）的参考语音

## 源数据集谱系

laion/reference_ai_voices_with_timbre_annotations （共17个tar归档文件，约32000条样本）
│
├── 质量过滤（`score_background_quality ≥ 3.5`且`score_content_enjoyment ≥5.0`）
│ → 11473条样本
│
├── 说话人去重（采用Orange/Speaker-wavLM-tbr嵌入与凝聚式聚类算法）
│ → 2004名独特说话人
│
└── laion/ai-voices-deduplicated （2004条样本，MP3格式）
│
├── ClearerVoice MossFormer2_SE_48K语音增强处理
├── 格式转换为48kHz WAV格式
├── Empathic Insight Voice Plus标注（59项评分）
│
└── laion/reference-voices-enhanced （2004条样本，48kHz WAV格式） ← 本数据集

## 使用方法
python
from huggingface_hub import hf_hub_download
import tarfile

# 下载tar归档文件
path = hf_hub_download(
"laion/reference-voices-enhanced",
"reference-voices-enhanced.tar",
repo_type="dataset"
)

# 解压文件
with tarfile.open(path) as tar:
tar.extractall("./reference-voices-enhanced")

## 引用说明
若使用本数据集，请引用以下源数据集与所用工具：
- [LAION AI Voices Deduplicated](https://huggingface.co/datasets/laion/ai-voices-deduplicated)
- [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)
- [Empathic Insight Voice Plus](https://huggingface.co/laion/Empathic-Insight-Voice-Plus)
- [Orange Speaker-wavLM-tbr](https://huggingface.co/Orange/Speaker-wavLM-tbr)

## 许可证
CC-BY-4.0