TTS-AGI/voice-taxonomy-val

---
license: cc-by-4.0
task_categories:
- audio-classification
tags:
- voice
- speech
- taxonomy
- whisper
- gemini
- tts
- voice-attributes
- evaluation
size_categories:
- 1K<n<10K
---

# Voice Taxonomy Validation Dataset (Gemini 3.1 Pro)

**~1,072 speech samples** annotated with **57 voice taxonomy dimensions** (0-6 ordinal scale) by **Gemini 3.1 Pro**. This is the gold-standard evaluation set for voice attribute classifiers.

## Related Datasets

| Dataset | Purpose | Link |
|---------|---------|------|
| Pre-training (large, Whisper ensemble) | Pre-training | [TTS-AGI/voice-taxonomy-pretrain](https://huggingface.co/datasets/TTS-AGI/voice-taxonomy-pretrain) |
| Fine-tuning (balanced, Gemini Flash) | Fine-tuning | [TTS-AGI/voice-taxonomy-flash-train](https://huggingface.co/datasets/TTS-AGI/voice-taxonomy-flash-train) |
| **This dataset** | Evaluation (gold standard) | — |

## Format

WebDataset TAR with MP3+JSON pairs:
```
{stem}.mp3 # Audio (mono, 44.1kHz, 64kbps, ≤30s)
{stem}.json # 57-dim taxonomy annotation
```

Each JSON:
```json
{
"AGEV": {"value": 3, "name": "Perceived Age", "label": "young adult"},
"GEND": {"value": 5, "name": "Gender Presentation", "label": "standard masculine"},
...
}
```

## Evaluation

```bash
# Download
huggingface-cli download TTS-AGI/voice-taxonomy-val --local-dir val

# Evaluate a trained model
python train_voice_taxonomy.py --phase eval --encoder laion/BUD-E-Whisper --gpu 0 \
--resume checkpoints/finetune_best.pt \
--val-tar val/voice_taxonomy_val.tar
```

## Metrics

| Metric | Description |
|--------|-------------|
| **Exact accuracy** | Prediction == ground truth |
| **Adj1 (primary)** | Prediction within ±1 of ground truth |
| **Mean difference** | Average |prediction - truth| |

### Baseline Results

| Model | Exact | Adj1 | Diff |
|-------|-------|------|------|
| V1.0 frozen + MLP | 0.235 | 0.633 | 1.40 |
| V1.1 frozen + MLP | 0.260 | 0.635 | 1.35 |
| V1.0 full finetune | 0.282 | 0.648 | — |
| Random baseline | 0.143 | 0.367 | 1.95 |

## Training Plan

See [TRAINING_PLAN.md](TRAINING_PLAN.md) for the full training strategy and `train_voice_taxonomy.py` for a self-contained training script.

## Labels

Labels were generated by **Gemini 3.1 Pro** — the most capable model in the annotation pipeline. These serve as the gold standard for evaluation.

## Taxonomy

57 dimensions covering: speaker identity, timbral quality, resonance placement, prosody, articulation, emotion, and speaking style. See `taxonomy_labels.json` for full definitions.

---
license: cc-by-4.0
任务类别：
- 音频分类
标签：
- 语音
- 语音
- 分类学
- Whisper（Whisper）
- Gemini（Gemini）
- TTS（TTS）
- 语音属性
- 评估
样本规模：
- 1K<n<10K
---

# 语音分类学验证数据集（Gemini 3.1 Pro）

**约1072条语音样本** 由 **Gemini 3.1 Pro** 按照 **57项语音分类学维度**（0至6的序数量表）完成标注。本数据集为语音属性分类器提供了金标准评估集。

## 相关数据集

| 数据集 | 用途 | 链接 |
|---------|---------|------|
| 预训练数据集（大规模，Whisper集成模型） | 预训练 | [TTS-AGI/voice-taxonomy-pretrain](https://huggingface.co/datasets/TTS-AGI/voice-taxonomy-pretrain) |
| 微调数据集（均衡化，Gemini Flash） | 微调 | [TTS-AGI/voice-taxonomy-flash-train](https://huggingface.co/datasets/TTS-AGI/voice-taxonomy-flash-train) |
| **本数据集** | 评估（金标准） | — |

## 数据集格式

采用WebDataset TAR打包格式，包含MP3与JSON成对文件：

{stem}.mp3 # 音频（单声道，44.1kHz采样率，64kbps码率，时长≤30秒）
{stem}.json # 57维分类学标注文件

每个JSON文件格式如下：
json
{
"AGEV": {"value": 3, "name": "感知年龄", "label": "青年成年人"},
"GEND": {"value": 5, "name": "性别表现", "label": "标准男性化"},
...
}

## 评估流程

bash
# 下载数据集
huggingface-cli download TTS-AGI/voice-taxonomy-val --local-dir val

# 评估已训练模型
python train_voice_taxonomy.py --phase eval --encoder laion/BUD-E-Whisper --gpu 0
--resume checkpoints/finetune_best.pt
--val-tar val/voice_taxonomy_val.tar

## 评估指标

| 评估指标 | 指标说明 |
|--------|-------------|
| **精确准确率** | 预测结果与标注完全一致 |
| **Adj1（核心指标）** | 预测结果与标注值相差不超过±1 |
| **平均差值** | |预测值 - 标注值| 的平均值 |

### 基准模型结果

| 模型 | 精确准确率 | Adj1指标 | 平均差值 |
|-------|-------|------|------|
| V1.0 冻结权重 + MLP | 0.235 | 0.633 | 1.40 |
| V1.1 冻结权重 + MLP | 0.260 | 0.635 | 1.35 |
| V1.0 全量微调 | 0.282 | 0.648 | — |
| 随机基准模型 | 0.143 | 0.367 | 1.95 |

## 训练方案

完整训练策略详见 [TRAINING_PLAN.md](TRAINING_PLAN.md)，独立训练脚本详见 `train_voice_taxonomy.py`。

## 标注说明

所有标签由 **Gemini 3.1 Pro** 生成，为标注流程中能力最强的模型，作为本评估集的金标准标注。

## 分类学维度

涵盖57项维度，涉及说话人身份、音色品质、共振位置、韵律、发音清晰度、情感与说话风格等。完整维度定义详见 `taxonomy_labels.json` 文件。