laions_got_talent_with_voice_emotion_speed_tags_for_orpheus_tuning
收藏资源简介：
**LAION's Got Talent: Generated Voice Acting Dataset** --- ### Overview "LAION's Got Talent" is a synthetic voice acting dataset designed to offer a broad range of emotional expressions, vocal bursts, and multi-language utterances. This dataset is a component of the BUD-E project, led by LAION with support from Intel, and aims to drive forward research in context-aware and empathetic AI voice assistants. --- ### Updated Composition 1. **Voices and Languages** - **English**: 11 OpenAI voices, each covering 40 distinct emotional categories. - **German, French, and Spanish**: 11 voices, each with 40 emotional categories. 2. **Utterances** - **English**: - Each of the 40 emotional categories contains approximately 1,000 utterances, each ~30 seconds. - Total ~ 11 (voices) × 40 (categories) × 1,000 (utterances) = **440,000 English utterances**. - **German, French, Spanish**: - Each of the 40 emotional categories contains ~250 utterances per voice. 3. **Vocal Diversity** - **Emotional Expressions**: Anger, joy, fear, sadness, excitement, etc. - **Vocal Bursts**: Laughter, sighs, gasps, and other non-verbal cues. 4. **Dataset Growth** - The dataset is still under construction. Future versions will include expanded metadata and improved emotion annotations. --- ### BUD-E Project This dataset supports **BUD-E (Buddy for Understanding and Digital Empathy)**, an open-source AI voice assistant initiative focused on naturalness, empathy, and high-quality conversational experiences. --- ### Data Format & Metadata The current release is processed to align with **Orpheus TTS** fine-tuning expectations, containing **text** and **audio** columns. Additional metadata fields include: - **`audio`**: Audio file or reference. - **`audioduration (s)`**: Duration of the audio sample in seconds. - **`emotion`**: High-level emotional label(s). - **`text`**: The text script for the utterance. - **`duration (float32)`**: The spoken length (seconds) of the utterance (often ~30 seconds). - **`whisper_transcription`**: Automatic transcription of the utterance via Whisper. - **`voice`**: The chosen voice identity (e.g., “nova”). - **`characters_per_second (int32)`**: Rounded speaking pace metric. - **`emotion_tags`**: A list of detailed emotion descriptors (e.g., `[ "acidity", "sourness", "tartness" ]`). - **`accent`**: Accent or locale-specific markers (e.g., “fr”). - **`language_code`**: Language indicator (e.g., `fr`, `en`, `de`, `es`). - **XML Format**: Contains speaker identity, speaking pace, emotion tags, and the target text to be spoken. Example snippet: ```xml <voice="nova";speed=14;emotion="";text="Ce jus de Thomas, oh là là ! ... Oh, je pourrais jamais m'y faire !"> ``` **Example Metadata Row** - **`emotion`**: intense sourness, tartness, acidity - **`voice`**: nova - **`characters_per_second`**: 14 - **`emotion_tags`**: [ "acidity", "sourness", "tartness" ] - **`accent`**: (not applicable or specific) - **`language_code`**: fr - **`whisper_transcription`**: “Ce jus de Thomas, oh là là ! ... Oh, je pourrais jamais m'y faire !” - **`duration`**: 28.958 --- ### Future Releases Planned updates include: - **Refined Emotion Captions**: More granular and accurate emotional descriptions. - **Extended Metadata**: Deeper annotations of speaker context, background noise, prosody, etc. --- ### Purpose & Use Cases 1. **Empathetic AI Assistants**: Training models to recognize and generate nuanced emotional speech. 2. **Text-to-Speech (TTS) Fine-Tuning**: Offering a structured format compatible with popular TTS pipelines (e.g., Orpheus TTS). 3. **Emotional Analysis**: Providing a breadth of vocal expressions to advance research in human–AI interactions. --- ### Acknowledgments This dataset was developed as part of the BUD-E project, led by **LAION** with support from **Intel**, **Nous Research** and **Camb AI**. We thank all contributors and collaborators for their efforts in curating and continuously improving this resource. Researchers and developers are encouraged to explore, experiment, and contribute to further enhance the capabilities of empathetic and context-aware AI voice assistants.
**LAION才艺计划：合成配音数据集（LAION's Got Talent: Generated Voice Acting Dataset）** ### 概述 "LAION才艺计划（LAION's Got Talent）"是一款合成配音数据集，旨在提供丰富多样的情感表达、人声突发音效及多语言语音语段。该数据集是由LAION主导、英特尔（Intel）提供支持的BUD-E项目的组成部分，旨在推动上下文感知型共情AI语音助手领域的研究发展。 ### 数据集更新构成 1. **语音与语言** - **英语**：11款OpenAI语音，每款覆盖40种独立情感类别。 - **德语、法语及西班牙语**：11款语音，每款覆盖40种情感类别。 2. **语音语段** - **英语**： - 40种情感类别中，每类约含1000条语音语段，每条时长约30秒。 - 总规模约为11（语音）×40（类别）×1000（语段）= **44万条英语语音语段**。 - **德语、法语、西班牙语**：每款语音的40种情感类别中，每类约含250条语音语段。 3. **人声多样性** - **情感表达**：愤怒、喜悦、恐惧、悲伤、兴奋等。 - **人声突发音效**：笑声、叹息、喘息及其他非语言听觉信号。 4. **数据集发展规划** 该数据集目前仍在构建中，未来版本将新增更丰富的元数据，并优化情感标注内容。 ### BUD-E项目 本数据集服务于**BUD-E（Buddy for Understanding and Digital Empathy，理解与数字共情伙伴）**，这是一项开源AI语音助手项目，致力于打造自然、共情且高品质的对话交互体验。 ### 数据格式与元数据 当前发布版本针对**Orpheus TTS**的微调需求进行了优化，包含`text`与`audio`两列数据。其余元数据字段如下： - **`audio`**：音频文件或引用路径。 - **`audioduration (s)`**：音频样本时长，单位为秒。 - **`emotion`**：高级情感标签。 - **`text`**：语音语段对应的文本脚本。 - **`duration (float32)`**：语音语段的时长（秒），通常约为30秒。 - **`whisper_transcription`**：通过Whisper自动生成的语音转写文本。 - **`voice`**：所选语音身份（例如"nova"）。 - **`characters_per_second (int32)`**：经取整后的语速指标（字符/秒）。 - **`emotion_tags`**：详细情感描述符列表（例如`[ "acidity", "sourness", "tartness" ]`）。 - **`accent`**：口音或地区标识（例如"fr"）。 - **`language_code`**：语言标识（例如`fr`、`en`、`de`、`es`）。 - **XML格式**：包含说话人身份、语速、情感标签及待合成的目标文本。示例片段如下： xml <voice="nova";speed=14;emotion="";text="Ce jus de Thomas, oh là là ! ... Oh, je pourrais jamais m'y faire !"> **元数据行示例** - **`emotion`**：强烈酸涩、尖刻、酸性 - **`voice`**：nova - **`characters_per_second`**：14 - **`emotion_tags`**：[ "acidity", "sourness", "tartness" ] - **`accent`**：（无适用口音或特定口音） - **`language_code`**：fr - **`whisper_transcription`**："Ce jus de Thomas, oh là là ! ... Oh, je pourrais jamais m'y faire !" - **`duration`**：28.958 ### 未来发布计划 规划中的更新内容包括： - **精细化情感标注**：更细致、准确的情感描述。 - **扩展元数据**：新增说话人背景、背景噪音、韵律等更深层次的标注内容。 ### 用途与应用场景 1. **共情型AI语音助手**：用于训练模型识别并生成细腻的情感语音。 2. **文本转语音（Text-to-Speech, TTS）微调**：提供与主流TTS处理管线（例如Orpheus TTS）兼容的结构化格式。 3. **情感分析**：提供丰富的人声表达样本，推动人机交互领域的研究进展。 ### 致谢 本数据集作为BUD-E项目的组成部分开发，由**LAION**主导，**Intel**、**Nous Research**及**Camb AI**提供支持。我们感谢所有贡献者与合作者为整理并持续优化该数据集所付出的努力。 我们鼓励研究人员与开发者探索、实验并参与贡献，以进一步提升共情型与上下文感知型AI语音助手的能力。
