ZonglinY/TOMATO-Star-SFT-Data-R1D-32B

--- license: cc-by-4.0 task_categories: - text-generation language: - en tags: - science - hypothesis-generation - inspiration-retrieval - sft - llama-factory - biomedical size_categories: - 100K<n<1M --- # TOMATO-Star SFT Data (R1D-32B) SFT training data for the two core tasks in MOOSE-Star: **Hypothesis Composition (HC)** and **Inspiration Retrieval (IR)**. All data is generated via **rejection sampling** with [DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) as the teacher model, followed by reranker filtering. All data is in **ShareGPT JSONL format**, directly compatible with [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory). ## Dataset Description - **Paper**: [MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier](https://arxiv.org/abs/2603.03756) - **Base Dataset**: [TOMATO-Star](https://huggingface.co/datasets/ZonglinY/TOMATO-Star) - **Teacher Model**: [DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) (for rejection sampling) - **License**: CC-BY-4.0 ## Files ### Hypothesis Composition (HC) | File | Samples | Description | |------|---------|-------------| | `HC/normal_composition.jsonl` | 96,879 | Standard HC: generate hypothesis from research question + background + inspirations | | `HC/bounded_composition.jsonl` | 17,669 | Bounded HC: generate hypothesis with imperfect (bounded) inspirations | | `HC/dataset_info.json` | - | LLaMA-Factory dataset config | **Recommended mixing**: Combine both files with bounded upsampled 1-2x (paper uses 1x for best results). ### Inspiration Retrieval (IR) | File | Samples | Description | |------|---------|-------------| | `IR/train.jsonl` | 150,218 | 15-way multiple choice: select correct inspiration from 15 candidates | | `IR/eval.jsonl` | 2,377 | Evaluation split (same format) | | `IR/dataset_info.json` | - | LLaMA-Factory dataset config | ## Data Format All files use ShareGPT multi-turn conversation format: ```json { "conversations": [ {"role": "user", "content": "[Task instruction + input data]"}, {"role": "assistant", "content": "[Model response]"} ] } ``` ### HC Task Format - **User**: System instruction for hypothesis composition + research question + background + inspirations (+ previous hypothesis components for bounded mode) - **Assistant**: Hypothesis with Motivation, Mechanism, and Methodology sections ### IR Task Format - **User**: Background information + 15 candidate papers (A-O), one correct + 14 hard negatives - **Assistant**: Selected inspiration ID + reasoning ## Usage with LLaMA-Factory ```bash # HC Training (mix normal + bounded with 1x upsample) # 1. Combine: cat normal_composition.jsonl bounded_composition.jsonl > mixed.jsonl # 2. Point dataset_info.json to the mixed file # 3. Run LLaMA-Factory SFT # IR Training # dataset_info.json is pre-configured, just point LLaMA-Factory to the IR/ directory ``` ### dataset_info.json (HC example) ```json { "train": { "file_name": "normal_composition.jsonl", "formatting": "sharegpt", "columns": {"messages": "conversations"}, "tags": { "role_tag": "role", "content_tag": "content", "user_tag": "user", "assistant_tag": "assistant" } } } ``` ## Training Details | Config | HC | IR | |--------|----|----| | Base Model | DeepSeek-R1-Distill-Qwen-7B | DeepSeek-R1-Distill-Qwen-7B | | Chat Template | deepseekr1 | deepseekr1 | | Cutoff Length | 8192 | 16384 | | Learning Rate | 1e-5 | 1e-5 | | Epochs | 1 | 1 | | Training | Full-param (ZeRO-3) | Full-param (ZeRO-3) | ## Data Generation Pipeline - **HC Normal**: Rejection sampling with DeepSeek-R1-Distill-Qwen-32B teacher → reranker filtering → SFT format conversion - **HC Bounded**: Same pipeline but with bounded (imperfect) inspirations selected by SPECTER2 embedding similarity - **IR**: Hard negative sampling (keyword overlap + embedding-based + random) → rejection sampling → SFT format conversion ## Citation ```bibtex @article{yang2025moosestar, title={MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier}, author={Yang, Zonglin and Bing, Lidong}, year={2025} } ``` ## License This dataset is released under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/) license.许可证：CC-BY-4.0 任务类别：文本生成 语言：英语 标签：科学、假设生成、灵感检索、监督微调（SFT）、LLaMA-Factory、生物医学 样本规模：10万<样本数<100万 # TOMATO-Star 监督微调（SFT）数据集（R1D-32B） 本数据集为MOOSE-Star两大核心任务的监督微调训练数据：**假设生成（Hypothesis Composition, HC）与灵感检索（Inspiration Retrieval, IR）。 所有数据均以[DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B)作为教师模型，通过**拒绝采样**生成，随后经过重排序器过滤。所有数据采用**ShareGPT JSONL格式**，可直接兼容[LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)。 ## 数据集说明 - **论文**：[MOOSE-Star: 打破复杂度壁垒，解锁科学发现的可训练性](https://arxiv.org/abs/2603.03756) - **基础数据集**：[TOMATO-Star](https://huggingface.co/datasets/ZonglinY/TOMATO-Star) - **教师模型**：[DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B)（用于拒绝采样） - **许可证**：CC-BY-4.0 ## 文件 ### 假设生成（HC） | 文件名 | 样本数 | 描述 | |--------|--------|--------| | `HC/normal_composition.jsonl` | 96879 | 标准假设生成：基于研究问题、背景材料与灵感生成假设 | | `HC/bounded_composition.jsonl` | 17669 | 受限假设生成：基于不完美（受限）灵感生成假设 | | `HC/dataset_info.json` | - | LLaMA-Factory 数据集配置文件 | **推荐混合方式**：将两个文件按受限数据上采样1-2倍的比例混合（论文中采用1倍上采样以获得最佳效果）。 ### 灵感检索（IR） | 文件名 | 样本数 | 描述 | |--------|--------|--------| | `IR/train.jsonl` | 150218 | 15选多项选择题任务：从15个候选灵感中选出正确灵感 | | `IR/eval.jsonl` | 2377 | 评估集（格式与训练集一致） | | `IR/dataset_info.json` | - | LLaMA-Factory 数据集配置文件 | ## 数据格式 所有文件均采用ShareGPT多轮对话格式： json { "conversations": [ {"role": "user", "content": "[任务指令 + 输入数据"}, {"role": "assistant", "content": "[模型回复]"} ] } ### HC任务格式 - **用户侧输入**：假设生成的系统指令 + 研究问题 + 背景材料 + 灵感（受限模式下还需加入此前的假设组件 - **助手侧输出**：包含动机、机制与方法论三个部分的假设内容 ### IR任务格式 - **用户侧输入**：背景信息 + 15篇候选论文（编号A-O），其中1篇为正确灵感，14篇为难负样本 - **助手侧输出**：选中的灵感编号 + 推理过程 ## LLaMA-Factory使用方法 bash # HC 训练（将标准与受限数据按1倍上采样比例混合） # 1. 合并文件：cat normal_composition.jsonl bounded_composition.jsonl > mixed.jsonl # 2. 将 dataset_info.json 指向合并后的文件 # 3. 运行 LLaMA-Factory 监督微调任务 # IR 训练 # dataset_info.json 已预先配置，仅需将 LLaMA-Factory 指向 IR/ 目录即可 ### 示例 dataset_info.json（HC示例） json { "train": { "file_name": "normal_composition.jsonl", "formatting": "sharegpt", "columns": {"messages": "conversations"}, "tags": { "role_tag": "role", "content_tag": "content", "user_tag": "user", "assistant_tag": "assistant" } } } ## 训练配置 | 配置项 | HC任务 | IR任务 | |--------|--------|--------| | 基础模型 | DeepSeek-R1-Distill-Qwen-7B | DeepSeek-R1-Distill-Qwen-7B | | 对话模板 | deepseekr1 | deepseekr1 | | 截断长度 | 8192 | 16384 | | 学习率 | 1e-5 | 1e-5 | | 训练轮次 | 1 | 1 | | 训练方式 | 全参数微调（ZeRO-3） | 全参数微调（ZeRO-3） | ## 数据生成流程 - **标准HC数据生成流程**：以DeepSeek-R1-Distill-Qwen-32B为教师模型进行拒绝采样 → 重排序器过滤 → 转换为SFT格式 - **受限HC数据生成流程**：流程与标准HC一致，但采用SPECTER2嵌入相似度筛选出的受限（不完美）灵感 - **IR数据生成流程**：难负样本采样（基于关键词重叠、嵌入相似度与随机采样） → 拒绝采样 → 转换为SFT格式 ## 引用 bibtex @article{yang2025moosestar, title={MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier}, author={Yang, Zonglin and Bing, Lidong}, year={2025} } ## 许可证 本数据集采用[CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)许可证发布。