eagle0504/multireward-grpo-gsm8k-rewards

该数据集包含从Qwen2.5-1.5B-Instruct模型在GSM8K测试提示上生成的原始奖励观察数据，用于多奖励GRPO（Group Policy Optimization或相关方法）研究。数据集基于150个GSM8K测试提示，每个提示通过16个独立种子和每个种子32次生成，共512次生成结果。每次生成通过三个奖励通道评分：正确性（Bernoulli类型，判断最终答案是否匹配GSM8K标准答案）、长度（连续值，基于令牌数的变换函数，峰值在200个令牌）和格式（Bernoulli类型，检查响应是否包含特定格式标记如oxed{...}或#### N）。数据集提供了原始奖励张量、未污染的长度奖励以及相关元数据，支持论文中定理3（相关性依赖的MSE下限）和命题4（符号变化条件偏差）的实证分析。数据用于验证多奖励优势估计的有限样本理论，并展示奖励通道间的相关性（如正确性与长度之间的强负相关）。

Raw rollout-level reward observations from the empirical section of Conditioned Multi-Reward Advantage Estimation: A Finite-Sample Analysis. The dataset contains rewards for 150 GSM8K test prompts, with 512 rollouts per prompt (16 independent seeds × 32 rollouts per seed), generated from Qwen2.5-1.5B-Instruct at temperature 0.7. Each rollout is scored on three verifiable reward channels: correctness (Bernoulli, matching GSM8K gold answer), length (continuous on [-1, 0], based on token count with a target peak at 200 tokens), and format (Bernoulli, presence of oxed{...} or #### N). The data includes raw reward tensors, uncontaminated length rewards, and metadata, supporting the empirical analysis of Theorem 3 (correlation-dependent MSE floor) and Proposition 4 (sign-changing conditioning bias) in the paper.