instruction-following-RL-FR-broadened

# instruction-following-RL-FR-broadened French **instruction-following RLVR** (Reinforcement Learning with Verifiable Rewards) training data for [Luth-2](https://huggingface.co/kurakurai). Each example is a French task with one or more **programmatically verifiable** constraints; the reward is computed by rule-based French IFEval verifiers (no judge model). This is the **broadened** recipe — the best-performing of the recipes we tested (see *Results*). Design follows AllenAI's *Generalizing Verifiable Instruction Following* / [IFBench](https://arxiv.org/abs/2507.02833) and Nvidia's [Nemotron-Cascade 2](https://arxiv.org/abs/2603.19220): train on a broad, verifiable constraint set and measure generalization on **held-out** constraint types. ## Conception & methodology ### Constraint taxonomy (42 types) Three **disjoint** registries; this dataset trains on the first two: | set | count | role | |---|---|---| | in-domain `fr:*` | **30** | the FULL IFEval-fr / Multi-IF-fr taxonomy | | `fr:ood:*` train-aug | **12** | broadening pool (added for generalization) | | `fr:ood:*` held-out | 12 | **excluded** — the OOD test set, [kurakurai/LuthBench-fr](https://huggingface.co/datasets/kurakurai/LuthBench-fr) | - **30 in-domain** (`fr:`): keywords (existence, frequency, forbidden_words, letter_frequency), length (number_sentences/paragraphs/words, nth_paragraph_first_word), content (number_placeholders, postscript, informal_address, no_digits), format (number_bullet_lists, constrained_response, number_highlighted_sections, multiple_sections, json_format, title), combination (two_responses, repeat_prompt), startend (end_checker, quotation), change_case (capital_word_frequency, french_capital, french_lowercase), punctuation (no_comma), special_character (ethel_or_cedilla, no_accents, accents), language (response_language). - **12 train-aug** (`fr:ood:`): markdown_table, numbered_list, code_block, markdown_link, blockquote, char_limit, num_questions, num_emojis, date_ddmmyyyy, euro_amount, percentage, email. - **12 held-out** (never trained on): hashtags, min_unique_words, number_count, all_punctuation, no_periods, paragraphs_divider, custom_sep_list, keyword_nth_sentence, parenthetical, time_format, phone_fr, italic_underscore. ### Base prompts Real French tasks — the first user turn of **instruction-following sources** in [`kurakurai/luth-2-mixture`](https://huggingface.co/datasets/kurakurai/luth-2-mixture): `dolci_precise_if`, `smoltalk2`, `persona_instruct`, `multi_turn_if_fr_gemma4`. Deduplicated, light quality filtering. A pool of ~120k unique base prompts is collected; **single_turn and multi_turn draw from disjoint halves**, so no base task is shared between the two configs and none is reused. ### Constraint attachment For each base prompt, 1–N verifiable constraints are stacked **conflict-free** via a tag-based mutual-exclusion model (mutually unsatisfiable constraints — e.g. all-uppercase + section markers, `no_digits` + `Section 1`, required-word ∩ forbidden-word, contradictory length bounds — can never co-occur; **0 unsatisfiable pairs** in the released data). Each constraint's **realized kwargs are captured** from the vendored French IFEval checker, so the training reward and the eval grade an identical constraint instance (no train/eval drift). - `single_turn` (25,000): IFEval format, 1–5 constraints/prompt (mean ~2.4). - `multi_turn` (25,000): Multi-IF format, 2–3 turn **cumulative** conversation (turn 1 = task + first constraint; each later turn states one new constraint; all must still hold). ### Cleanliness (verified) - **Held-out never present** (asserted at build + checked empirically). - **Conflict-safe:** 0 tag-conflicting / unsatisfiable constraint pairs. - **Decontaminated** vs `kurakurai/IFEval-FR` + `facebook/Multi-IF` (French), 8-gram overlap (Tülu-3/OLMo style, threshold 0.5): **0 task-prompt leakage** (the only n-gram matches are Multi-IF later-turn *constraint sentences* that share the IFEval constraint template by design). - **Verifier parity:** every row round-trips through the vendored verifier with no error. ## Columns - `single_turn`: `prompt`, `instruction_id_list` (JSON string list of `fr:*` / `fr:ood:*` ids), `kwargs` (JSON string list of per-constraint kwarg dicts), `source`. - `multi_turn`: `prompt` (turn-1 content), `info` (JSON string `{"turns":[{prompt, instruction_id_list, kwargs}, ...]}`, ids cumulative per turn), `source`. ## Results RLVR training on this data (GRPO, dense reward, ~1000 steps), evaluated with sampling (temp 0.6 / top_p 0.95 / top_k 20), n_runs=3: | model | IFEval-fr (prompt strict) | Multi-IF-fr (turn-3 composite) | LuthBench-fr (held-out OOD) | |---|---|---|---| | base Luth-2-0.8B | 59.2 | 44.1 | 42.3 | | **broadened @ step_900** | **80.3** | **64.5** | **69.6** | The broadened recipe generalizes best: it lifts the **held-out OOD** benchmark by **+27 points** over base (constraint types never seen in training), while matching an in-domain-only model on IFEval-fr and decisively beating it on Multi-IF-fr and OOD. Clean, unique, disjoint base prompts (this dataset) further improved OOD **+6.6** vs an earlier reused-prompt build. ## Related - [kurakurai/LuthBench-fr](https://huggingface.co/datasets/kurakurai/LuthBench-fr) — the 12 held-out `fr:ood:*` constraints, the OOD generalization benchmark. - [kurakurai/luth-2-mixture](https://huggingface.co/datasets/kurakurai/luth-2-mixture) — base-prompt source. ## References - *Generalizing Verifiable Instruction Following* (IFBench), arXiv:2507.02833 - *Nemotron-Cascade 2: Cascade RL + Multi-Domain On-Policy Distillation*, arXiv:2603.19220 - *Tülu 3*, arXiv:2411.15124