AbstractPhil/json-coco-format
收藏资源简介：
--- license: cc-by-4.0 task_categories: - text-generation language: - en pretty_name: JSON COCO Format — task-differentiated SFT data size_categories: - 10K<n<100K configs: - config_name: task_1 data_files: - split: train path: data/task_1.jsonl - config_name: task_2 data_files: - split: train path: data/task_2.jsonl - config_name: task_3 data_files: - split: train path: data/task_3.jsonl - config_name: all data_files: - split: train path: data/all.jsonl tags: - sft - qwen3.5 - tool-use - structured-output - coco - synthetic --- # JSON COCO Format — task-differentiated SFT data A multi-task supervised fine-tuning dataset that teaches a model to convert image-synthesis caption prompts into JSON whose **structure varies by task**. Built from MS-COCO captions (Karpathy split) with Claude Sonnet 4.6 as the teacher; designed for training per-task LoRAs on [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B). Each row is in the Qwen3.5-native tool-call shape: a `messages` array with an assistant turn whose `tool_calls[0].function.arguments` is a dict matching the caption schema, plus a `tools` sibling carrying the per-task tool definition. `tokenizer.apply_chat_template(messages, tools=tools, enable_thinking=False)` produces the qwen3_coder XML training target without further transformation. ## The three tasks The same caption produces three structurally distinct outputs, one per task: ### `task_1` — hallucination_reduction Grounded literal extraction. Subject/action/attribute values come from the caption verbatim. Style and mood are forbidden (always `null`). The validator runs a per-leaf grounding check against the input caption — anything not traceable to the caption is rejected. ```json { "subjects": [ {"name": "bicycle", "attributes": ["black"]}, {"name": "kitchen cabinets", "attributes": []} ], "actions": ["leaning against the kitchen cabinets"], "setting": "indoor", "style": null, "mood": null } ``` ### `task_2` — useful_generalization Encouraged categorical abstraction. Every open-vocab string is a bracketed canonical generic like `[pet]`, `[vehicle]`, `[playing]`. Schema constrains values to regex `^\[[a-z_]+\]$`. The validator enforces the format; semantic correctness is intentionally a soft target — the open vocabulary is what emerged from Claude's actual outputs on real captions. ```json { "subjects": [ {"name": "[vehicle]", "attributes": ["[color]"]}, {"name": "[furniture]", "attributes": []} ], "actions": ["[resting]"], "setting": "[indoor]", "style": null, "mood": null } ``` ### `task_3` — generic_symbolism Pure positional placeholders. Each slot has its own type prefix and 1-based monotonic index. The output captures structural roles with zero semantic content — two completely different captions with the same shape produce identical JSON. ```json { "subjects": [ {"name": "[ENTITY_1]", "attributes": ["[ATTRIBUTE_1]"]}, {"name": "[ENTITY_2]", "attributes": []} ], "actions": ["[ACTION_1]"], "setting": "[INDOOR]", "style": null, "mood": null } ``` ## Dataset statistics | Config | Accepted rows | Rejected (audit only) | Accept rate | |---|---:|---:|---:| | `task_1` (hallucination_reduction) | **20,505** | 2,762 | 88.1% | | `task_2` (useful_generalization) | **21,918** | 159 | 99.3% | | `task_3` (generic_symbolism) | **21,962** | 72 | 99.7% | | `all` (combined, task-tagged) | **64,385** | 2,993 | 95.6% | - Source captions: 22,000 unique COCO captions (min 4 words, deduplicated) - Same 22,000 captions processed by all three tasks - Total rows across tasks: 64,385 accepted, 2,993 rejected - Cost-to-build: ~$293 USD at `claude-sonnet-4-6` rates with prompt caching ## Row schema Every row follows the same outer shape. The variable part is `messages[2].tool_calls[0].function.arguments`, which is the task-shaped JSON. ```json { "prompt": "<source COCO caption verbatim>", "task": "task_1" | "task_2" | "task_3", "messages": [ {"role": "system", "content": "<task-specific system prompt>"}, {"role": "user", "content": "<source caption>"}, {"role": "assistant", "tool_calls": [{ "type": "function", "function": { "name": "emit_caption_schema", "arguments": { "subjects": [...], "actions": [...], "setting": "...", "style": null, "mood": null } } }]} ], "tools": [{"type": "function", "function": { "name": "emit_caption_schema", "description": "Emit the structured caption representation.", "parameters": { ...task-specific JSON Schema overlay... } }}], "meta": { "model": "claude-sonnet-4-6", "schema_valid": true, "validator_passed": true, "tokens_in_uncached": 437, "tokens_out": 145, "tokens_cache_read": 1034, "tokens_cache_create": 0, "cost_usd": 0.00328, "timestamp": "2026-05-14T..." } } ``` The `arguments` field is a real Python dict (not a stringified JSON), which is what Qwen3.5's chat template requires — it iterates `arguments.items()` to emit one `<parameter=NAME>` block per key. ## Usage — load a single task ```python from datasets import load_dataset ds = load_dataset("AbstractPhil/json-coco-format", "task_1", split="train") print(ds[0]["prompt"]) print(ds[0]["messages"][2]["tool_calls"][0]["function"]["arguments"]) ``` ## Usage — produce a Qwen3.5 training prompt ```python from transformers import AutoProcessor processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-0.8B", trust_remote_code=True) row = ds[0] prompt_text = processor.apply_chat_template( row["messages"], tools=row["tools"], add_generation_prompt=False, # full conversation including assistant turn tokenize=False, enable_thinking=False, ) print(prompt_text) # <|im_start|>system # # Tools ... <tools>{...}</tools> # ...task system prompt... # <|im_end|> # <|im_start|>user # <caption> # <|im_end|> # <|im_start|>assistant # <think> # # </think> # # <tool_call> # <function=emit_caption_schema> # <parameter=subjects> # [...] # </parameter> # ... # </function> # </tool_call><|im_end|> ``` ## How this dataset was made 1. Stream 22,000 unique captions from [yerevann/coco-karpathy](https://huggingface.co/datasets/yerevann/coco-karpathy) train split (min 4 words, deduplicated, take order from the stream). 2. For each caption × each task, prompt Claude Sonnet 4.6 via the Anthropic API using a task-specific system prompt + a task-specific tool schema overlay on a shared base `Caption` JSON Schema. 3. Validate every output: - `task_1`: per-leaf grounding check (every leaf string must trace to the source caption by substring or token match). - `task_2`: every open-vocab value must match `^\[[a-z_]+\]$`. - `task_3`: typed numbered placeholders (`[ENTITY_N]`, `[ATTRIBUTE_N]`, `[ACTION_N]`) with monotonic numbering within each slot. 4. Accepted rows → `data/task_N.jsonl`. Rejected rows → `rejects/task_N.jsonl` with reasons, retained for audit. The combined task-tagged dataset is at `data/all.jsonl`. Per-task system prompts and validators live in the [qwen-test-runner](https://github.com/AbstractEyes/qwen-test-runner) repo's `qwen_test_runner/tasks.py`. The build pipeline (`prompt_maker.py`) is resumable — re-running picks up from the last on-disk row via a per-task seen-set of input-prompt hashes. ## Rejection patterns (so you know what was filtered out) - **task_1 (2,762 rejects):** virtually all are grounding flags — Claude tends to phrase actions slightly beyond literal, e.g. `"flower arrangement placed in the middle of the table"` for a caption that says `"with a flower arrangement in the middle"`. The validator is intentionally strict for the hallucination-reduction objective. - **task_2 (159) / task_3 (72):** schema slips — Claude occasionally wrapping an action in a dict instead of a plain string (`{"name": "[resting]"}`), or one missing-bracket typo (`"[young"`). Rare tail-cases. `rejects/task_N.jsonl` rows include the raw model output and reject reasons in case you want to relax the validator or recover them. ## Intended use - Train per-task LoRAs on Qwen3.5-0.8B (or other small instruction-tuned models that emit qwen3_coder-format tool calls). - Each task is a distinct training distribution; one LoRA per task is the default plan. A single multi-task LoRA could also be trained off `all` with `row["task"]` as a conditioning tag. ## Source attribution and license - Source captions: MS-COCO captions (Chen et al., 2015), Karpathy split as redistributed in [yerevann/coco-karpathy](https://huggingface.co/datasets/yerevann/coco-karpathy). Licensed CC-BY-4.0. - Generated JSON outputs: produced by Anthropic's Claude Sonnet 4.6 under user-provided prompts. - This derivative dataset is released under **CC-BY-4.0**, preserving the upstream COCO license. ## Companion repo Build pipeline, task registry, validators, and Qwen test harness: [github.com/AbstractEyes/qwen-test-runner](https://github.com/AbstractEyes/qwen-test-runner)
JSON COCO Format — task-differentiated SFT data is a multi-task supervised fine-tuning dataset that teaches a model to convert image-synthesis caption prompts into JSON whose structure varies by task. Built from MS-COCO captions (Karpathy split) with Claude Sonnet 4.6 as the teacher; designed for training per-task LoRAs on Qwen/Qwen3.5-0.8B. The dataset includes three tasks: task_1 (hallucination_reduction) requires grounded literal extraction from captions; task_2 (useful_generalization) encourages categorical abstraction with canonical generic terms; task_3 (generic_symbolism) uses pure positional placeholders. It contains 64,385 accepted rows and 2,993 rejected rows, licensed under CC-BY-4.0.
