UPShf/FlowTalk-V1.1_ImageNet-1k-captions_captions-only

---
license: apache-2.0
pretty_name: UPShf/FlowTalk (planned) V1.1 - ImageNet-1k captions (captions only)
tags:
- image-captioning
- imagenet
- qwen
task_categories:
- image-to-text
language:
- en
size_categories:
- 100K<n<1M
---


# UPShf/FlowTalk (planned) V1.1 - ImageNet-1k captions (captions only)

This repository is intended to be the **captions-only** release used for *UPShf/FlowTalk (planned) V1.1* training.

It **does not** ship any ImageNet images (public redistribution is typically not permitted). Users must link these captions to **their own local copy** of ImageNet-1k images (for example the 256x256 variant from `benjamin-paine/imagenet-1k-256x256`).

Base images: `benjamin-paine/imagenet-1k-256x256`.

Caption generation pipeline: Sigma-Captioner (SGLANG branch):
https://github.com/uninterruptedpowersupply3-NEW/Sigma-Captioner/tree/SGLANG

## What is in the dataset

This repo can contain two **captions-only** JSONL formats (both are sharded, and may also be provided as a best store-level (in WinRar) ZIP with "Best" compression):

### A) Path-mapped captions (recommended)

Files:
- `mapped-*.jsonl` (keyed by ImageNet parquet `image.path`)

Each JSONL line looks like:

```json
{"imagenet_path":"...","split":"train|validation|test","label":123,"global_idx":456789,"caption":"...","caption_source":"..."}
```

Notes:
- `imagenet_path` is the stable identifier from the parquet column `image.path` in `benjamin-paine/imagenet-1k-256x256`.
- Some rows are intentionally missing (low quality filtering / evaluation removals), so you should expect a partial match vs the full ImageNet-1k set.

### B) pHash captions (robust to renames / slight re-encodes)

Files:
- `captions-*.jsonl` (keyed by a perceptual hash stored in the `sha256` field for historical reasons)

Each JSONL line looks like:

```json
{"sha256":"d8fa381417597327","hash_mode":"perceptual_hash_phash","caption":"...","caption_source":"...","image_filename":"..."}
```

Notes:
- `hash_mode=perceptual_hash_phash` means `sha256` is **NOT** a real SHA-256; it is a pHash string (16 hex chars).

### Build stats (current release)

- Captions exported / mapped: **891,549**
- JSONs with no caption (filtered out): **2,298**
- Full ImageNet-1k rows scanned (parquet): **1,431,167**

## Linking captions to your local images (order does not matter)

If your images are extracted in a different order (e.g. "image 12 is image 31"), you can still link captions by filename:
- Find the local file that matches `imagenet_path`.
- Use `split` to know whether it is train/validation/test.

If your local copy renamed files, use the pHash JSONL:
- Compute the same pHash (`imagehash.phash`) over your local images
- Join on the 16-hex string in the `sha256` field

Warning:
- pHash is designed for similarity, not cryptographic uniqueness. If you want extra safety, do a second-stage verification (e.g., also compare byte-sha256 after a candidate match).

## How to reproduce locally (Windows)

From the root of this workspace, this command:
- uses both caption sources (`CaptinedIMGNET` and `extracted_images`)
- reads only parquet metadata (`image.path`, `label`) and caption JSONs
- does not load image bytes

```powershell
python .\codexGPT5.2HIGH.py build-mapped-index `
--parquet_dir .\imagenet-1k-256x256\data `
--out_dir .\flowtalk_mapped_shards `
--out_zip .\flowtalk_v1.1_imagenet1k_captions_mapped_store.zip `
--batch_size 50000 --queue 50000 --shard_size 50000 --workers 8
```

To be explicit (no auto-detect), add:
- `--pairs_dir .\imagenet-1k-256x256\CaptinedIMGNET`
- `--pairs_dir .\imagenet-1k-256x256\extracted_images`

To generate the pHash JSONL:

```powershell
python .\codexGPT5.2HIGH.py export-captions `
--caption_dir .\imagenet-1k-256x256\CaptinedIMGNET `
--caption_dir .\imagenet-1k-256x256\extracted_images `
--out_dir .\flowtalk_phash_shards `
--hash_mode phash `
--shard_size 50000 --queue 5000 --workers 8 `
--skip_missing_images
```

### Performance notes

- `--pairs_lookup ram` (default) scans the caption directories once and does O(1) in-RAM lookups instead of millions of per-row `exists()` calls.
- Optional (faster JSON): `pip install orjson` (the script uses it automatically when available).
- Optional (progress bar): `pip install tqdm`.
- Optional (perceptual hashing for `export-captions --hash_mode phash`): `pip install ImageHash` (imports as `imagehash`).

## Licensing & Dataset Usage

*Disclaimer: This section is provided for informational purposes only and does not constitute legal advice.*

- **Captions & Metadata:** The text captions and JSON metadata generated in this repository are released under the **Apache-2.0** license.
- **Underlying Images:** The original ImageNet images are **NOT** redistributed in this repository. Users must obtain the images independently and comply with the official ImageNet terms of access, as well as any upstream dataset terms (such as `benjamin-paine/imagenet-1k-256x256`).
- **Copyright:** This dataset provides derivative text descriptions. Users are responsible for ensuring their use of the combined text and image data complies with all applicable licenses.

## Known Limitations & Bugs

- **Language & Vocabulary Constraints:** This dataset is intended to be entirely in English. However, because the captions and tags were generated by automated AI models, there are a few edge cases to be aware of:
- **Hallucinations:** Rare instances of non-English characters or words may occur due to standard Vision-Language Model hallucinations.
- **Loanwords & Entities:** Tag-based captions may include widely accepted loanwords (e.g., "taco", "sushi"), proper nouns, or domain-specific terminology that some strict language filters might flag as non-English.

If you are training a strict English-only model, you may want to apply a basic vocabulary filter to the text before training to catch any edge cases.

## Model credits (captioning)

Captions were produced using a multi-model pipeline, including:
- BLIP captions: `Salesforce/blip-image-captioning-large` (stored as `blip.caption`)
- Tag-style captions (previously labeled `wd_tagger` in JSON): `Qwen/Qwen3-VL-Embedding-2B` using `UPShf/Vocabulary-Qwen3-VL-Embedding-2B` (stored as `wd_tagger.caption`)
- Sigma-Captioner (SGLANG) QA captions: `Qwen/Qwen3.5-2B` (stored under `sglang.qa_pairs` in the raw JSON)

Captioner mix (from a metadata scan over **893,847** caption JSON files; selection order `sglang > blip > wd_tagger`):

| Captioner | Count | % |
|---|---:|---:|
| `Salesforce/blip-image-captioning-large` | 496,043 | 55.50% |
| `Qwen/Qwen3-VL-Embedding-2B` (+ `UPShf/Vocabulary-Qwen3-VL-Embedding-2B`) | 376,544 | 42.13% |
| `Qwen/Qwen3.5-2B` (Sigma-Captioner / SGLANG) | 21,260 | 2.38% |

license: Apache-2.0
pretty_name: UPShf/FlowTalk（规划中）V1.1 - ImageNet-1k 标题（仅标题版）
tags:
- 图像-标题生成（image-captioning）
- ImageNet
- Qwen
task_categories:
- 图像到文本（image-to-text）
language:
- 英语
size_categories:
- 100K<n<1M
---


# UPShf/FlowTalk（规划中）V1.1 - ImageNet-1k 标题（仅标题版）

本仓库为**仅标题版**发布，用于 *UPShf/FlowTalk（规划中）V1.1* 的模型训练。

它**不包含**任何ImageNet图像（公开再分发通常不符合许可要求）。用户需将这些标题与**自有本地存储的**ImageNet-1k图像进行关联（例如可使用`benjamin-paine/imagenet-1k-256x256`提供的256×256版本图像）。

基础图像源：`benjamin-paine/imagenet-1k-256x256`。

标题生成流水线：Sigma-Captioner（SGLANG分支）：https://github.com/uninterruptedpowersupply3-NEW/Sigma-Captioner/tree/SGLANG

## 数据集内容说明

本仓库包含两种**仅标题版**JSONL格式（均已分块，也可提供采用“最佳”压缩格式的WinRar打包文件）：

### A) 路径映射式标题（推荐）

文件：
- `mapped-*.jsonl`（以ImageNet parquet的`image.path`作为键）

每行JSONL格式示例如下：

json
{"imagenet_path":"...","split":"train|validation|test","label":123,"global_idx":456789,"caption":"...","caption_source":"..."}

说明：
- `imagenet_path` 对应`benjamin-paine/imagenet-1k-256x256`中parquet文件的`image.path`列的稳定标识符。
- 部分行已被有意移除（经低质量过滤/评估剔除），因此与完整ImageNet-1k数据集相比，本数据集存在部分缺失。

### B) pHash标题（支持重命名/轻微重编码场景）

文件：
- `captions-*.jsonl`（出于历史原因，以`sha256`字段存储的感知哈希作为键）

每行JSONL格式示例如下：

json
{"sha256":"d8fa381417597327","hash_mode":"perceptual_hash_phash","caption":"...","caption_source":"...","image_filename":"..."}

说明：
- `hash_mode=perceptual_hash_phash` 表示`sha256`字段**并非真实SHA-256哈希**，而是16位十六进制的pHash字符串。

## 当前版本构建统计

- 已导出/映射的标题数：**891,549**
- 无有效标题（已过滤移除）的JSON文件数：**2,298**
- 已扫描的完整ImageNet-1k数据集行（parquet文件）数：**1,431,167**

## 将标题与本地图像关联（顺序无要求）

若你的图像提取顺序与默认不同（例如“图像12对应原图像31”），仍可通过文件名完成关联：
- 找到与`imagenet_path`匹配的本地文件。
- 使用`split`字段判断该图像属于训练集、验证集还是测试集。

若你的本地副本已重命名文件，可使用pHash格式的JSONL：
- 对本地图像计算相同的pHash（`imagehash.phash`）
- 匹配`sha256`字段中的16位十六进制字符串

警告：
- pHash设计用于相似度匹配，而非密码学唯一性验证。若需更高安全性，可执行二级验证（例如匹配候选结果的字节级SHA-256哈希）。

## 本地复现方法（Windows系统）

在本工作区根目录执行以下命令：
- 同时使用两种标题源（`CaptinedIMGNET`与`extracted_images`）
- 仅读取parquet元数据（`image.path`、`label`）与标题JSON文件
- 无需加载图像字节数据

powershell
python .codexGPT5.2HIGH.py build-mapped-index `
--parquet_dir .imagenet-1k-256x256data `
--out_dir .flowtalk_mapped_shards `
--out_zip .flowtalk_v1.1_imagenet1k_captions_mapped_store.zip `
--batch_size 50000 --queue 50000 --shard_size 50000 --workers 8

若需显式指定（不自动检测），需添加参数：
- `--pairs_dir .imagenet-1k-256x256CaptinedIMGNET`
- `--pairs_dir .imagenet-1k-256x256extracted_images`

生成pHash格式JSONL的命令如下：

powershell
python .codexGPT5.2HIGH.py export-captions `
--caption_dir .imagenet-1k-256x256CaptinedIMGNET `
--caption_dir .imagenet-1k-256x256extracted_images `
--out_dir .flowtalk_phash_shards `
--hash_mode phash `
--shard_size 50000 --queue 5000 --workers 8 `
--skip_missing_images

### 性能说明

- `--pairs_lookup ram`（默认选项）会扫描标题目录一次，并通过O(1)的内存查找替代数百万次逐行`exists()`调用。
- 可选加速（更快的JSON处理）：`pip install orjson`（脚本会在检测到该库时自动启用）。
- 可选添加进度条：`pip install tqdm`。
- 可选为`export-captions --hash_mode phash`添加感知哈希支持：`pip install ImageHash`（导入时以`imagehash`为名）。

## 许可证与数据集使用规则

*免责声明：本部分仅为提供信息参考，不构成法律建议。*

- **标题与元数据**：本仓库生成的文本标题与JSON元数据采用**Apache-2.0**许可证发布。
- **底层图像**：本仓库未分发任何原始ImageNet图像。用户需自行获取图像，并遵守ImageNet官方使用条款，以及上游数据集（如`benjamin-paine/imagenet-1k-256x256`）的相关许可要求。
- **版权**：本数据集仅提供衍生文本描述。用户需确保其对文本与图像组合数据的使用符合所有适用许可条款。

## 已知局限性与问题

- **语言与词汇约束**：本数据集旨在完全使用英语。但由于标题与标签由自动化AI模型生成，需注意以下边缘情况：
- **幻觉输出**：由于视觉语言模型的幻觉现象，可能会出现少量非英语字符或词汇。
- **外来词与实体**：基于标签的标题可能包含广泛使用的外来词（例如“taco”“sushi”）、专有名词或领域特定术语，部分严格的语言过滤器可能将其判定为非英语内容。

若你正在训练严格的纯英语模型，可在训练前对文本应用基础词汇过滤器，以过滤此类边缘情况。

## 标题生成模型致谢

标题通过多模型流水线生成，包括：
- BLIP标题：`Salesforce/blip-image-captioning-large`（在JSON中存储为`blip.caption`）
- 标签式标题（此前在JSON中标记为`wd_tagger`）：`Qwen/Qwen3-VL-Embedding-2B`，结合`UPShf/Vocabulary-Qwen3-VL-Embedding-2B`使用（在JSON中存储为`wd_tagger.caption`）
- Sigma-Captioner（SGLANG）问答式标题：`Qwen/Qwen3.5-2B`（在原始JSON中以`sglang.qa_pairs`存储）

标题生成器使用比例（基于**893,847**个标题JSON文件的元数据扫描；选择优先级为`sglang > blip > wd_tagger`）：

| 标题生成器 | 数量 | 占比 |
|---|---:|---:|
| `Salesforce/blip-image-captioning-large` | 496,043 | 55.50% |
| `Qwen/Qwen3-VL-Embedding-2B`（结合`UPShf/Vocabulary-Qwen3-VL-Embedding-2B`） | 376,544 | 42.13% |
| `Qwen/Qwen3.5-2B`（Sigma-Captioner / SGLANG） | 21,260 | 2.38% |