mrdbourke/food-or-drink-10m

---
dataset_info:
features:
- name: url
dtype: string
- name: re_caption
dtype: string
- name: org_caption
dtype: string
- name: sha256
dtype: string
- name: key
dtype: string
- name: re_clip_score
dtype: float64
- name: org_clip_score
dtype: float64
- name: re_length
dtype: int64
- name: org_length
dtype: int64
- name: re_gpt4v_score
dtype: int64
- name: org_gpt4v_score
dtype: int64
- name: re_caption_condition_diverse_topk
dtype: string
- name: re_condition_length
dtype: int64
- name: label
dtype:
class_label:
names:
'0': food_or_drink
'1': not_food_or_drink
- name: score
dtype: float64
splits:
- name: train
num_examples: 1415619
- name: test
num_examples: 157292
configs:
- config_name: default
data_files:
- split: train
path: data/train-*.parquet
- split: test
path: data/test-*.parquet
license: apache-2.0
task_categories:
- text-classification
- zero-shot-classification
language:
- en
tags:
- food
- drink
- food-classification
- caption-classification
- modernbert
- knowledge-distillation
size_categories:
- 1M<n<10M
---

# Food or Drink 10M

A balanced binary classification dataset for detecting **food or drink** content in image captions.

## Overview

| | Count |
|---|---|
| **Total rows** | 1,572,911 |
| **Train split** | 1,415,619 (90%) |
| **Test split** | 157,292 (10%) |
| **Food/drink rows** | 785,980 (50.0%) |
| **Not food/drink rows** | 786,931 (50.0%) |

## How it was made

1. **Source**: [UCSC-VLAA/Recap-DataComp-1B](https://huggingface.co/datasets/UCSC-VLAA/Recap-DataComp-1B) (`condition_diverse_topk` subset)
2. **Teacher model**: [MoritzLaurer/ModernBERT-large-zeroshot-v2.0](https://huggingface.co/MoritzLaurer/ModernBERT-large-zeroshot-v2.0) — a 400M parameter zero-shot NLI classifier
3. **Classification**: 10M captions streamed and classified as `food_or_drink` / `not_food_or_drink` using zero-shot NLI with candidate labels `["food or drink", "not food or drink"]`
4. **Balancing**: All food/drink rows saved (100%), not-food/drink rows dynamically sampled to match — resulting in a ~50/50 balanced dataset

## Labels

- **`food_or_drink`** — caption describes food, beverages, meals, ingredients, drinks, or food/drink items
- **`not_food_or_drink`** — caption describes anything else (objects, scenes, people, animals, etc.)

## Fields

| Field | Description |
|---|---|
| `url` | Original image URL from DataComp-1B |
| `re_caption` | AI-generated re-caption (detailed, descriptive) |
| `org_caption` | Original caption (often noisy alt-text) |
| `re_caption_condition_diverse_topk` | Condition-diverse re-caption variant |
| `label` | Classification label: `food_or_drink` or `not_food_or_drink` |
| `score` | Teacher model confidence score (0.5–1.0) |
| `sha256` | Image content hash |
| `key` | Row key from source dataset |
| `re_clip_score` / `org_clip_score` | CLIP alignment scores |
| `re_length` / `org_length` | Caption token lengths |
| `re_gpt4v_score` / `org_gpt4v_score` | GPT-4V quality scores |
| `re_condition_length` | Condition caption token length |

## Intended use

This dataset is designed for:

- **Knowledge distillation**: Fine-tuning smaller encoders (e.g., [Ettin-encoder-150m](https://huggingface.co/jhu-clsp/ettin-encoder-150m)) to replicate the teacher model's classification, then running the fine-tuned model on the full 1B-row dataset
- **Food/drink content filtering**: Filtering large-scale image-text datasets for food and drink related content
- **Caption classification research**: Studying food/drink detection in image captions

## Caption types

The dataset contains multiple caption styles per image, useful for training robust classifiers:

- **`re_caption`**: Clean, AI-generated descriptions (e.g., *"A glass of red wine next to a cheese board on a wooden table"*)
- **`org_caption`**: Original noisy alt-text (e.g., *"IMG_2847 wine night"*)
- **`re_caption_condition_diverse_topk`**: Longer, more detailed AI captions

## Example samples

### Food or drink

| label | score | re_caption | org_caption |
|---|---|---|---|
| `food_or_drink` | 0.9988 | A glass of minty chocolate latte with a straw is placed on a yellow surface next to a bag of coffee beans. | Mint Chocolate Coffee Frappe Recipe |
| `food_or_drink` | 0.9978 | A plate of food with a mix of lettuce, meat, and sauce. The plate is on a dining table with a glass of orange juice and a bowl of salad. | Summer lunch spread with fresh OJ |

### Not food or drink

| label | score | re_caption | org_caption |
|---|---|---|---|
| `not_food_or_drink` | 0.9998 | A row of identical figures in black suits and ties is standing in a line against a white background. | O 6-Car Flat End Offset Hopper Car Set, B&O 727022 |
| `not_food_or_drink` | 0.9997 | A white wainscoting panel with a decorative molding at the top and a plain lower section is attached to a wall. | How to Install Board and Batten Wainscoting (White Painted Square over Rectangle Pattern) |

## Usage

```python
from datasets import load_dataset

# Load the full dataset
ds = load_dataset("mrdbourke/food-or-drink-10m")

# Access splits
train = ds["train"]
test = ds["test"]

print(f"Train: {len(train):,} rows")
print(f"Test: {len(test):,} rows")

# Look at a sample
print(train[0])

# Filter to food/drink only
food_only = train.filter(lambda x: x["label"] == 0) # 0 = food_or_drink
print(f"Food/drink rows: {len(food_only):,}")

# Stream instead of downloading
ds_stream = load_dataset("mrdbourke/food-or-drink-10m", split="train", streaming=True)
for row in ds_stream:
print(row["re_caption"], row["label"])
break
```

## Confidence scores

The `score` field contains the teacher model's confidence. Higher scores indicate more certain classifications:

| Score range | Meaning |
|---|---|
| 0.95–1.0 | Very confident |
| 0.80–0.95 | Confident |
| 0.60–0.80 | Moderate confidence |
| 0.50–0.60 | Low confidence (borderline) |

## License

Apache 2.0 — same as the source dataset and teacher model.

dataset_info:
特征:
- 名称: url
数据类型: 字符串
- 名称: re_caption
数据类型: 字符串
- 名称: org_caption
数据类型: 字符串
- 名称: sha256
数据类型: 字符串
- 名称: key
数据类型: 字符串
- 名称: re_clip_score
数据类型: float64
- 名称: org_clip_score
数据类型: float64
- 名称: re_length
数据类型: int64
- 名称: org_length
数据类型: int64
- 名称: re_gpt4v_score
数据类型: int64
- 名称: org_gpt4v_score
数据类型: int64
- 名称: re_caption_condition_diverse_topk
数据类型: 字符串
- 名称: re_condition_length
数据类型: int64
- 名称: label
数据类型:
类别标签:
标签名称:
'0': 食品或饮品
'1': 非食品或饮品
- 名称: score
数据类型: float64
数据集划分:
- 名称: 训练集
样本数量: 1415619
- 名称: 测试集
样本数量: 157292
配置项:
- 配置名称: 默认配置
数据文件:
- 划分: 训练集
路径: data/train-*.parquet
- 划分: 测试集
路径: data/test-*.parquet
许可证: Apache 2.0
任务类别:
- 文本分类
- 零样本分类
语言:
- 英语
标签:
- 食品
- 饮品
- 食品分类
- 标题分类
- ModernBERT
- 知识蒸馏
样本量区间:
- 100万 < 样本数 < 1000万

# 食品与饮品10M数据集

这是一款用于检测图像标题中食品或饮品内容的均衡二分类数据集。

## 概览
| | 样本数量 |
|---|---|
| **总样本数** | 1,572,911 |
| **训练集划分** | 1,415,619（占比90%） |
| **测试集划分** | 157,292（占比10%） |
| **食品/饮品样本** | 785,980（占比50.0%） |
| **非食品/饮品样本** | 786,931（占比50.0%） |

## 数据集构建流程
1. **数据源**：[UCSC-VLAA/Recap-DataComp-1B](https://huggingface.co/datasets/UCSC-VLAA/Recap-DataComp-1B)（`condition_diverse_topk`子集）
2. **教师模型**：[MoritzLaurer/ModernBERT-large-zeroshot-v2.0](https://huggingface.co/MoritzLaurer/ModernBERT-large-zeroshot-v2.0)——一款参数规模达4亿的零样本自然语言推理（Natural Language Inference, NLI）分类器
3. **分类流程**：对1000万条标题进行流式处理，并使用候选标签`["食品或饮品", "非食品或饮品"]`的零样本NLI分类，将其标记为`food_or_drink`或`not_food_or_drink`
4. **均衡处理**：保留全部食品/饮品样本（100%），动态采样非食品/饮品样本以匹配二者数量，最终得到约50/50的均衡数据集

## 标签说明
- **`food_or_drink`**：标题描述食品、饮品、餐食、食材、饮料或食品/饮品类物品
- **`not_food_or_drink`**：标题描述其他内容（物体、场景、人物、动物等）

## 字段说明
| 字段 | 详细说明 |
|---|---|
| `url` | 源自DataComp-1B的原始图像URL |
| `re_caption` | AI生成的重生成标题（描述性强且细节丰富） |
| `org_caption` | 原始标题（通常为带有噪声的替代文本） |
| `re_caption_condition_diverse_topk` | 条件多样化重生成标题变体 |
| `label` | 分类标签：`food_or_drink` 或 `not_food_or_drink` |
| `score` | 教师模型的分类置信度分数（取值范围0.5~1.0） |
| `sha256` | 图像内容SHA256哈希值 |
| `key` | 源数据集中的行键 |
| `re_clip_score` / `org_clip_score` | 重生成/原始标题的CLIP对齐分数 |
| `re_length` / `org_length` | 标题的Token长度 |
| `re_gpt4v_score` / `org_gpt4v_score` | GPT-4V生成的标题质量分数 |
| `re_condition_length` | 条件标题的Token长度 |

## 预期应用场景
本数据集适用于以下场景：
- **知识蒸馏**：微调小型编码器（如[Ettin-encoder-150m](https://huggingface.co/jhu-clsp/ettin-encoder-150m)）以复现教师模型的分类能力，随后在完整的10亿行数据集上运行微调后的模型
- **食品与饮品内容过滤**：对大规模图像-文本数据集进行过滤，提取与食品和饮品相关的内容
- **标题分类研究**：开展图像标题中的食品/饮品检测相关研究

## 标题类型
本数据集为每张图像提供多种标题样式，有助于训练鲁棒性更强的分类器：
- **`re_caption`**：简洁规整的AI生成描述（例如：*"木质桌面上摆放着一杯薄荷巧克力拿铁，旁侧放置一袋咖啡豆"*）
- **`org_caption`**：原始的带噪声替代文本（例如：*"IMG_2847 葡萄酒之夜"*）
- **`re_caption_condition_diverse_topk`**：更长、更详尽的AI生成标题

## 示例样本
### 食品与饮品样本
| 标签 | 置信分数 | 重生成标题 | 原始标题 |
|---|---|---|---|
| `food_or_drink` | 0.9988 | 一杯插有吸管的薄荷巧克力拿铁放置在黄色台面上，旁侧摆放着一袋咖啡豆。 | 薄荷巧克力咖啡冰沙食谱 |
| `food_or_drink` | 0.9978 | 一盘混合了生菜、肉类与酱汁的食物，放置在餐桌上，旁侧有一杯橙汁与一碗沙拉。 | 夏日午餐盛宴搭配鲜榨橙汁 |

### 非食品与饮品样本
| 标签 | 置信分数 | 重生成标题 | 原始标题 |
|---|---|---|---|
| `not_food_or_drink` | 0.9998 | 一排身着黑色西装、系着领带的同款人偶，整齐排列在纯白色背景前。 | 6节平端偏置漏斗车套装，B&O 727022 |
| `not_food_or_drink` | 0.9997 | 一面安装在墙面上的白色护墙板，顶部带有装饰性线条，下部为简洁的平面设计。 | 如何安装板条式护墙板（白色涂装的方形叠矩形图案） |

## 使用示例
python
from datasets import load_dataset

# 加载完整数据集
ds = load_dataset("mrdbourke/food-or-drink-10m")

# 访问数据集划分
train = ds["train"]
test = ds["test"]

print(f"训练集样本数: {len(train):,}")
print(f"测试集样本数: {len(test):,}")

# 查看单条样本
print(train[0])

# 仅筛选出食品与饮品样本
food_only = train.filter(lambda x: x["label"] == 0) # 0 代表 food_or_drink
print(f"食品与饮品样本数: {len(food_only):,}")

# 流式加载（无需提前下载完整数据集）
ds_stream = load_dataset("mrdbourke/food-or-drink-10m", split="train", streaming=True)
for row in ds_stream:
print(row["re_caption"], row["label"])
break

## 置信分数说明
`score`字段存储了教师模型的分类置信度，分数越高代表分类结果越确定：
| 分数区间 | 含义 |
|---|---|
| 0.95~1.0 | 置信度极高 |
| 0.80~0.95 | 置信度较高 |
| 0.60~0.80 | 置信度中等 |
| 0.50~0.60 | 置信度较低（边界样本） |

## 许可证
采用Apache 2.0许可证，与源数据集及教师模型的许可证保持一致。