Sudoku-Bench

<h1 align="center">
<b>Sudoku-Bench</b><br>
</h1>

<p align="center">
📝 <a href="https://pub.sakana.ai/sudoku">[Leaderboard]</a>
📝 <a href="https://arxiv.org/abs/2505.16135">[Technical Report]</a>
📝 <a href="https://sakana.ai/sudoku-bench">[Blog Post]</a><br>
🤗 <a href="https://huggingface.co/datasets/SakanaAI/Sudoku-Bench">[Sudoku-Bench puzzle dataset]</a>
🤗 <a href="https://huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning">[Sudoku-CTC-Reasoning dataset]</a>
</p>

## Sudoku-Bench puzzle dataset

The `SakanaAI/Sudoku-Bench` puzzle dataset contains three subsets:

- `challenge_100`: A collection of 100 creative Sudoku puzzles.
- `test` split: 100 puzzles
- `nikoli_100`: A collection of 100 beautiful handmade standard Sudoku puzzles designed by Nikoli.
- `test` split: 100 puzzles
- `ctc`: A larger collection of puzzles featured as puzzles solves in the [Cracking the Cryptic](https://www.youtube.com/c/CrackingTheCryptic) (CTC) YouTube channel.
- `test` split: 2565 puzzles

## Subset details

### `challenge_100` subset

The purpose of the `challenge_100` subset is to evaluate the reasoning capabilities of LLMs on a diverse set of Sudokus.

The subset includes
- 15 4×4 puzzles (Sudoku variants)
- 15 6×6 puzzles (Sudoku variants)
- 50 9×9 puzzles (Sudoku variants)
- 20 9×9 puzzles (standard Sudoku) taken from the `nikoli_100` set

The selection of puzzles covers a range of difficulty. The 9×9 puzzles are roughly evenly distributed across difficulty levels 1 through 5 (using the [Logic Masters](https://logic-masters.de/Raetselportal/) difficulty scale). Around 5 puzzles are more difficult than the standard 5-star difficulty and are considered a challenge to the best human solvers. Difficulty is not a reflection of how complex the puzzle appears, and is not necessarily related to the length of the ruleset. Difficulty is a measure of how much skill and time is required for a human solver and is more closely a reflection of the depth of the idea required to find the puzzle's break-in.

The 4×4 puzzles are significantly easier and most are rated 1-star difficulty. A subset of the 4×4 puzzles are quite simple and predominately test the model's ability to understand the constraints of Sudoku variant.

Taken as a whole, the `challenge_100` includes a broad spectrum of difficulty and can be used to evaluate the performance of reasoning models of varying capabilities.

### `nikoli_100` subset

The `nikoli_100` subset contains 100 beautiful handmade standard Sudoku puzzles designed by Nikoli, the Japanese puzzle company that popularized Sudoku.

[Algorithimcally generated Sudoku puzzles](https://www.kaggle.com/datasets/rohanrao/sudoku) tend to find only puzzles of a certain type, namely whose solution path is similarly algorithmic. Human setters are more capable of creating puzzles that require deeper reasoning and creativity in the solve: see [this video](https://www.youtube.com/watch?v=mlLq8qaTLBo), for an example.

### `ctc` subset

The `ctc` subset contains 2565 puzzles featured as puzzles solves in the Cracking the Cryptic channel. The `ctc` subset can be used in conjunction with the reasoning traces in [huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning](https://huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning). That is, you may wish to use the reasoning traces together with prompts derived from the content of the puzzle being solved, which the `ctc` subset can provide.

## Puzzle details

Each puzzle in `SakanaAI/Sudoku-Bench` contains the fields:

#### Puzzle data
- `puzzle_id`: Identifier for the puzzle
- `sudokupad_url`: Link to play the puzzle on [Sudokupad](https://sudokupad.app)
- `author`: Creator of the puzzle
- `title`: Name of the puzzle
- `rules`: The puzzle rules
- `initial_board`: String representation of the starting grid (empty cells shown as '.')
- `solution`: String representation of the completed grid (81 digits for a 9×9 puzzle)
- `rows`: Number of rows in the puzzle
- `cols`: Number of columns in the puzzle
- `visual_elements`: JSON-encoded string containing detailed specifications for visual components like circles, lines, and other custom markings specific to the puzzle variant (see [Sudoku-Bench/src/sudokupad_interaction/puzzle_tools](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/puzzle_tools.py) for the extraction of the visual elements)
- `encoded_puzzle`: A compressed representation of the puzzle using SudokuPad's encoding scheme; for loading the puzzle directly in an offline SudokuPad (see [Sudoku-Bench/src/sudokupad_interaction/README.md](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/README.md))

The puzzles from the `ctc` subset contain additional fields:

#### Video metadata
- `youtube_id`: The YouTube ID of the video from which the puzzle was solved
- `sequential_number`: The index of the puzzle in the video (for videos where multiple puzzles are solved; in most cases this is 1)
- `date`: The upload date of the video
- `lgc_timestamp`: The time in seconds when the phrase "let's get cracking" is said indicating the start of the solve in the video

## Example puzzle: Parity Fish

<img width="229" alt="Image" src="https://github.com/user-attachments/assets/48820b54-78bf-47af-ad04-f64ad7a0dd13" style="float: right;"/>

The puzzle Parity Fish by Marty Sears is included in the `challenge_100` dataset.

- `puzzle_id`: `'sxsm_MartySears_580c6fdbbba9bfb0e71ae19044f02d4c'` (using SudokuPad's internal `id` field)
- `sudokupad_url`: `'https://sudokupad.app/wsj7iunsg6'` (link to the puzzle on SudokuPad)
- `author`: `'Marty Sears'`
- `title`: `'Parity Fish'`
- `rules`: `'Normal sudoku rules apply; fill the grid with the digits 1-9 so that digits don\'t repeat in any row, column, and marked 3x3 box.\\nTwo cells adjacent along a red line must contain one even digit and one odd digit.\\nTwo cells connected by a white dot contain consecutive digits.\\nTwo cells connected by a black dot contain digits where one is double the other.',`
- `initial_board`: `'.................................................................................'` (empty cells are represented as `.`)
- `solution`: `'854369172976251834123478956419582367568937421237146598785694213691823745342715689'`
- `rows`: `9`
- `cols`: `9`

### Visual elements

The `visual_elements` field is a JSON-encoded string containing detailed specifications for visual components of the puzzle. In the Parity Fish puzzle, there are 24 visual elements: 5 black dots, 16 white dots, and 3 red lines. You can display the visual elements using the `pretty_print_visual_elements` function in [`src/eval.utils`](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/eval/utils.py) in the [SakanaAI/Sudoku-Bench](https://github.com/SakanaAI/Sudoku-Bench) repo

```python
import datasets
import json
from eval.utils import pretty_print_visual_elements

puzzle = datasets.load_dataset("SakanaAI/Sudoku-Bench", "challenge_100")['test'][23] # Parity Fish puzzle
print(pretty_print_visual_elements(json.loads(puzzle['visual_elements'])))
# - shape: circle, color: white (stroke color: black), location: between r4c8 and r4c9
# - shape: circle, color: white (stroke color: black), location: between r5c8 and r5c9
# - shape: circle, color: white (stroke color: black), location: between r6c8 and r6c9
# - shape: circle, color: white (stroke color: black), location: between r5c1 and r5c2
# - shape: circle, color: white (stroke color: black), location: between r8c3 and r9c3
# - shape: circle, color: white (stroke color: black), location: between r7c1 and r8c1
# - shape: circle, color: white (stroke color: black), location: between r1c1 and r2c1
# - shape: circle, color: white (stroke color: black), location: between r7c7 and r7c8
# - shape: circle, color: white (stroke color: black), location: between r7c1 and r7c2
# - shape: circle, color: white (stroke color: black), location: between r9c8 and r9c9
# - shape: circle, color: white (stroke color: black), location: between r8c5 and r8c6
# - shape: circle, color: white (stroke color: black), location: between r1c4 and r2c4
# - shape: circle, color: white (stroke color: black), location: between r7c6 and r8c6
# - shape: circle, color: white (stroke color: black), location: between r2c7 and r3c7
# - shape: circle, color: white (stroke color: black), location: between r1c2 and r1c3
# - shape: circle, color: white (stroke color: black), location: between r1c5 and r2c5
# - shape: circle, color: black, location: between r3c2 and r4c2
# - shape: circle, color: black, location: between r4c7 and r4c8
# - shape: circle, color: black, location: between r2c3 and r3c3
# - shape: circle, color: black, location: between r9c2 and r9c3
# - shape: circle, color: black, location: between r8c8 and r9c8
# - line, color: red, coords: r3c2, r3c3, r3c4, r3c5, r3c6, r4c7, r5c8, r6c7, r7c6, r7c5, r7c4, r7c3, r7c2
# - line, color: red, coords: r4c1, r4c2, r5c3, r6c4, r7c4
# - line, color: red, coords: r6c1, r6c2, r5c3, r4c4, r3c5
```

The intermediate `json.loads(puzzle['visual_elements']))` is a list of dictionaries, each of which is a verbose description extracted from the SudokuPad rendering engine. We encourage the user to adopt their own `pretty_print_visual_elements` function to display the visual elements in a way that is most useful for their application.

Please see [`src/sudokupad_interaction/puzzle_tools`](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/puzzle_tools.py) for more details on the `visual_elements` field.

### Encoded puzzle

The `encoded_puzzle` field is a byte64 encoding of the puzzle using SudokuPad's internal encoding method.

The `encoded_puzzle` field can be used to obtain an alternate URL for the puzzle. Namely, `https://sudokupad.app/wsj7iunsg6` and [`https://sudokupad.app/{parity fish's encoded_puzzle string}`](https://sudokupad.app/sclN4SwJgXA5AzgHjAtgfQLIEMBOAXAngZQFMsZkBWADgAYBjANgDMwAjV9ATmYeasIHYAjOkID2VACziGVAExhxNKABpEhbOjDp1EYDAD2AV0w1C0fAbB6A1gYAEGK4Uy2AbjNlkAdLM8zxAWhoaBjJ0CjpldANsAAs9TGgMHFxbIhJlTAMAG0IYaAA5eMR0LNsYC2s7TJyYW3QAB3qs3ABuWwYQLNLYwlsAc0xwWwB3EFjbHtswED6x2oF/djK9CZitKZm5qb0AOwAdKGxbTEJ64iOQHbqdlMw9YaVbGj0sg0Qdx/QdsFtizEcfgBmOCA2zMPRwTx7HYAFWGKxMXVqGgAVugTDsjiVdn06sdCD8spdeogDDAjs9MehLrZdr1CC5CFdprMsd9aTtenowD8WWMobD4U9CEinrtOTRsASwSl0CMYmNepYKbt1DTKTBCDRoiBGRtWTABXCESKsrVKRKpT9mLKwVl0VZtiqqTS+dhasMYk4uZzbCBapYDMwcqsuT1MJ5lPpXtgQLtoBQyOJAXR2AI+DJ2Hw6DIyAIKIDxAIZIW+BR2GQ6EWKxQS3Q+JXy4C+OIZMXmwJxHQyOwKGXK+xWwIU2na83xGRCzJBI32MpmLNin1TFAYthsPU8gB6LcgTwgRB9IyeZ6ILc7QEgADy2HE+U8KLOfXni/Qy709XRY1wEAESmw0AAApYN+tgAGL+jEUAAL5KCYEAANoIUoKGoShAC6SjIWhqGYdhOFKHhBEYVhxGEaRxFEZRFEEVRtGYSciFIVQShUHhLECOxSgyHhf5sVhf6cQJ3F4TIrGiUoQkIWJPF4QhLGAlx4hcWQvFKIpwnKcJqlYWJGnSUoWkGapcksXQXF8FxFBqeZwmWcJ1m6UotkGfZBnWXJgLiVhXlSV5PFYeI3kIUFUlBQFCFkMFUVSVFslYQhXn6V5RleTpIXqXhQVGUF6VRfpUVGVFJkJV5LleW5XmORlLlBW5QXVVFLlRW5UUeQldDBZ1UmdRFfDBf1Un9RFFDBaNUmjfFSGdfpnVGZ16X9fp/VGf16Wjfpo1GaNJXTc5eGdW5nXVf1Ln9W5/XVaNLmjW5o0eZheiIcAkqIYCnhtZhwwQFQniFkoMQ/X9QWxL+vijVK0DKDQEAAMQsQk2BwTIcNgXBAhw1QsEvQBIUfUojnfb9/2A8TIOAwI4P/iu0OY0oiPI6j6OY9jr2RfjhNAyTXPk2DMgQzTcF0wzNAo7DaM0Bj8Os7j4j40JRPAwDPP/hTVOQ1AtPw/TEBI6LTOSyzSg44hk0fV9Kuk0roOU/z1NQ0L2si2LEtS1jxts/1v06Yr3Nk6rfMCw7MNO7rjPi8z0se7jfHm0ovtBVb/02+rgshwjYf6xHhtRybCF0Pj9kJ8r/sp3bGtaxnesu5H7t5wXMUW/7Se87bQea47Vfh67Rt5xQHNN0rLcB239sd+nOvVwbbsy4hfD4z7lsq2X7eV5P3e17PCF/u9i/N8vavl2nwuZzXOd157SheHvQ8H4HY9r870+92zYkN4PfvW4fq+d+vWc97nNm3sRLxyXqXb+D9f5P2zjPaOiE/xyxvp/ZOECK5QNPs/QBuNfJx19l5JOXk0EhzRtAi+2DC4f3wVzQhx8I6kK3jJXB1CS5/RocHJm9C4EIX7nFShLDARsPHhwjBWDTZKHnkgqhxNBFaxISIrG6FoJAA) will load the same puzzle. Both URLs point to Sven's SudokuPad website. However, only the second method works when running SudokuPad locally and avoids a call to the SudokuPad puzzle database. To ensure longevity of the benchmark, we provide a local usage in [`src/sudokupad_interaction`](https://github.com/SakanaAI/Sudoku-Bench/tree/main/src/sudokupad_interaction).

The `encoded_puzzle` field can be ignored if using the text-only approach outlined in `src.eval` in this repo as all relevant information has already been extracted.

## Puzzle edge cases

Because of the wide array of puzzles solved, the `ctc` subset is provided "as-is". There are a number of edge cases that make a pure text representation of the puzzle incomplete:
1. Some puzzles have visual elements that are difficult to encode in the `visual_elements` field (see below for a description of the `visual_elements` field). For example, the delightful [RatRun puzzles](https://www.youtube.com/watch?v=-KXjRMkYpA4) will not have a coherent textual description of the visual elements due to the visual complexity of the puzzle.
2. Other puzzles have the `solution` field omitted as many puzzle setters choose not to disclose the solution in SudokuPad.
3. A popular recent trend is the use of fog-of-war in Sudoku puzzles. For such puzzles, all hidden elements will be exposed in the `visual_elements` field meaning the puzzle will not be presented as intended by the puzzle setter.

Please consider filtering the `ctc` subset based on your needs.

## Citation
```bibtex
@misc{seely2025sudoku-bench,
title={{Sudoku-Bench}},
author={Seely, Jeffrey and Imajuku, Yuki and Zhao, Tianyu and Cetin, Edoardo and Jones, Llion},
howpublished = {\url{https://github.com/SakanaAI/Sudoku-Bench}},
year={2025}
}
```

<h1 align="center">
<b>Sudoku-Bench 数独基准数据集</b><br>
</h1>

<p align="center">
📝 <a href="https://pub.sakana.ai/sudoku">[排行榜]</a>
📝 <a href="https://arxiv.org/abs/2505.16135">[技术报告]</a>
📝 <a href="https://sakana.ai/sudoku-bench">[博客文章]</a><br>
🤗 <a href="https://huggingface.co/datasets/SakanaAI/Sudoku-Bench">[数独基准谜题数据集]</a>
🤗 <a href="https://huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning">[数独-CTC-推理数据集]</a>
</p>

## 数独基准谜题数据集

`SakanaAI/Sudoku-Bench` 谜题数据集包含三个子集：
- `challenge_100`：包含100款原创数独谜题的集合。
- `test` 划分：100款谜题
- `nikoli_100`：由日本谜题公司Nikoli设计的100款精美手工标准数独谜题集合。
- `test` 划分：100款谜题
- `ctc`：收录自[Cracking the Cryptic（简称CTC）](https://www.youtube.com/c/CrackingTheCryptic) YouTube频道解谜环节的大型谜题集合，共2565款谜题。
- `test` 划分：2565款谜题

## 子集详情

### `challenge_100` 子集
`challenge_100` 子集旨在评估大语言模型（Large Language Model，简称LLM）在多样化数独谜题上的推理能力。
该子集包含：
- 15款4×4数独变体谜题
- 15款6×6数独变体谜题
- 50款9×9数独变体谜题
- 20款9×9标准数独谜题，取自`nikoli_100`数据集

谜题的选择覆盖了广泛的难度区间。9×9谜题的难度大致均匀分布在1至5级（采用[Logic Masters](https://logic-masters.de/Raetselportal/)难度量表），其中约5款谜题难度超过标准5星级别，对顶尖人类解谜者而言仍具挑战性。此处的难度并非指谜题外观的复杂程度，也不一定与规则集的长度相关，而是衡量人类解谜者所需投入的技巧与时间，更准确地反映了找到谜题破局点所需的思路深度。
4×4谜题整体难度较低，多数被评为1星难度，其中部分谜题极为简单，主要用于测试模型对数独约束规则的理解能力。
整体而言，`challenge_100` 涵盖了广泛的难度区间，可用于评估不同性能层级的推理模型的表现。

### `nikoli_100` 子集
`nikoli_100` 子集包含100款由日本谜题公司Nikoli设计的精美手工标准数独谜题，Nikoli是推广数独运动的知名厂商。
[算法生成的数独谜题](https://www.kaggle.com/datasets/rohanrao/sudoku)通常仅能生成特定类型的谜题，即求解路径类似算法式的谜题。而人类出题者更擅长创作需要更深层次推理与创造力的解谜谜题，详见[该视频](https://www.youtube.com/watch?v=mlLq8qaTLBo)中的示例。

### `ctc` 子集
`ctc` 子集包含2565款曾在CTC YouTube频道中被解谜的谜题。该子集可与[huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning](https://huggingface.co/datasets/SakanaAI/Sudoku-CTC-Reasoning)中的推理轨迹结合使用，即可以将推理轨迹与基于待解谜谜题内容衍生的提示结合使用，而`ctc`子集可提供此类谜题的原始内容。

## 谜题详情

`SakanaAI/Sudoku-Bench` 中的每款谜题均包含以下字段：

#### 谜题数据
- `puzzle_id`：谜题的唯一标识符
- `sudokupad_url`：可在[Sudokupad](https://sudokupad.app)上游玩该谜题的链接
- `author`：谜题创作者
- `title`：谜题名称
- `rules`：谜题规则说明
- `initial_board`：起始网格的字符串表示（空单元格以`.`表示）
- `solution`：完成网格的字符串表示（9×9谜题为81位数字）
- `rows`：谜题的行数
- `cols`：谜题的列数
- `visual_elements`：JSON编码的字符串，包含谜题变体特有的视觉组件（如圆圈、线条及其他自定义标记）的详细规格（详见[SakanaAI/Sudoku-Bench仓库](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/puzzle_tools.py)中的`puzzle_tools.py`文件，其中包含视觉元素的提取代码）
- `encoded_puzzle`：采用SudokuPad编码方案对谜题进行的压缩表示，可用于在离线SudokuPad中直接加载谜题（详见[SakanaAI/Sudoku-Bench仓库](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/README.md)）

`ctc` 子集的谜题还包含以下额外的视频元数据字段：

#### 视频元数据
- `youtube_id`：该谜题所在解谜视频的YouTube ID
- `sequential_number`：视频中该谜题的索引（对于同时包含多个解谜谜题的视频，多数情况下该值为1）
- `date`：视频上传日期
- `lgc_timestamp`：视频中出现“let's get cracking”字样的时间点，该语句标志着解谜环节的正式开始

## 示例谜题：Parity Fish 奇偶鱼

<img width="229" alt="Image" src="https://github.com/user-attachments/assets/48820b54-78bf-47af-ad04-f64ad7a0dd13" style="float: right;"/>

由Marty Sears创作的奇偶鱼（Parity Fish）谜题已收录至`challenge_100`数据集中。
- `puzzle_id`：`'sxsm_MartySears_580c6fdbbba9bfb0e71ae19044f02d4c'`（采用SudokuPad内部`id`字段）
- `sudokupad_url`：`'https://sudokupad.app/wsj7iunsg6'`（该谜题在Sudokupad上的链接）
- `author`：`'Marty Sears'`
- `title`：`'Parity Fish'`
- `rules`：`'标准数独规则适用：将数字1-9填入网格，确保每行、每列及每个3×3粗线框内数字不重复。
沿红线相邻的两个单元格必须分别包含一个奇数和一个偶数。
由白点连接的两个单元格中的数字必须连续。
由黑点连接的两个单元格中的数字必须满足其中一个是另一个的两倍。',`
- `initial_board`：`'.................................................................................'`（空单元格以`.`表示）
- `solution`：`'854369172976251834123478956419582367568937421237146598785694213691823745342715689'`
- `rows`：`9`
- `cols`：`9`

### 视觉元素
`visual_elements` 字段为JSON编码字符串，包含谜题视觉组件的详细规格。在奇偶鱼谜题中，共有24个视觉元素：5个黑点、16个白点及3条红线。可通过[SakanaAI/Sudoku-Bench仓库](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/eval/utils.py)中的`pretty_print_visual_elements`函数展示视觉元素。

python
import datasets
import json
from eval.utils import pretty_print_visual_elements

# 加载奇偶鱼谜题
puzzle = datasets.load_dataset("SakanaAI/Sudoku-Bench", "challenge_100")['test'][23]
# 解析并打印视觉元素
print(pretty_print_visual_elements(json.loads(puzzle['visual_elements'])))
# 输出示例：
# - 形状：圆形，颜色：白色（描边颜色：黑色），位置：r4c8与r4c9之间
# - 形状：圆形，颜色：白色（描边颜色：黑色），位置：r5c8与r5c9之间
# ...（其余输出内容与原文一致）

注：`json.loads(puzzle['visual_elements'])` 的结果为字典列表，每个字典均为从SudokuPad渲染引擎中提取的详细描述。我们鼓励用户根据自身应用场景自定义`pretty_print_visual_elements`函数以展示视觉元素。
更多关于`visual_elements`字段的细节，请参阅[SakanaAI/Sudoku-Bench仓库](https://github.com/SakanaAI/Sudoku-Bench/blob/main/src/sudokupad_interaction/puzzle_tools.py)中的`puzzle_tools.py`文件。

### 编码谜题
`encoded_puzzle` 字段为采用SudokuPad内部编码方法得到的Base64编码的谜题表示。
该字段可用于获取谜题的备用链接，例如`https://sudokupad.app/wsj7iunsg6`与`https://sudokupad.app/{奇偶鱼的encoded_puzzle字符串}`均可加载同一谜题，两者均指向Sven开发的SudokuPad网站。但仅第二种方法可在本地运行SudokuPad时使用，且无需调用SudokuPad谜题数据库。为确保本基准数据集的长期可用性，我们在[SakanaAI/Sudoku-Bench仓库](https://github.com/SakanaAI/Sudoku-Bench/tree/main/src/sudokupad_interaction)中提供了本地使用方案。
若使用该仓库中`src.eval`章节介绍的纯文本分析方法，则可忽略`encoded_puzzle`字段，因为所有关键信息均已提取完毕。

## 谜题边缘案例
由于涵盖了广泛的解谜谜题，`ctc` 子集按“原样”提供。存在若干边缘情况导致纯文本表示无法完整呈现谜题：
1. 部分谜题的视觉元素难以在`visual_elements`字段中完整编码，例如[RatRun谜题](https://www.youtube.com/watch?v=-KXjRMkYpA4)，由于其视觉复杂度极高，无法通过文本描述完整还原其视觉元素。
2. 部分谜题的`solution`字段被省略，因为许多谜题创作者选择不在SudokuPad中公开谜题答案。
3. 近期数独谜题中流行使用战争迷雾（fog-of-war）机制，对于此类谜题，`visual_elements`字段会暴露所有隐藏元素，导致无法呈现谜题创作者的原始设计意图。

请根据自身研究需求对`ctc`子集进行筛选。

## 引用
bibtex
@misc{seely2025sudoku-bench,
title={{Sudoku-Bench}},
author={Seely, Jeffrey and Imajuku, Yuki and Zhao, Tianyu and Cetin, Edoardo and Jones, Llion},
howpublished = {url{https://github.com/SakanaAI/Sudoku-Bench}},
year={2025}
}

搜集汇总

背景与挑战

背景概述

Sudoku-Bench是一个用于评估大型语言模型在数独变体上推理能力的数据集，包含三个子集：challenge_100（100个创意谜题）、nikoli_100（100个手工标准谜题）和ctc（2565个来自YouTube频道的谜题）。每个谜题提供详细字段如规则、初始棋盘和解决方案，旨在测试模型在不同难度和类型谜题上的表现。