APPS

APPS 数据集包含从不同的开放访问编码网站（如 Codeforces、Kattis 等）收集的问题。 APPS 基准测试试图通过以不受限制的自然语言提出编码问题并评估解决方案的正确性来反映人类程序员的评估方式。问题的难度范围从入门级到大学竞赛级别，并衡量编码能力以及解决问题的能力。 自动化编程进度标准，缩写为 APPS，总共包含 10,000 个编码问题，有 131,836 个用于检查解决方案的测试用例和 232,444 个由人类编写的真实解决方案。问题可能很复杂，因为问题的平均长度为 293.2 个单词。数据被平均分成训练集和测试集，每个有 5,000 个问题。在测试集中，每个问题都有多个测试用例，平均测试用例数为21.2。每个测试用例都是专门针对相应问题设计的，使我们能够严格评估程序功能。The Automated Programming Progress Standard (APPS) dataset comprises coding problems collected from diverse open-access coding platforms such as Codeforces, Kattis, and others. The APPS benchmark is designed to mirror the evaluation process of human programmers by posing unconstrained natural language coding problems and evaluating the correctness of submitted solutions. The problems span a difficulty spectrum from introductory level to university-level competition, assessing both coding proficiency and problem-solving capabilities. In total, the APPS dataset contains 10,000 coding problems, accompanied by 131,836 test cases for solution validation and 232,444 human-written real-world solutions. The problems can be structurally complex, with an average length of 293.2 words per problem. The dataset is evenly partitioned into training and test subsets, each containing 5,000 problems. Within the test subset, each problem is paired with multiple test cases, averaging 21.2 test cases per problem. Each test case is custom-designed for its corresponding problem, enabling rigorous evaluation of program functionality.

搜集汇总

背景与挑战

背景概述

APPS数据集是一个包含10,000个编码问题的基准测试数据集，问题来源于Codeforces、Kattis等开放访问编码网站，难度从入门级到大学竞赛级别不等。该数据集旨在通过自然语言提出的编码问题和严格的测试用例评估程序员的编码能力和问题解决能力，包含131,836个测试用例和232,444个人类编写的解决方案，数据平均分为训练集和测试集。