ENSEONG/stratified-1k-math-private-n256-Qwen2.5-3B-Instruct-bon

--- dataset_info: - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 441775100 num_examples: 1000 download_size: 71965406 dataset_size: 441775100 - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 442573395 num_examples: 1000 download_size: 72109233 dataset_size: 442573395 - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 441974717 num_examples: 1000 download_size: 71852188 dataset_size: 441974717 - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 442822240 num_examples: 1000 download_size: 72030901 dataset_size: 442822240 - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 442478016 num_examples: 1000 download_size: 72110546 dataset_size: 442478016 - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 441993184 num_examples: 1000 download_size: 71919174 dataset_size: 441993184 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 436329337 num_examples: 1000 download_size: 86987957 dataset_size: 436329337 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 436524949 num_examples: 1000 download_size: 87016847 dataset_size: 436524949 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 436161069 num_examples: 1000 download_size: 86934973 dataset_size: 436161069 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 435799584 num_examples: 1000 download_size: 86992413 dataset_size: 435799584 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 435556612 num_examples: 1000 download_size: 86798243 dataset_size: 435556612 - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 435975516 num_examples: 1000 download_size: 86847843 dataset_size: 435975516 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 429760007 num_examples: 1000 download_size: 95247607 dataset_size: 429760007 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 429974986 num_examples: 1000 download_size: 95269244 dataset_size: 429974986 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 429905944 num_examples: 1000 download_size: 95212677 dataset_size: 429905944 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 429419634 num_examples: 1000 download_size: 95171510 dataset_size: 429419634 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 430586976 num_examples: 1000 download_size: 95296555 dataset_size: 430586976 - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 429814470 num_examples: 1000 download_size: 95540151 dataset_size: 429814470 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 423717908 num_examples: 1000 download_size: 101060635 dataset_size: 423717908 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 423415863 num_examples: 1000 download_size: 101124224 dataset_size: 423415863 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 424020131 num_examples: 1000 download_size: 101253699 dataset_size: 424020131 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 424227104 num_examples: 1000 download_size: 101269853 dataset_size: 424227104 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 423970936 num_examples: 1000 download_size: 101217641 dataset_size: 423970936 - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 423184013 num_examples: 1000 download_size: 101164183 dataset_size: 423184013 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 417943339 num_examples: 1000 download_size: 106132943 dataset_size: 417943339 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 417561493 num_examples: 1000 download_size: 106187016 dataset_size: 417561493 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 417985446 num_examples: 1000 download_size: 106206299 dataset_size: 417985446 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 416799917 num_examples: 1000 download_size: 105898306 dataset_size: 416799917 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 417907433 num_examples: 1000 download_size: 106144618 dataset_size: 417907433 - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 417240600 num_examples: 1000 download_size: 106018991 dataset_size: 417240600 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 412099395 num_examples: 1000 download_size: 110492336 dataset_size: 412099395 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 411401432 num_examples: 1000 download_size: 110009092 dataset_size: 411401432 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 411282163 num_examples: 1000 download_size: 110072797 dataset_size: 411282163 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 411495498 num_examples: 1000 download_size: 110560178 dataset_size: 411495498 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 411232728 num_examples: 1000 download_size: 110086431 dataset_size: 411232728 - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 411708896 num_examples: 1000 download_size: 110154301 dataset_size: 411708896 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 406821183 num_examples: 1000 download_size: 114841230 dataset_size: 406821183 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 405807978 num_examples: 1000 download_size: 114504986 dataset_size: 405807978 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 406542601 num_examples: 1000 download_size: 114467937 dataset_size: 406542601 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 406504741 num_examples: 1000 download_size: 114424757 dataset_size: 406504741 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 406327194 num_examples: 1000 download_size: 114213920 dataset_size: 406327194 - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 405919551 num_examples: 1000 download_size: 114562390 dataset_size: 405919551 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 401856396 num_examples: 1000 download_size: 118524989 dataset_size: 401856396 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 402192138 num_examples: 1000 download_size: 118916715 dataset_size: 402192138 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 402171353 num_examples: 1000 download_size: 119342494 dataset_size: 402171353 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 401874776 num_examples: 1000 download_size: 119027899 dataset_size: 401874776 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 401766561 num_examples: 1000 download_size: 118793498 dataset_size: 401766561 - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 401664059 num_examples: 1000 download_size: 119655348 dataset_size: 401664059 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 398802979 num_examples: 1000 download_size: 124702049 dataset_size: 398802979 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 399018244 num_examples: 1000 download_size: 125009633 dataset_size: 399018244 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 399172243 num_examples: 1000 download_size: 124227870 dataset_size: 399172243 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 398931347 num_examples: 1000 download_size: 125074266 dataset_size: 398931347 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 398600629 num_examples: 1000 download_size: 124483748 dataset_size: 398600629 - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 399104413 num_examples: 1000 download_size: 124446229 dataset_size: 399104413 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396578405 num_examples: 1000 download_size: 132040374 dataset_size: 396578405 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396799059 num_examples: 1000 download_size: 132062503 dataset_size: 396799059 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396730413 num_examples: 1000 download_size: 132286346 dataset_size: 396730413 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396536542 num_examples: 1000 download_size: 132170815 dataset_size: 396536542 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396598099 num_examples: 1000 download_size: 131740668 dataset_size: 396598099 - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 396417981 num_examples: 1000 download_size: 132157138 dataset_size: 396417981 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397608105 num_examples: 1000 download_size: 145886861 dataset_size: 397608105 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397139422 num_examples: 1000 download_size: 146184268 dataset_size: 397139422 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397649163 num_examples: 1000 download_size: 146790309 dataset_size: 397649163 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397485471 num_examples: 1000 download_size: 146171554 dataset_size: 397485471 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397582299 num_examples: 1000 download_size: 146417032 dataset_size: 397582299 - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 397601172 num_examples: 1000 download_size: 146092040 dataset_size: 397601172 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-0--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 618569175 num_examples: 1000 download_size: 373822299 dataset_size: 618569175 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-128--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 616580997 num_examples: 1000 download_size: 371948452 dataset_size: 616580997 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-256--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 616570422 num_examples: 1000 download_size: 371819018 dataset_size: 616570422 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-42--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 618586859 num_examples: 1000 download_size: 374261890 dataset_size: 618586859 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-512--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 618031468 num_examples: 1000 download_size: 373172294 dataset_size: 618031468 - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-64--agg_strategy-last features: - name: problem dtype: string - name: level dtype: string - name: type dtype: string - name: solution dtype: string - name: answer dtype: string - name: completions list: string - name: scores list: 'null' - name: pred dtype: string - name: completion_tokens list: int64 - name: pred_maj@1 dtype: string - name: pred_maj@2 dtype: string - name: pred_maj@4 dtype: string - name: pred_maj@8 dtype: string - name: pred_maj@16 dtype: string - name: pred_maj@32 dtype: string - name: pred_maj@64 dtype: string - name: pred_maj@128 dtype: string - name: pred_maj@256 dtype: string - name: preds list: string - name: pass@1 dtype: float64 - name: pass@2 dtype: float64 - name: pass@4 dtype: float64 - name: pass@8 dtype: float64 - name: pass@16 dtype: float64 - name: pass@32 dtype: float64 - name: pass@64 dtype: float64 - name: pass@128 dtype: float64 - name: pass@256 dtype: float64 - name: is_correct_maj@1 dtype: bool - name: is_correct_maj@2 dtype: bool - name: is_correct_maj@4 dtype: bool - name: is_correct_maj@8 dtype: bool - name: is_correct_maj@16 dtype: bool - name: is_correct_maj@32 dtype: bool - name: is_correct_maj@64 dtype: bool - name: is_correct_maj@128 dtype: bool - name: is_correct_maj@256 dtype: bool - name: is_correct_preds list: bool - name: _preprocessing_metadata struct: - name: num_pred_fields dtype: int64 - name: preprocessed_at dtype: string - name: preprocessing_version dtype: string - name: total_evaluations dtype: int64 - name: model_difficulty dtype: int64 splits: - name: train num_bytes: 619987458 num_examples: 1000 download_size: 374914374 dataset_size: 619987458 configs: - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.2--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.3--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.4--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.5--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.6--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.7--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.8--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-0.9--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.0--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.1--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-0--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-128--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-256--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-42--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-512--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - config_name: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-64--agg_strategy-last data_files: - split: train path: ENSEONG_math-private--T-1.2--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* ---数据集信息： - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last，其中`T`为生成温度参数，`top_p`为核采样阈值，`n`为单次生成的样本数，`seed`为随机种子，`agg_strategy-last`表示采用最后一次生成结果作为聚合策略 特征： - 字段名：problem，数据类型：字符串型（string），表示题目内容 - 字段名：level，数据类型：字符串型（string），表示题目难度等级 - 字段名：type，数据类型：字符串型（string），表示题目类型 - 字段名：solution，数据类型：字符串型（string），表示参考解答 - 字段名：answer，数据类型：字符串型（string），表示标准答案 - 字段名：completions，数据类型：字符串列表（list: string），表示模型生成的补全结果列表 - 字段名：scores，数据类型：空值列表（list: 'null'） - 字段名：pred，数据类型：字符串型（string），表示单条预测结果 - 字段名：completion_tokens，数据类型：64位整数列表（list: int64），表示生成补全的Token数量列表 - 字段名：pred_maj@1，数据类型：字符串型（string），表示多数投票预测结果@1（pred_maj@1） - 字段名：pred_maj@2，数据类型：字符串型（string），表示多数投票预测结果@2（pred_maj@2） - 字段名：pred_maj@4，数据类型：字符串型（string），表示多数投票预测结果@4（pred_maj@4） - 字段名：pred_maj@8，数据类型：字符串型（string），表示多数投票预测结果@8（pred_maj@8） - 字段名：pred_maj@16，数据类型：字符串型（string），表示多数投票预测结果@16（pred_maj@16） - 字段名：pred_maj@32，数据类型：字符串型（string），表示多数投票预测结果@32（pred_maj@32） - 字段名：pred_maj@64，数据类型：字符串型（string），表示多数投票预测结果@64（pred_maj@64） - 字段名：pred_maj@128，数据类型：字符串型（string），表示多数投票预测结果@128（pred_maj@128） - 字段名：pred_maj@256，数据类型：字符串型（string），表示多数投票预测结果@256（pred_maj@256） - 字段名：preds，数据类型：字符串列表（list: string），表示所有生成的预测结果列表 - 字段名：pass@1，数据类型：64位浮点型（float64），表示通过率@1（pass@1），即k=1时的生成通过率 - 字段名：pass@2，数据类型：64位浮点型（float64），表示通过率@2（pass@2），即k=2时的生成通过率 - 字段名：pass@4，数据类型：64位浮点型（float64），表示通过率@4（pass@4），即k=4时的生成通过率 - 字段名：pass@8，数据类型：64位浮点型（float64），表示通过率@8（pass@8），即k=8时的生成通过率 - 字段名：pass@16，数据类型：64位浮点型（float64），表示通过率@16（pass@16），即k=16时的生成通过率 - 字段名：pass@32，数据类型：64位浮点型（float64），表示通过率@32（pass@32），即k=32时的生成通过率 - 字段名：pass@64，数据类型：64位浮点型（float64），表示通过率@64（pass@64），即k=64时的生成通过率 - 字段名：pass@128，数据类型：64位浮点型（float64），表示通过率@128（pass@128），即k=128时的生成通过率 - 字段名：pass@256，数据类型：64位浮点型（float64），表示通过率@256（pass@256），即k=256时的生成通过率 - 字段名：is_correct_maj@1，数据类型：布尔型（bool），表示多数投票预测正确性@1（is_correct_maj@1） - 字段名：is_correct_maj@2，数据类型：布尔型（bool），表示多数投票预测正确性@2（is_correct_maj@2） - 字段名：is_correct_maj@4，数据类型：布尔型（bool），表示多数投票预测正确性@4（is_correct_maj@4） - 字段名：is_correct_maj@8，数据类型：布尔型（bool），表示多数投票预测正确性@8（is_correct_maj@8） - 字段名：is_correct_maj@16，数据类型：布尔型（bool），表示多数投票预测正确性@16（is_correct_maj@16） - 字段名：is_correct_maj@32，数据类型：布尔型（bool），表示多数投票预测正确性@32（is_correct_maj@32） - 字段名：is_correct_maj@64，数据类型：布尔型（bool），表示多数投票预测正确性@64（is_correct_maj@64） - 字段名：is_correct_maj@128，数据类型：布尔型（bool），表示多数投票预测正确性@128（is_correct_maj@128） - 字段名：is_correct_maj@256，数据类型：布尔型（bool），表示多数投票预测正确性@256（is_correct_maj@256） - 字段名：is_correct_preds，数据类型：布尔值列表（list: bool），表示各预测结果的正确性标识列表 - 字段名：_preprocessing_metadata，数据类型：结构体（struct），表示预处理元数据，包含以下字段： - 字段名：num_pred_fields，数据类型：64位整数型（int64），表示预测字段数量 - 字段名：preprocessed_at，数据类型：字符串型（string），表示预处理完成时间 - 字段名：preprocessing_version，数据类型：字符串型（string），表示预处理版本 - 字段名：total_evaluations，数据类型：64位整数型（int64），表示总评估次数 - 字段名：model_difficulty，数据类型：64位整数型（int64），表示模型难度 数据集拆分： - 拆分名称：train，数据字节数：441775100，样本数量：1000 下载大小：71965406，数据集大小：441775100 后续所有配置项的结构与上述配置完全一致，仅配置名称、数据字节数、下载大小及数据集大小存在差异，配置名称格式为`ENSEONG_math-private--T-[温度参数]--top_p-1.0--n-256--seed-[随机种子]--agg_strategy-last`，其中温度参数取值范围为0.1至1.2，随机种子取值为0、128、256、42、512、64。 配置列表： - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-0--agg_strategy-last/train-* - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-128--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-128--agg_strategy-last/train-* - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-256--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-256--agg_strategy-last/train-* - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-42--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-42--agg_strategy-last/train-* - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-512--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-512--agg_strategy-last/train-* - 配置名称：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-64--agg_strategy-last，数据文件： - 拆分：train，数据路径：ENSEONG_math-private--T-0.1--top_p-1.0--n-256--seed-64--agg_strategy-last/train-* 后续配置项按此格式类推，仅配置名称与对应数据路径随温度参数及随机种子的变化调整。