distilabel-internal-testing/mt-bench-eval-critique

--- dataset_info: features: - name: idx dtype: int64 - name: response_source dtype: string - name: category dtype: string - name: instruction dtype: string - name: response dtype: string splits: - name: train num_bytes: 970116 num_examples: 320 download_size: 398377 dataset_size: 970116 configs: - config_name: default data_files: - split: train path: data/train-* --- ## Description This dataset is used to check criticon prompts/responses while testing, it contains instructions/responses from mt_bench_eval, as extracted from: https://github.com/kaistAI/prometheus/blob/main/evaluation/benchmark/data/mt_bench_eval.json The dataset has been obtained cleaning the data with: ```python import re import pandas as pd from datasets import Dataset df = pd.read_json("mt_bench_eval.json", lines=True) ds = Dataset.from_pandas(df, preserve_index=False) def prepare_ds(example): example["instruction"].split("###The instruction to evaluate:") pat1 = r'###The instruction to evaluate:(.*?)###Response to evaluate:' pat2 = r'###Response to evaluate:(.*?)###Reference Answer' matches_1 = re.search(pat1, example["instruction"], re.DOTALL) matches_2 = re.search(pat2, example["instruction"], re.DOTALL) instruction = response = None if matches_1: instruction = matches_1.group(1).strip() if matches_2: response = matches_2.group(1).strip() return { "instruction": instruction, "response": response } ds = ds.map(prepare_ds) ```