ai2-adapt-dev/flan_v2_converted

This is a converted version of the Flan dataset into Tulu SFT training format.

The conversion script can be found in our [open-instruct](https://github.com/allenai/open-instruct/blob/main/scripts/data/sft/flan.py) repo.
The conversion took the following parameters:
- apply_keyword_filters: True
- apply_empty_message_filters: True
- push_to_hub: True
- hf_entity: ai2-adapt-dev
- converted_dataset_name: flan_v2_converted
- local_save_dir: ./data/sft/flan

The original FLAN dataset needs extensive efforts to be regenerated, so we are using [a reproduced version by the OpenOrca team](https://huggingface.co/datasets/Open-Orca/FLAN).More specifically, we only use their top level jsonl files, which is a subset of the original dataset.And by default, we only use the `cot_fsopt_data`, `cot_zsopt_data`, `niv2_fsopt_data`, `niv2_zsopt_data` `flan_fsopt_data`, `flan_zsopt_data`, `t0_fsopt_data` subsets.If you want to use more data, you can modify this script to load more data from their Huggingface repo.Please refer to their Huggingface repo [here](https://huggingface.co/datasets/Open-Orca/FLAN) and the [original FLAN v2 repo](https://github.com/google-research/FLAN/tree/main/flan/v2) for more information about this dataset and the license.

本数据集是将FLAN数据集转换为Tulu监督微调（SFT，Supervised Fine-tuning）训练格式的版本。
转换脚本可在我们的[open-instruct](https://github.com/allenai/open-instruct/blob/main/scripts/data/sft/flan.py)代码仓库中获取。
本次转换采用了如下参数：
- 启用关键词过滤：是
- 启用空消息过滤：是
- 推送到Hugging Face Hub：是
- Hugging Face Hub实体：ai2-adapt-dev
- 转换后数据集名称：flan_v2_converted
- 本地保存路径：./data/sft/flan
原始FLAN数据集的重新生成需要投入大量工作，因此我们采用了OpenOrca团队复现的版本([OpenOrca团队复现版FLAN数据集](https://huggingface.co/datasets/Open-Orca/FLAN))。
具体而言，我们仅使用其顶层JSONL文件，该文件是原始数据集的一个子集。且默认情况下，我们仅使用`cot_fsopt_data`、`cot_zsopt_data`、`niv2_fsopt_data`、`niv2_zsopt_data`、`flan_fsopt_data`、`flan_zsopt_data`以及`t0_fsopt_data`这些子集。
若需使用更多数据，您可修改本转换脚本，从其Hugging Face代码仓库中加载更多数据。有关本数据集及其授权协议的更多信息，请参阅其Hugging Face仓库[此处](https://huggingface.co/datasets/Open-Orca/FLAN)以及原始FLAN v2代码仓库[此处](https://github.com/google-research/FLAN/tree/main/flan/v2)。