pbelcak/pmc-train-3900000-to-4000000-GemmaTokens

--- dataset_info: features: - name: input_ids sequence: int32 - name: attention_mask sequence: int8 splits: - name: train num_bytes: 4024855802 num_examples: 834789 download_size: 1430956064 dataset_size: 4024855802 configs: - config_name: default data_files: - split: train path: data/train-* ---The dataset includes two features: input_ids (sequence type, int32) and attention_mask (sequence type, int8). It is divided into a training set with 834789 examples, totaling 4024855802 bytes. The download size of the dataset is 1430956064 bytes. The dataset has a default configuration with training data files located at data/train-*.

原始信息汇总

数据集概述

数据集特征

• input_ids

  • 数据类型：int32
  • 序列类型：sequence

• attention_mask

  • 数据类型：int8
  • 序列类型：sequence

数据集分割

• 训练集（train）
  • 数据大小：4024855802 字节
  • 样本数量：834789

数据集下载与大小

• 下载大小：1430956064 字节
• 数据集总大小：4024855802 字节

配置信息

• 配置名称：default
• 数据文件路径：
  • 分割类型：train
  • 文件路径模式：data/train-*