junsong9001/TencentGR-1M

--- configs: - config_name: candidate data_files: - split: train path: candidate/**/*.parquet - config_name: item_feat data_files: - split: train path: item_feat/**/*.parquet - config_name: seq data_files: - split: train path: seq/**/*.parquet - config_name: user_feat data_files: - split: train path: user_feat/**/*.parquet - config_name: mm_emb_81_32 data_files: - split: train path: mm_emb/emb_81_32_parquet/**/*.parquet - config_name: mm_emb_82_1024 data_files: - split: train path: mm_emb/emb_82_1024_parquet/**/*.parquet - config_name: mm_emb_83_3584 data_files: - split: train path: mm_emb/emb_83_3584_parquet/**/*.parquet - config_name: mm_emb_84_4096 data_files: - split: train path: mm_emb/emb_84_4096_parquet/**/*.parquet - config_name: mm_emb_85_3584 data_files: - split: train path: mm_emb/emb_85_3584_parquet/**/*.parquet - config_name: mm_emb_86_3584 data_files: - split: train path: mm_emb/emb_86_3584_parquet/**/*.parquet license: cc-by-4.0 --- # TencentGR-1M Dataset TAAC2025 Preliminary Round Dataset(2025年腾讯广告算法大赛初赛数据集) TencentGR-1M Dataset is a large-scale, all-modality dataset designed specifically for generative recommendation (GR) in industrial advertising. Constructed from real, de-identified Tencent Ads logs, it aims to address the lack of realistic, public multi-modal datasets in the GR field. - Data Features: Contains rich collaborative IDs and multi-modal representations (text and vision) extracted using state-of-the-art embedding models. - Dataset Size: Provides 1 million user sequences, with each user sequence containing up to 100 interacted items. - Labels: Each interaction within the sequence is explicitly labeled with **exposure(0)** and **click(1)** signals. ## Dataset Structure ### Overview | Config Name | Path | Approx. Size | Description | |---|---|---|---| | `candidate` | `candidate/` | ~22 MB | Candidate item set | | `item_feat` | `item_feat/` | ~104 MB | Item features | | `seq` | `seq/` | ~881 MB | User behavior sequences | | `user_feat` | `user_feat/` | ~8.4 MB | User features | | `mm_emb_81_32` | `mm_emb/emb_81_32_parquet/` | ~901 MB | Multimodal embedding (dim=32) | | `mm_emb_82_1024` | `mm_emb/emb_82_1024_parquet/` | ~9.4 GB | Multimodal embedding (dim=1024) | | `mm_emb_83_3584` | `mm_emb/emb_83_3584_parquet/` | ~31 GB | Multimodal embedding (dim=3584) | | `mm_emb_84_4096` | `mm_emb/emb_84_4096_parquet/` | ~30 GB | Multimodal embedding (dim=4096) | | `mm_emb_85_3584` | `mm_emb/emb_85_3584_parquet/` | ~31 GB | Multimodal embedding (dim=3584) | | `mm_emb_86_3584` | `mm_emb/emb_86_3584_parquet/` | ~26 GB | Multimodal embedding (dim=3584) | ### Additional Files | File | Size | Description | |---|---|---| | `indexer.pkl` | ~142 MB | Index mapping file (From original ID to remapped ID) | ### Data Format All data files are stored in **Snappy-compressed Parquet** format. ### Schema For clarity and brevity, we provide detailed schema descriptions for each table below. Need to notice that we use two types of IDs in the dataset: the original IDs and the remapped IDs, for simplicity, we will denote them as OID and RID. OIDs are used in `mm_emb`, and RIDs are used in all the training data and can be used for building models. The mapping between OIDs and RIDs can be found in the `indexer.pkl` file. #### `item_feat` The `item_feat` table contains the features of each item appeared in the `seq` set. <!-- 15 x 3 table: --> | **Field** | **Type** | **Description** | \# Non-None Values | |:---:|:---:|:---:|:---:| | `item_id` | int64 | RID for each item. |4783154 | | `100` | int64 | An encrypted feature. | 4779045 | | `101` | int64 | An encrypted feature. | 4779045 | | `102` | int64 | An encrypted feature. | 4735917 | | `112` | int64 | An encrypted feature. | 4701740 | | `114` | int64 | An encrypted feature. | 4778327 | | `115` | int64 | An encrypted feature. | 1531415 | | `116` | int64 | An encrypted feature. | 4778146 | | `117` | int64 | An encrypted feature. | 4701740 | | `118` | int64 | An encrypted feature. | 4700703 | | `119` | int64 | An encrypted feature. | 4699894 | | `120` | int64 | An encrypted feature. | 4694982 | | `121` | int64 | An encrypted feature. | 4783154 | | `122` | int64 | An encrypted feature. | 4779045 | #### `user_feat` The `user_feat` table contains the features of each user appeared in the dataset. | **Field** | **Type** | **Description** | \# Non-None Values | |:---:|:---:|:---:|:---:| | `user_id` | int64 | RID for each user. | 1001845 | | `103` | int64 | An encrypted feature. | 1000964 | | `104` | int64 | An encrypted feature. | 998043 | | `105` | int64 | An encrypted feature. | 859602 | | `106` | List\[int64\] | An encrypted feature. | 880754 | | `107` | List\[int64\] | An encrypted feature. | 387686 | | `108` | List\[int64\] | An encrypted feature. | 170678 | | `109` | int64 | An encrypted feature. | 1001467 | | `110` | List\[int64\] | An encrypted feature. | 430598 | #### `seq` The `seq` table contains the behavior sequence for each user. | **Field** | **Type** | **Description** | \# Non-None Values | |:---:|:---:|:---:|:---:| | `user_id` | int64 | RID for each user. | 1001845 | | `seq` | List\[Dict\] | The behavior sequence for each user, each dict contains 3 keys: `item_id`(RID), `action_type`, and `timestamp`, where the values are all integers | 1001845 | #### `candidate` The `candidate` table contains the candidate items for the competition. Note: - This `candidate` is for the competition, but we do not provide the ground truth labels. People may refer to this format to build their own candidate set. - The `candidate` contains some items that are not in the `seq`. | **Field** | **Type** | **Description** | \# Non-None Values | |:---:|:---:|:---:|:---:| | `item_id` | int64 | OID for each item. | 660000 | | `retrieval_id` | int64 | The remapped ID for faiss retrieval (Start from 0). | 660000 | | `100` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 659206 | | `101` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 659206 | | `102` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 653852 | | `112` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 654893 | | `114` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 659093 | | `115` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 195552 | | `116` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 659090 | | `117` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 654893 | | `118` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 654886 | | `119` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 654882 | | `120` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 654870 | | `121` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 660000 | | `122` | Dict\[ "cold\_start": int64, "feature_value": string \] | An encrypted feature. | 659206 | #### `mm_emb` The `mm_emb` tables contain the multimodal embeddings for each item. There are 6 different embedding dimensions(`[32, 1024, 3584, 4096, 3584, 3584]`) for 6 different embeddings(`[81, 82, 83, 84, 85, 86]`) placed in 6 files. Take the `81` embedding as an example: | **Field** | **Type** | **Description** | \# Non-None Values | |:---:|:---:|:---:|:---:| | `anonymous_cid` | string | OID for each item. | 4742961 | | `emb` | List\[ double \] | Embedding for each item. | 4742961 | #### `indexer.pkl` This is a remapping file that maps the original IDs/Values to the remapped IDs/Values. The format is a dictionary with the following structure: ```json { "u": { OID: RID, ... }, "i": { OID: RID, ... }, "f": { 101: { 10100000000: 1, // original value: remapped value 10100000001: 2, ... }, 102: { 1020000000: 1, 1020000001: 2, ... }, ... } } ``` ## Usage ```python from datasets import load_dataset # Load a specific config ds = load_dataset("TAAC2025/TencentGR-1M", name="candidate", split="train") # Load item features ds_item = load_dataset("TAAC2025/TencentGR-1M", name="item_feat", split="train") # Load user behavior sequences ds_seq = load_dataset("TAAC2025/TencentGR-1M", name="seq", split="train") # Load user features ds_user = load_dataset("TAAC2025/TencentGR-1M", name="user_feat", split="train") # Load multimodal embeddings ds_emb = load_dataset("TAAC2025/TencentGR-1M", name="mm_emb_81_32", split="train") ```配置项： - 配置名称：candidate 数据文件： - 拆分集：train 路径：candidate/**/*.parquet - 配置名称：item_feat 数据文件： - 拆分集：train 路径：item_feat/**/*.parquet - 配置名称：seq 数据文件： - 拆分集：train 路径：seq/**/*.parquet - 配置名称：user_feat 数据文件： - 拆分集：train 路径：user_feat/**/*.parquet - 配置名称：mm_emb_81_32 数据文件： - 拆分集：train 路径：mm_emb/emb_81_32_parquet/**/*.parquet - 配置名称：mm_emb_82_1024 数据文件： - 拆分集：train 路径：mm_emb/emb_82_1024_parquet/**/*.parquet - 配置名称：mm_emb_83_3584 数据文件： - 拆分集：train 路径：mm_emb/emb_83_3584_parquet/**/*.parquet - 配置名称：mm_emb_84_4096 数据文件： - 拆分集：train 路径：mm_emb/emb_84_4096_parquet/**/*.parquet - 配置名称：mm_emb_85_3584 数据文件： - 拆分集：train 路径：mm_emb/emb_85_3584_parquet/**/*.parquet - 配置名称：mm_emb_86_3584 数据文件： - 拆分集：train 路径：mm_emb/emb_86_3584_parquet/**/*.parquet 许可证：cc-by-4.0 # TencentGR-1M 数据集 （TAAC2025 初赛数据集）TencentGR-1M 数据集是专为工业广告领域生成式推荐（Generative Recommendation, GR）设计的大规模全模态数据集，其数据源自腾讯广告已完成去标识化的真实日志，旨在弥补生成式推荐领域缺乏真实公开多模态数据集的空白。 - 数据特征：包含丰富的协同ID与采用前沿嵌入模型提取的多模态表征（文本与视觉模态）。 - 数据集规模：提供100万条用户行为序列，每条序列最多包含100个交互商品。 - 标签：序列内的每一次交互均标注了**曝光（0）**与**点击（1）**信号。 ## 数据集结构 ### 概览 | 配置名称 | 路径 | 近似大小 | 描述 | |---|---|---|---| | `candidate` | `candidate/` | ~22 MB | 候选商品集 | | `item_feat` | `item_feat/` | ~104 MB | 商品特征 | | `seq` | `seq/` | ~881 MB | 用户行为序列 | | `user_feat` | `user_feat/` | ~8.4 MB | 用户特征 | | `mm_emb_81_32` | `mm_emb/emb_81_32_parquet/` | ~901 MB | 多模态嵌入（维度=32） | | `mm_emb_82_1024` | `mm_emb/emb_82_1024_parquet/` | ~9.4 GB | 多模态嵌入（维度=1024） | | `mm_emb_83_3584` | `mm_emb/emb_83_3584_parquet/` | ~31 GB | 多模态嵌入（维度=3584） | | `mm_emb_84_4096` | `mm_emb/emb_84_4096_parquet/` | ~30 GB | 多模态嵌入（维度=4096） | | `mm_emb_85_3584` | `mm_emb/emb_85_3584_parquet/` | ~31 GB | 多模态嵌入（维度=3584） | | `mm_emb_86_3584` | `mm_emb/emb_86_3584_parquet/` | ~26 GB | 多模态嵌入（维度=3584） | ### 附加文件 | 文件 | 大小 | 描述 | |---|---|---| | `indexer.pkl` | ~142 MB | 索引映射文件（用于原始ID到重映射ID的转换） | ### 数据格式 所有数据文件均采用**Snappy压缩的Parquet**格式存储。 ### 数据Schema 为清晰简洁，下文将对每张表的详细schema进行说明。 需注意：本数据集使用两类ID：原始ID（Original ID, OID）与重映射ID（Remapped ID, RID）。其中OID仅用于`mm_emb`相关文件，RID可用于所有训练数据并直接用于模型构建。OID与RID的映射关系可在`indexer.pkl`文件中获取。 #### `item_feat` `item_feat`表包含`seq`数据集中出现的所有商品的特征。 | **字段名** | **数据类型** | **描述** | **非空值数量** | |:---:|:---:|:---:|:---:| | `item_id` | int64 | 商品的重映射ID | 4783154 | | `100` | int64 | 加密特征字段 | 4779045 | | `101` | int64 | 加密特征字段 | 4779045 | | `102` | int64 | 加密特征字段 | 4735917 | | `112` | int64 | 加密特征字段 | 4701740 | | `114` | int64 | 加密特征字段 | 4778327 | | `115` | int64 | 加密特征字段 | 1531415 | | `116` | int64 | 加密特征字段 | 4778146 | | `117` | int64 | 加密特征字段 | 4701740 | | `118` | int64 | 加密特征字段 | 4700703 | | `119` | int64 | 加密特征字段 | 4699894 | | `120` | int64 | 加密特征字段 | 4694982 | | `121` | int64 | 加密特征字段 | 4783154 | | `122` | int64 | 加密特征字段 | 4779045 | #### `user_feat` `user_feat`表包含数据集中所有用户的特征。 | **字段名** | **数据类型** | **描述** | **非空值数量** | |:---:|:---:|:---:|:---:| | `user_id` | int64 | 用户的重映射ID | 1001845 | | `103` | int64 | 加密特征字段 | 1000964 | | `104` | int64 | 加密特征字段 | 998043 | | `105` | int64 | 加密特征字段 | 859602 | | `106` | List[int64] | 加密特征字段 | 880754 | | `107` | List[int64] | 加密特征字段 | 387686 | | `108` | List[int64] | 加密特征字段 | 170678 | | `109` | int64 | 加密特征字段 | 1001467 | | `110` | List[int64] | 加密特征字段 | 430598 | #### `seq` `seq`表包含每个用户的行为序列。 | **字段名** | **数据类型** | **描述** | **非空值数量** | |:---:|:---:|:---:|:---:| | `user_id` | int64 | 用户的重映射ID | 1001845 | | `seq` | List[Dict] | 用户的行为序列，每个字典包含3个键：`item_id`（重映射ID）、`action_type`与`timestamp`，值均为整数类型 | 1001845 | #### `candidate` `candidate`表包含大赛所用的候选商品集。 注意： - 该`candidate`表仅为大赛专用候选集，未提供真实标签，参赛者可参考其格式自行构建候选集。 - 该表包含部分未在`seq`数据集中出现的商品。 | **字段名** | **数据类型** | **描述** | **非空值数量** | |:---:|:---:|:---:|:---:| | `item_id` | int64 | 商品的原始ID（OID） | 660000 | | `retrieval_id` | int64 | 用于Faiss检索的重映射ID（从0开始计数） | 660000 | | `100` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 659206 | | `101` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 659206 | | `102` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 653852 | | `112` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 654893 | | `114` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 659093 | | `115` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 195552 | | `116` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 659090 | | `117` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 654893 | | `118` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 654886 | | `119` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 654882 | | `120` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 654870 | | `121` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 660000 | | `122` | Dict["cold_start": int64, "feature_value": string] | 加密特征字段 | 659206 | #### `mm_emb` `mm_emb`系列表包含商品的多模态嵌入向量，共6组不同维度的嵌入（维度依次为32、1024、3584、4096、3584、3584），对应编号分别为81至86，分别存储于6个文件中。 以81号嵌入为例： | **字段名** | **数据类型** | **描述** | **非空值数量** | |:---:|:---:|:---:|:---:| | `anonymous_cid` | string | 商品的原始ID（OID） | 4742961 | | `emb` | List[double] | 商品的嵌入向量 | 4742961 | #### `indexer.pkl` 该文件为ID与特征值的重映射文件，格式为字典，结构如下： json { "u": { OID: RID, ... }, "i": { OID: RID, ... }, "f": { 101: { 10100000000: 1, // 原始特征值: 重映射特征值 10100000001: 2, ... }, 102: { 1020000000: 1, 1020000001: 2, ... }, ... } } ## 使用方法 python from datasets import load_dataset # 加载指定配置的数据集 ds = load_dataset("TAAC2025/TencentGR-1M", name="candidate", split="train") # 加载商品特征数据集 ds_item = load_dataset("TAAC2025/TencentGR-1M", name="item_feat", split="train") # 加载用户行为序列数据集 ds_seq = load_dataset("TAAC2025/TencentGR-1M", name="seq", split="train") # 加载用户特征数据集 ds_user = load_dataset("TAAC2025/TencentGR-1M", name="user_feat", split="train") # 加载多模态嵌入数据集 ds_emb = load_dataset("TAAC2025/TencentGR-1M", name="mm_emb_81_32", split="train")