数字孪生环境图像重建

本数据集主要面向智能体仿真训练与虚实迁移研究，针对动态操作场景下仿真环境高精度建模需求建设。基于清华大学智能技术与系统国家重点实验室观测站的RealSense D455深度相机与GoPro相机阵列产生，主要记录了多视角RGB-D图像序列、三维点云数据、NeRF/3DGS模型参数及可交互场景资产等观测值。数据来源于对3C产线与航天舱段场景的实景扫描与摄影测量，采用多视角同步采集方案，通过COLMAP进行外参预对齐，利用Instant-NGP进行神经辐射场训练，最终经MeshLab后处理生成可插拔USD格式场景资产。产生方法融合计算机视觉与深度学习技术，实现从真实场景到数字孪生环境的高保真重建。数据集主要内容包括高精度NeRF/3DGS模型（空间分辨率达毫米级）、多视角纹理映射、场景几何结构及光照参数等。所有模型均通过严格质控流程，确保虚实关键尺寸偏差≤5.0%，重投影图像信噪比保持在高品质区间，为仿真训练提供可靠的视觉一致性与物理准确性保障。数据体量达100MB，包含约200万张多视角RGB-D帧，采集周期自2020年9月至2024年12月。遵循CC BY-NC 4.0许可协议，设置5年保存期限。本数据集通过提供高保真数字孪生环境，有效支撑智能体在仿真环境中的技能训练与迁移验证，显著提升技能学习效率并降低实机调试风险，为复杂操作任务的分层学习与跨平台演示提供关键环境支撑。This dataset is primarily developed for AI Agent simulation training and sim-to-real transfer research, targeting the demand for high-precision modeling of simulation environments in dynamic operation scenarios. It is generated using RealSense D455 depth cameras and GoPro camera arrays from the observation station of the State Key Laboratory of Intelligent Technology and Systems, Tsinghua University, and mainly records observational data including multi-view RGB-D image sequences, 3D point cloud data, NeRF/3DGS model parameters, and interactive scene assets. The data is sourced from real-scene scanning and photogrammetry of 3C production lines and spacecraft module segment scenarios. It adopts a multi-view synchronous acquisition scheme, with extrinsic parameter pre-alignment performed via COLMAP, Neural Radiance Field training conducted using Instant-NGP, and finally post-processed with MeshLab to generate plug-and-play USD format scene assets. Its generation method integrates computer vision and deep learning technologies, achieving high-fidelity reconstruction from real-world scenes to digital twin environments. The main contents of the dataset include high-precision NeRF/3DGS models with a spatial resolution up to millimeter level, multi-view texture mapping, scene geometric structures, lighting parameters, and other related data. All models have passed strict quality control procedures, ensuring that the key dimensional deviation between virtual and real scenes is ≤5.0%, and the signal-to-noise ratio (SNR) of reprojected images remains in the high-quality range, providing reliable guarantees of visual consistency and physical accuracy for simulation training. The dataset has a total volume of 100 MB, containing approximately 2 million multi-view RGB-D frames, with an acquisition period from September 2020 to December 2024. It follows the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0) license, with a 5-year retention period. By providing high-fidelity digital twin environments, this dataset effectively supports AI Agents in skill training and transfer validation within simulation environments, significantly improving skill learning efficiency and reducing the risks of real-machine debugging, and providing critical environmental support for hierarchical learning and cross-platform demonstration of complex operation tasks.

搜集汇总

背景与挑战

背景概述

该数据集面向智能体仿真训练与虚实迁移研究，基于清华大学实验室的相机阵列采集3C产线与航天舱段场景的多视角RGB-D图像序列和三维点云数据，通过计算机视觉与深度学习技术实现高保真数字孪生环境重建。数据集包含高精度NeRF/3DGS模型、场景几何结构及光照参数，体量约100MB，采集周期从2020年9月至2024年12月，遵循CC BY-NC 4.0许可协议，为智能体技能训练提供视觉一致性与物理准确性保障。