RoboManip-Traj-Demo
收藏资源简介：
# Codatta Robotic Manipulation Trajectory (Sample) ## Overview This dataset contains high-quality annotated trajectories of robotic gripper manipulations. Produced by **Codatta**, it focuses on third-person views of robotic arms performing pick-and-place or manipulation tasks. The dataset is designed to train models for fine-grained control, trajectory prediction, and object interaction tasks. The scope specifically includes third-person views (fixed camera recording the robot) while explicitly excluding first-person views (Eye-in-Hand) to ensure consistent coordinate mapping. ## Dataset Contents Each sample in this dataset includes the raw video, a visualization of the trajectory, and a rigorous JSON annotation of keyframes and coordinate points. ### Data Fields * **`id`** (string): Unique identifier for the trajectory sequence. * **`total_frames`** (int32): Total number of frames in the video sequence. * **`video_path`** (string): Path to the source MP4 video file recording the manipulation action. * **`trajectory_image`** (image): A JPEG preview showing the overlaid trajectory path or keyframe visualization. * **`annotations`** (string): A JSON-formatted string containing the detailed coordinate data. It contains lists of keyframes, timestamps, and 5-point coordinates for the gripper. ### Annotation Standards The data follows a strict protocol to ensure precision: **1. Keyframe Selection** Annotations are sparse, focusing on specific Keyframes defined by the following events: * **Start Frame:** The gripper first appears in the screen. * **End Frame:** The gripper leaves the screen. * **Velocity Change:** Frames where the speed direction suddenly changes (marking the minimum speed point). * **State Change:** Frames where the gripper opens or closes. * **Contact:** The precise moment the gripper touches the object. **2. The 5-Point Annotation Method** For every annotated keyframe, the gripper is labeled with **5 specific coordinate points** to capture its pose and state accurately: | Point ID | Description | Location Detail | | :--- | :--- | :--- | | **Point 1 & 2** | **Fingertips** | Center of the bottom edge of the gripper tips. | | **Point 3 & 4** | **Gripper Ends** | The rearmost points of the closing area (indicating the finger direction). | | **Point 5** | **Tiger's Mouth** | The center of the crossbeam (base of the gripper). | **3. Quality Control** * **Accuracy:** All datasets passed a rigorous quality assurance process with a minimum **95% accuracy rate**. * **Occlusion Handling:** Sequences where the gripper is fully occluded or only shows a side profile without clear features are discarded. ## Key Statistics * **Total Examples:** 50 annotated examples (Sample Dataset). * **Language:** English (`en`). * **Splits:** Train split available. * **Download Size:** ~38.7 MB. * **Dataset Size:** ~39.0 MB. ## Usage This dataset is suitable for research and development in the field of Embodied AI and Computer Vision. It is specifically curated to support the following downstream tasks and application scenarios: * **Trajectory Prediction:** The high-precision coordinate data allows for training models to predict the future path of a gripper based on initial visual contexts. * **Keyframe Extraction & Event Detection:** By leveraging the labeled event types (e.g., "Contact", "Velocity Change"), models can be trained to automatically identify critical moments in long-horizon manipulation tasks. * **Fine-Grained Robotic Control:** The 5-point annotation system provides detailed pose information, enabling Imitation Learning (IL) from human-demonstrated or teleoperated data for precise pick-and-place operations. * **Object Interaction Analysis:** The dataset helps in understanding gripper-object relationships, specifically modeling the transition states when the gripper opens, closes, or makes contact with an object. ### Usage Example ```python from datasets import load_dataset import json # Load the dataset ds = load_dataset("Codatta/robotic-manipulation-trajectory", split="train") # Access a sample sample = ds[0] # View the image print(f"Trajectory ID: {sample['id']}") sample['trajectory_image'].show() # Parse annotations annotations = json.loads(sample['annotations']) print(f"Keyframes count: {len(annotations)}") ``` ## License and Open-Source Details * **License:** This dataset is released under the **OpenRAIL** license.
# Codatta 机器人操作轨迹数据集（样本版） ## 数据集概述 本数据集包含高质量的标注机器人夹持器操作轨迹，旨在为细粒度控制、轨迹预测以及物体交互任务的模型训练提供支撑。 本数据集由**Codatta**团队制作，聚焦于机械臂执行拾取-放置或操作任务的第三人称视角场景。每个样本均包含原始视频、轨迹可视化结果，以及包含关键帧与坐标点的标准化JSON标注文件。 **注意：本数据集为样本集，仅包含50个标注样本。** ## 支持任务 * **轨迹预测**：基于视觉上下文预测夹持器的运动路径。 * **关键帧提取**：识别操作任务中的关键节点（如接触、速度变化时刻）。 * **机器人控制**：基于人类演示或遥操作数据开展模仿学习。 ## 数据集结构 ### 数据字段 * **`id`**（字符串型）：轨迹序列的唯一标识符。 * **`total_frames`**（int32 类型）：视频序列的总帧数。 * **`video_path`**（字符串型）：记录操作动作的原始MP4视频文件路径。 * **`trajectory_image`**（图像类型）：叠加了轨迹路径或关键帧可视化结果的JPEG预览图。 * **`annotations`**（字符串型）：包含详细坐标数据的JSON格式字符串。 * *结构说明*：包含关键帧列表、时间戳，以及每个标注帧中夹持器的5点坐标信息。 ### 数据预览 *(Hugging Face 可视化工具将自动渲染`trajectory_image`字段内容)* ## 标注规范 本数据集遵循严格的标注协议以确保精度与一致性。 ### 1. 视角范围 * **包含场景**：第三人称视角（固定相机拍摄机械臂）。 * **排除场景**：明确排除第一人称视角（Eye-in-Hand），以确保坐标映射的一致性[引用来源：5、15]。 ### 2. 关键帧选取 本次标注并非逐帧密集标注，而是采用稀疏标注策略，聚焦于定义运动逻辑的**关键帧**。[引用来源：20-25]关键帧由以下事件定义： 1. **起始帧**：夹持器首次出现在画面中的帧[引用来源：21]。 2. **结束帧**：夹持器离开画面的帧[引用来源：22]。 3. **速度变化帧**：运动方向突然改变的帧（对应速度极小值点）[引用来源：23]。 4. **状态变化帧**：夹持器开合状态发生改变的帧[引用来源：24]。 5. **接触帧**：夹持器与物体接触的精确时刻[引用来源：25]。 ### 3. 五点标注法 [引用来源：27]针对每个标注的关键帧，我们通过5个特定坐标点来精准捕捉夹持器的位姿与状态： | 点ID | 描述 | 位置细节 | | :--- | :--- | :--- | | **点1与点2** | **指尖** | [引用来源：28、29]夹持器指尖底部边缘的中心点。 | | **点3与点4** | **夹持器末端** | [引用来源：31]夹持区域的最后端点位（用于指示手指朝向）。 | | **点5** | **虎口（Tiger's Mouth）** | [引用来源：32]横梁（夹持器基座）的中心点。 | ### 4. 质量管控 * **精度要求**：所有数据集均通过严格的质量保证流程，标注准确率不低于95%[引用来源：78]。 * **遮挡处理**：若夹持器被部分遮挡，将基于物体几何结构估算点位。[引用来源：58、63]若夹持器被完全遮挡，或仅呈现侧面轮廓且无清晰特征，则该序列将被弃用。 ## 使用示例 python from datasets import load_dataset import json # 加载数据集 ds = load_dataset("Codatta/robotic-manipulation-trajectory", split="train") # 访问单个样本 sample = ds[0] # 查看可视化图像 print(f"轨迹ID：{sample['id']}") sample['trajectory_image'].show() # 解析标注信息 annotations = json.loads(sample['annotations']) print(f"关键帧数量：{len(annotations)}")
