EmokineDataset

EmokineDataset Companion resources Paper Christensen, Julia F. and Fernandez, Andres and Smith, Rebecca and Michalareas, Georgios and Yazdi, Sina H. N. and Farahi, Fahima and Schmidt, Eva-Madeleine and Bahmanian, Nasimeh and Roig, Gemma (2024): "EMOKINE: A Software Package and Computational Framework for Scaling Up the Creation of Highly Controlled Emotional Full-Body Movement Datasets". Code https://github.com/andres-fr/emokine EmokineDataset is a pilot dataset showcasing the usefulness of the emokine software library. It featuers a single dancer performing 63 short sequences, which have been recorded and analyzed in different ways. This pilot dataset is organized in 3 folders: Stimuli: The sequences are presented in 4 visual presentations that can be used as stimulus in observer experiments: Silhouette: Videos with a white silhouette of the dancer on black background. FLD (Full-Light Display): video recordings with the performer's face blurred out. PLD (Point-Light Display): videos featuring a black background with white circles corresponding to the selected body landmarks. Avatar: Videos produced by the XSENS motion capture propietary software, featuring a robot-like avatar performing the captured movements on a light blue background. Data: In order to facilitate computation and analysis of the stimuli, this pilot dataset also includes several data formats: MVNX: Raw motion capture data directly recorded from the XSENS motion capture system. CSV: Translation of a subset of the MVNX sequences into CSV, included for easier integration with mainstream analysis software tools). The subset includes the following features: acceleration, angularAcceleration, angularVelocity, centerOfMass, footContacts, orientation, position and velocity. CamPos: While the MVNX provides 3D positions with respect to a global frame of reference, the CamPos [JSON](https://www.json.org/json-en.html) files represent the position from the perspective of the camera used to render the PLD videos. Specifically, their 3D positions are given with respect to the camera as (x, y, z), where (x, y) go from (0, 0) (left, bottom) to (1, 1) (right, top), and z is the distance between the camera and the point in meters. It can be useful to get a 2-dimensional projection of the dancer position (simply ignore z). Kinematic: Analysis of a selection of relevant kinematic features, using information from MVNX, Silhouette and CamPos, provided in tabular form. Validation: Data and experiments reported in our paper as part of the data validation, to support its meaningfulness and usefulness for downstream tasks. TechVal: A collection of plots presenting relevant statistics of the pilot dataset. ObserverExperiment: Results in tabular form of an online study conducted with human participants, tasked to recognize emotions of the stimuli and rate their beauty. More specifically, the 63 unique sequences are divided into 9 unique choreographies, each one being performed once as an explanation, and then 6 times with different intended emotions (angry, content, fearful, joy, neutral and sad). Once downloaded, the pilot dataset should have the following structure: EmokineDataset ├── Stimuli │ ├── Avatar │ ├── FLD │ ├── PLD │ └── Silhouette ├── Data │ ├── CamPos │ ├── CSV │ ├── Kinematic │ ├── MVNX │ └── TechVal └── Validation ├── TechVal └── ObserverExperiment Where each <MODALITY> of the stimuli, MVNX, CamPos and Kinematic have this structure: <MODALITY> ├── explanation │ ├── <MODALITY>_seq1_explanation.<EXTENSION> │ ├── ... │ └── <MODALITY>_seq9_explanation.<EXTENSION> ├── <MODALITY>_seq1_angry.<EXTENSION> ├── <MODALITY>_seq1_content.<EXTENSION> ├── <MODALITY>_seq1_fearful.<EXTENSION> ├── <MODALITY>_seq1_joy.<EXTENSION> ├── <MODALITY>_seq1_neutral.<EXTENSION> ├── <MODALITY>_seq1_sad.<EXTENSION> ... └── <MODALITY>_seq9_sad.<EXTENSION> The CSV directory is slightly different, because instead of a single file for each seq and emotion, it features a folder containing a .csv file for each one of the 8 features being extracted (acceleration, velocity...).EmokineDataset 配套资源与论文 作者为 Christensen, Julia F.、Fernandez, Andres、Smith, Rebecca、Michalareas, Georgios、Yazdi, Sina H. N.、Farahi, Fahima、Schmidt, Eva-Madeleine、Bahmanian, Nasimeh 与 Roig, Gemma（2024）：《EMOKINE：一款用于规模化构建高度可控的情感全身运动数据集的软件包与计算框架》。代码仓库：https://github.com/andres-fr/emokine。 EmokineDataset 是一款用于演示 emokine 软件库实用价值的试点数据集。该数据集包含一名舞者完成的63段简短序列，所有序列均已通过多种方式完成录制与分析。 该试点数据集共分为3个文件夹： 1. **Stimuli（刺激素材）**：该目录下的序列可作为观察者实验的刺激材料，包含4种视觉呈现形式： - Silhouette（剪影视频）：以纯黑色为背景，呈现舞者白色剪影的视频文件； - FLD（全光照视频，Full-Light Display）：表演者面部已做模糊处理的实拍录像； - PLD（点光源显示视频，Point-Light Display）：以纯黑色为背景，使用白色圆点标注选定身体关键点的视频； - Avatar（化身视频）：由 XSENS 动作捕捉（motion capture）专有软件生成的视频，以浅蓝色为背景，呈现类机器人化身执行捕捉到的动作序列。 2. **Data（数据文件）**：为便于对刺激素材进行计算与分析，该试点数据集还包含多种格式的数据： - MVNX：直接从 XSENS 动作捕捉系统采集的原始动作捕捉数据； - CSV（逗号分隔值，Comma-Separated Values）：将部分 MVNX 序列转换为 CSV 格式的数据，旨在更便捷地与主流分析软件工具集成。该子集包含以下特征：加速度、角加速度、角速度、质心位置、足部接触点、姿态、位置与速度； - CamPos [JSON（JavaScript 对象标记）]：尽管 MVNX 提供了基于全局参考坐标系的三维位置信息，但 CamPos JSON 文件呈现的是渲染 PLD 视频所用相机视角下的位置信息。具体而言，其三维位置以相机为参考坐标系，采用 (x, y, z) 形式表示，其中 (x, y) 的取值范围为 (0, 0)（左下角）至 (1, 1)（右上角），z 为相机与该点之间的距离（单位：米）。若需获取舞者位置的二维投影，可直接忽略 z 分量，该特性具备实用价值； - Kinematic（运动学数据）：基于 MVNX、剪影视频与 CamPos 信息，对选定的关键运动学特征进行分析后得到的表格形式数据； - TechVal（技术验证数据）：收录展示该试点数据集相关统计信息的图表集合。 3. **Validation（验证数据集）**：包含论文中报告的用于数据验证的实验与数据，用以支撑该数据集在下游任务中的有效性与实用价值。其中 **ObserverExperiment（观察者实验数据）**：以表格形式呈现的人类参与者在线实验结果，参与者需识别刺激素材中的情感并对其美感进行评分。具体而言，63 段独特序列被划分为9段独特编舞，每段编舞先作为示范表演一次，随后分别以愤怒、愉悦、恐惧、喜悦、中性与悲伤六种预设情感各表演一次。 下载完成后，该试点数据集的完整目录结构如下： EmokineDataset ├── Stimuli │ ├── Avatar │ ├── FLD │ ├── PLD │ └── Silhouette ├── Data │ ├── CamPos │ ├── CSV │ ├── Kinematic │ ├── MVNX │ └── TechVal └── Validation ├── TechVal └── ObserverExperiment 刺激素材的各模态（即 Stimuli 下的子目录）、MVNX、CamPos 与 Kinematic 均采用如下统一目录结构： <MODALITY> ├── explanation │ ├── <MODALITY>_seq1_explanation.<EXTENSION> │ ├── ... │ └── <MODALITY>_seq9_explanation.<EXTENSION> ├── <MODALITY>_seq1_angry.<EXTENSION> ├── <MODALITY>_seq1_content.<EXTENSION> ├── <MODALITY>_seq1_fearful.<EXTENSION> ├── <MODALITY>_seq1_joy.<EXTENSION> ├── <MODALITY>_seq1_neutral.<EXTENSION> ├── <MODALITY>_seq1_sad.<EXTENSION> ... └── <MODALITY>_seq9_sad.<EXTENSION> CSV 目录的结构略有不同：其并非为每个序列与情感对应单个文件，而是为提取的8项特征（加速度、速度等）分别创建一个文件夹，每个文件夹内包含对应的 .csv 文件。