MMHOI Dataset: Modeling Complex 3D Multi-Human Multi-Object Interactions

Introduction

The MMHOI dataset addresses the challenge of modeling real-world scenes with multiple humans and objects interacting in causal, goal-oriented, or cooperative ways. Existing 3D human-object interaction (HOI) benchmarks capture only a subset of these complex interactions. MMHOI closes this gap by providing a large-scale dataset with comprehensive 3D shape and pose annotations for every person and object, covering 78 action categories and 14 interaction-specific body parts across 12 everyday scenarios. MMHOI is designed to foster research in multi-human multi-object interactions and serves as a benchmark for next-generation HOI algorithms. All necessary data for training, validation, and testing are publicly released.

At a Glance

Unzipped Size: 173 GB

The unzipped folder contains a README.md that describes the contents of each subfolder.

Contents:

20240418_personM_personC_from4C_all_30skip_start-end: 13G

20240417_personM_personC_to3C_finish: 1.7G

20240418_personB_personC_noC1andC2_all_30_skip_start-end: 3.1G

20240412_personA_personC: 19G

20240508_personJ_personK_30_skip_start-end: 16G

20240423_personB_personC_and_C2_all__start-end: 3.3G

20240417_personM_personB_finish: 4.6G

20240508_personE_personD_all_30skip_start-end: 17G

Single20240531_personA_all_30skip_start-end: 4.2G

20240412_personA_personB: 24G

20240426_personL_personC_all_30skip_start-end: 17G

20240424_personB_personD_all_start-end: 13G

20240510_personH_personI: 16G

20240426_personB_personG_all_30skip_start-end: 7.0G

20240423_personM_personC_30_skip_all_start-end: 17G

Other Resources

The code associated with the approach will be released separately on GitHub.

Citation

If you use the MMHOI dataset in your research, please cite our contribution:

@inproceedings{cherian2021insegan,
title={MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions},
author={K. Kogashi, A. Cherian, and M-Y. J. Kuo},
booktitle={Proceedings of the Winter Conference on Applications of Computer Vision},
year={2026}
}

License

The MMHOI dataset is released under CC-BY-SA-4.0 license.

All data:

Created by Mitsubishi Electric Research Laboratories (MERL), 2025-2026

SPDX-License-Identifier: CC-BY-SA-4.0