aslessor/MMMU

Name: aslessor/MMMU
Creator: aslessor
Published: 2024-01-06 17:28:39
License: 暂无描述

Hugging Face2024-01-06 更新2024-03-04 收录

下载链接：

https://hf-mirror.com/datasets/aslessor/MMMU

下载链接

链接失效反馈

官方服务：

资源简介：

--- language: - en license: apache-2.0 size_categories: - 10K<n<100K task_categories: - question-answering - visual-question-answering - multiple-choice pretty_name: mmmu dataset_info: - config_name: Accounting features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 262599.0 num_examples: 5 - name: validation num_bytes: 1598285.0 num_examples: 30 - name: test num_bytes: 22135625.0 num_examples: 380 download_size: 37363379 dataset_size: 23996509.0 - config_name: Agriculture features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 22082656.0 num_examples: 5 - name: validation num_bytes: 119217558.0 num_examples: 30 - name: test num_bytes: 993664077.0 num_examples: 287 download_size: 1158036990 dataset_size: 1134964291.0 - config_name: Architecture_and_Engineering features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 137750.0 num_examples: 5 - name: validation num_bytes: 721378.0 num_examples: 30 - name: test num_bytes: 16054607.0 num_examples: 551 download_size: 48763955 dataset_size: 16913735.0 - config_name: Art features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 6241184.0 num_examples: 5 - name: validation num_bytes: 29934534.0 num_examples: 30 - name: test num_bytes: 237801390.0 num_examples: 231 download_size: 585798641 dataset_size: 273977108.0 - config_name: Art_Theory features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 7435106.0 num_examples: 5 - name: validation num_bytes: 33481558.0 num_examples: 30 - name: test num_bytes: 553174647.0 num_examples: 429 download_size: 930525695 dataset_size: 594091311.0 - config_name: Basic_Medical_Science features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 814310.0 num_examples: 5 - name: validation num_bytes: 4125930.0 num_examples: 30 - name: test num_bytes: 48125891.0 num_examples: 326 download_size: 84666454 dataset_size: 53066131.0 - config_name: Biology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 574342.0 num_examples: 5 - name: validation num_bytes: 8491863.0 num_examples: 30 - name: test num_bytes: 132966151.0 num_examples: 345 download_size: 410242502 dataset_size: 142032356.0 - config_name: Chemistry features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 262397.0 num_examples: 5 - name: validation num_bytes: 1518573.0 num_examples: 30 - name: test num_bytes: 37219529.0 num_examples: 603 download_size: 108345562 dataset_size: 39000499.0 - config_name: Clinical_Medicine features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1467945.0 num_examples: 5 - name: validation num_bytes: 10882484.0 num_examples: 30 - name: test num_bytes: 98201863.0 num_examples: 325 download_size: 160611488 dataset_size: 110552292.0 - config_name: Computer_Science features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 440523.0 num_examples: 5 - name: validation num_bytes: 2072018.0 num_examples: 30 - name: test num_bytes: 32047381.0 num_examples: 371 download_size: 55640991 dataset_size: 34559922.0 - config_name: Design features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2259873.0 num_examples: 5 - name: validation num_bytes: 17923120.0 num_examples: 30 - name: test num_bytes: 77676331.0 num_examples: 169 download_size: 142866617 dataset_size: 97859324.0 - config_name: Diagnostics_and_Laboratory_Medicine features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2056117.0 num_examples: 5 - name: validation num_bytes: 37106233.0 num_examples: 30 - name: test num_bytes: 157003069.0 num_examples: 162 download_size: 603957093 dataset_size: 196165419.0 - config_name: Economics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 171434.0 num_examples: 5 - name: validation num_bytes: 1487048.0 num_examples: 30 - name: test num_bytes: 11852300.0 num_examples: 267 download_size: 20777635 dataset_size: 13510782.0 - config_name: Electronics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 123632.0 num_examples: 5 - name: validation num_bytes: 641377.0 num_examples: 30 - name: test num_bytes: 5717686.0 num_examples: 256 download_size: 11602832 dataset_size: 6482695.0 - config_name: Energy_and_Power features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 105006.0 num_examples: 5 - name: validation num_bytes: 1641935.0 num_examples: 30 - name: test num_bytes: 14748428.0 num_examples: 432 download_size: 35246567 dataset_size: 16495369.0 - config_name: Finance features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 296124.0 num_examples: 5 - name: validation num_bytes: 1071060.0 num_examples: 30 - name: test num_bytes: 12065803.0 num_examples: 355 download_size: 29551521 dataset_size: 13432987.0 - config_name: Geography features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1494060.0 num_examples: 5 - name: validation num_bytes: 6671316.0 num_examples: 30 - name: test num_bytes: 137218400.0 num_examples: 565 download_size: 374766631 dataset_size: 145383776.0 - config_name: History features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1444231.0 num_examples: 5 - name: validation num_bytes: 8819857.0 num_examples: 30 - name: test num_bytes: 115228815.0 num_examples: 278 download_size: 232549641 dataset_size: 125492903.0 - config_name: Literature features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2451201.0 num_examples: 5 - name: validation num_bytes: 14241046.0 num_examples: 30 - name: test num_bytes: 50301541.0 num_examples: 112 download_size: 132145895 dataset_size: 66993788.0 - config_name: Manage features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 449514.0 num_examples: 5 - name: validation num_bytes: 3277436.0 num_examples: 30 - name: test num_bytes: 29963963.0 num_examples: 245 download_size: 51186888 dataset_size: 33690913.0 - config_name: Marketing features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 116960.0 num_examples: 5 - name: validation num_bytes: 1472981.0 num_examples: 30 - name: test num_bytes: 7732976.0 num_examples: 181 download_size: 13146078 dataset_size: 9322917.0 - config_name: Materials features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 239632.0 num_examples: 5 - name: validation num_bytes: 2305211.0 num_examples: 30 - name: test num_bytes: 25256854.0 num_examples: 458 download_size: 78365794 dataset_size: 27801697.0 - config_name: Math features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 175861.0 num_examples: 5 - name: validation num_bytes: 1444458.0 num_examples: 30 - name: test num_bytes: 27701878.0 num_examples: 505 download_size: 89368153 dataset_size: 29322197.0 - config_name: Mechanical_Engineering features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 152542.0 num_examples: 5 - name: validation num_bytes: 874988.0 num_examples: 30 - name: test num_bytes: 15093746.0 num_examples: 429 download_size: 30450114 dataset_size: 16121276.0 - config_name: Music features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1417615.0 num_examples: 5 - name: validation num_bytes: 9359372.0 num_examples: 30 - name: test num_bytes: 134096770.0 num_examples: 334 download_size: 174725052 dataset_size: 144873757.0 - config_name: Pharmacy features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 207924.0 num_examples: 5 - name: validation num_bytes: 1656342.0 num_examples: 30 - name: test num_bytes: 31866248.0 num_examples: 430 download_size: 62721263 dataset_size: 33730514.0 - config_name: Physics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 233734.0 num_examples: 5 - name: validation num_bytes: 1114130.0 num_examples: 30 - name: test num_bytes: 15905705.0 num_examples: 408 download_size: 35238571 dataset_size: 17253569.0 - config_name: Psychology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 600864.0 num_examples: 5 - name: validation num_bytes: 4403886.0 num_examples: 30 - name: test num_bytes: 53813915.0 num_examples: 305 download_size: 102466671 dataset_size: 58818665.0 - config_name: Public_Health features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 234781.0 num_examples: 5 - name: validation num_bytes: 1508761.0 num_examples: 30 - name: test num_bytes: 32150088.0 num_examples: 509 download_size: 48231609 dataset_size: 33893630.0 - config_name: Sociology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 3769220.0 num_examples: 5 - name: validation num_bytes: 18455336.0 num_examples: 30 - name: test num_bytes: 144301123.0 num_examples: 252 download_size: 310313826 dataset_size: 166525679.0 configs: - config_name: Accounting data_files: - split: dev path: Accounting/dev-* - split: validation path: Accounting/validation-* - split: test path: Accounting/test-* - config_name: Agriculture data_files: - split: dev path: Agriculture/dev-* - split: validation path: Agriculture/validation-* - split: test path: Agriculture/test-* - config_name: Architecture_and_Engineering data_files: - split: dev path: Architecture_and_Engineering/dev-* - split: validation path: Architecture_and_Engineering/validation-* - split: test path: Architecture_and_Engineering/test-* - config_name: Art data_files: - split: dev path: Art/dev-* - split: validation path: Art/validation-* - split: test path: Art/test-* - config_name: Art_Theory data_files: - split: dev path: Art_Theory/dev-* - split: validation path: Art_Theory/validation-* - split: test path: Art_Theory/test-* - config_name: Basic_Medical_Science data_files: - split: dev path: Basic_Medical_Science/dev-* - split: validation path: Basic_Medical_Science/validation-* - split: test path: Basic_Medical_Science/test-* - config_name: Biology data_files: - split: dev path: Biology/dev-* - split: validation path: Biology/validation-* - split: test path: Biology/test-* - config_name: Chemistry data_files: - split: dev path: Chemistry/dev-* - split: validation path: Chemistry/validation-* - split: test path: Chemistry/test-* - config_name: Clinical_Medicine data_files: - split: dev path: Clinical_Medicine/dev-* - split: validation path: Clinical_Medicine/validation-* - split: test path: Clinical_Medicine/test-* - config_name: Computer_Science data_files: - split: dev path: Computer_Science/dev-* - split: validation path: Computer_Science/validation-* - split: test path: Computer_Science/test-* - config_name: Design data_files: - split: dev path: Design/dev-* - split: validation path: Design/validation-* - split: test path: Design/test-* - config_name: Diagnostics_and_Laboratory_Medicine data_files: - split: dev path: Diagnostics_and_Laboratory_Medicine/dev-* - split: validation path: Diagnostics_and_Laboratory_Medicine/validation-* - split: test path: Diagnostics_and_Laboratory_Medicine/test-* - config_name: Economics data_files: - split: dev path: Economics/dev-* - split: validation path: Economics/validation-* - split: test path: Economics/test-* - config_name: Electronics data_files: - split: dev path: Electronics/dev-* - split: validation path: Electronics/validation-* - split: test path: Electronics/test-* - config_name: Energy_and_Power data_files: - split: dev path: Energy_and_Power/dev-* - split: validation path: Energy_and_Power/validation-* - split: test path: Energy_and_Power/test-* - config_name: Finance data_files: - split: dev path: Finance/dev-* - split: validation path: Finance/validation-* - split: test path: Finance/test-* - config_name: Geography data_files: - split: dev path: Geography/dev-* - split: validation path: Geography/validation-* - split: test path: Geography/test-* - config_name: History data_files: - split: dev path: History/dev-* - split: validation path: History/validation-* - split: test path: History/test-* - config_name: Literature data_files: - split: dev path: Literature/dev-* - split: validation path: Literature/validation-* - split: test path: Literature/test-* - config_name: Manage data_files: - split: dev path: Manage/dev-* - split: validation path: Manage/validation-* - split: test path: Manage/test-* - config_name: Marketing data_files: - split: dev path: Marketing/dev-* - split: validation path: Marketing/validation-* - split: test path: Marketing/test-* - config_name: Materials data_files: - split: dev path: Materials/dev-* - split: validation path: Materials/validation-* - split: test path: Materials/test-* - config_name: Math data_files: - split: dev path: Math/dev-* - split: validation path: Math/validation-* - split: test path: Math/test-* - config_name: Mechanical_Engineering data_files: - split: dev path: Mechanical_Engineering/dev-* - split: validation path: Mechanical_Engineering/validation-* - split: test path: Mechanical_Engineering/test-* - config_name: Music data_files: - split: dev path: Music/dev-* - split: validation path: Music/validation-* - split: test path: Music/test-* - config_name: Pharmacy data_files: - split: dev path: Pharmacy/dev-* - split: validation path: Pharmacy/validation-* - split: test path: Pharmacy/test-* - config_name: Physics data_files: - split: dev path: Physics/dev-* - split: validation path: Physics/validation-* - split: test path: Physics/test-* - config_name: Psychology data_files: - split: dev path: Psychology/dev-* - split: validation path: Psychology/validation-* - split: test path: Psychology/test-* - config_name: Public_Health data_files: - split: dev path: Public_Health/dev-* - split: validation path: Public_Health/validation-* - split: test path: Public_Health/test-* - config_name: Sociology data_files: - split: dev path: Sociology/dev-* - split: validation path: Sociology/validation-* - split: test path: Sociology/test-* tags: - biology - medical - finance - chemistry - music - art - art_theory - design - music - business - accounting - economics - finance - manage - marketing - health - medicine - basic_medical_science - clinical - pharmacy - public_health - humanities - social_science - history - literature - sociology - psychology - science - biology - chemistry - geography - math - physics - engineering - agriculture - architecture - computer_science - electronics - energy_and_power - materials - mechanical_engineering --- # MMMU (A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI) [**🌐 Homepage**](https://mmmu-benchmark.github.io/) | [**🤗 Dataset**](https://huggingface.co/datasets/MMMU/MMMU/) | [**🤗 Paper**](https://huggingface.co/papers/2311.16502) | [**📖 arXiv**](https://arxiv.org/abs/2311.16502) | [**GitHub**](https://github.com/MMMU-Benchmark/MMMU) ## 🔔News - **🔥[2023-12-04]: Our evaluation server for test set is now availble on [EvalAI](https://eval.ai/web/challenges/challenge-page/2179/overview). We welcome all submissions and look forward to your participation! 😆** ## Dataset Details ### Dataset Description We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes **11.5K meticulously collected multimodal questions** from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span **30 subjects** and **183 subfields**, comprising **30 highly heterogeneous image types**, such as charts, diagrams, maps, tables, music sheets, and chemical structures. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence (AGI). 🎯 **We have released a full set comprising 150 development samples and 900 validation samples. We have released 10,500 test questions without their answers.** The development set is used for few-shot/in-context learning, and the validation set is used for debugging models, selecting hyperparameters, or quick evaluations. The answers and explanations for the test set questions are withheld. You can submit your model's predictions for the **test set** on **[EvalAI](https://eval.ai/web/challenges/challenge-page/2179/overview)**. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6230d750d93e84e233882dbc/2Ulh9yznm1dvISV4xJ_Ok.png) ### Dataset Creation MMMU was created to challenge multimodal models with tasks that demand college-level subject knowledge and deliberate reasoning, pushing the boundaries of what these models can achieve in terms of expert-level perception and reasoning. The data for the MMMU dataset was manually collected by a team of college students from various disciplines, using online sources, textbooks, and lecture materials. - **Content:** The dataset contains 11.5K college-level problems across six broad disciplines (Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, Tech & Engineering) and 30 college subjects. - **Image Types:** The dataset includes 30 highly heterogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures, interleaved with text. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6230d750d93e84e233882dbc/Mbf8O5lEH8I8czprch0AG.png) ## 🏆 Mini-Leaderboard We show a mini-leaderboard here and please find more information in our paper or [**homepage**](https://mmmu-benchmark.github.io/). | Model | Val (900) | Test (10.5K) | |----------------------------|:---------:|:------------:| | Gemini Ultra* | **59.4** | - | | GPT-4V(ision) (Playground) | 56.8 | **55.7** | | Gemini Pro* | 47.9 | - | | Yi-VL-34B* | 45.9 | 41.6 | | Qwen-VL-PLUS* | 45.2 | 40.8 | | InfiMM-Zephyr-7B* | 39.4 | 35.5 | | SVIT* | 38.0 | 34.1 | | Emu2-Chat* | 36.3 | 34.1 | | BLIP-2 FLAN-T5-XXL | 35.4 | 34.0 | | InstructBLIP-T5-XXL | 35.7 | 33.8 | | LLaVA-1.5-13B | 36.4 | 33.6 | | Qwen-VL-7B | 35.9 | 32.9 | | mPLUG-OWL2* | 32.7 | 32.1 | | BLIP-2 FLAN-T5-XL | 34.4 | 31.0 | | InstructBLIP-T5-XL | 32.9 | 30.6 | | SPHINX* | 32.9 | 32.9 | | Gemini Nano2* | 32.6 | - | | CogVLM | 32.1 | 30.1 | | Otter | 32.2 | 29.1 | | LLaMA-Adapter2-7B | 29.8 | 27.7 | | MiniGPT4-Vicuna-13B | 26.8 | 27.6 | | Fuyu-8B | 27.9 | 27.4 | | Kosmos2 | 24.4 | 26.6 | | OpenFlamingo2-9B | 28.7 | 26.3 | | Frequent Choice | 22.1 | 23.9 | | Random Choice | 26.8 | 25.8 | *: results provided by the authors. ## Limitations Despite its comprehensive nature, MMMU, like any benchmark, is not without limitations. The manual curation process, albeit thorough, may carry biases. And the focus on college-level subjects might not fully be a sufficient test for Expert AGI. However, we believe it should be necessary for an Expert AGI to achieve strong performance on MMMU to demonstrate their broad and deep subject knowledge as well as expert-level understanding and reasoning capabilities. In future work, we plan to incorporate human evaluations into MMMU. This will provide a more grounded comparison between model capabilities and expert performance, shedding light on the proximity of current AI systems to achieving Expert AGI. ## Disclaimers The guidelines for the annotators emphasized strict compliance with copyright and licensing rules from the initial data source, specifically avoiding materials from websites that forbid copying and redistribution. Should you encounter any data samples potentially breaching the copyright or licensing regulations of any site, we encourage you to notify us. Upon verification, such samples will be promptly removed. ## Contact - Xiang Yue: xiangyue.work@gmail.com - Yu Su: su.809@osu.edu - Wenhu Chen: wenhuchen@uwaterloo.ca ## Citation **BibTeX:** ```bibtex @article{yue2023mmmu, title={MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI}, author={Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen}, journal={arXiv preprint arXiv:2311.16502}, year={2023}, } ```

提供机构：

aslessor

原始信息汇总

数据集概述

基本信息

语言: 英语
许可证: Apache 2.0
数据量: 10K < n < 100K
任务类别:
- 问答
- 视觉问答
- 多选题
数据集名称: mmmu

数据集配置

会计 (Accounting)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 262599字节
- validation: 30个样本, 1598285字节
- test: 380个样本, 22135625字节
下载大小: 37363379字节
数据集大小: 23996509字节

农业 (Agriculture)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 22082656字节
- validation: 30个样本, 119217558字节
- test: 287个样本, 993664077字节
下载大小: 1158036990字节
数据集大小: 1134964291字节

建筑与工程 (Architecture_and_Engineering)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 137750字节
- validation: 30个样本, 721378字节
- test: 551个样本, 16054607字节
下载大小: 48763955字节
数据集大小: 16913735字节

艺术 (Art)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 6241184字节
- validation: 30个样本, 29934534字节
- test: 231个样本, 237801390字节
下载大小: 585798641字节
数据集大小: 273977108字节

艺术理论 (Art_Theory)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 7435106字节
- validation: 30个样本, 33481558字节
- test: 429个样本, 553174647字节
下载大小: 930525695字节
数据集大小: 594091311字节

基础医学科学 (Basic_Medical_Science)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 814310字节
- validation: 30个样本, 4125930字节
- test: 326个样本, 48125891字节
下载大小: 84666454字节
数据集大小: 53066131字节

生物学 (Biology)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 574342字节
- validation: 30个样本, 8491863字节
- test: 345个样本, 132966151字节
下载大小: 410242502字节
数据集大小: 142032356字节

化学 (Chemistry)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 262397字节
- validation: 30个样本, 1518573字节
- test: 603个样本, 37219529字节
下载大小: 108345562字节
数据集大小: 39000499字节

临床医学 (Clinical_Medicine)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 1467945字节
- validation: 30个样本, 10882484字节
- test: 325个样本, 98201863字节
下载大小: 160611488字节
数据集大小: 110552292字节

计算机科学 (Computer_Science)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 440523字节
- validation: 30个样本, 2072018字节
- test: 371个样本, 32047381字节
下载大小: 55640991字节
数据集大小: 34559922字节

设计 (Design)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 2259873字节
- validation: 30个样本, 17923120字节
- test: 169个样本, 77676331字节
下载大小: 142866617字节
数据集大小: 97859324字节

诊断与实验室医学 (Diagnostics_and_Laboratory_Medicine)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 2056117字节
- validation: 30个样本, 37106233字节
- test: 162个样本, 157003069字节
下载大小: 603957093字节
数据集大小: 196165419字节

经济学 (Economics)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 171434字节
- validation: 30个样本, 1487048字节
- test: 267个样本, 11852300字节
下载大小: 20777635字节
数据集大小: 13510782字节

电子学 (Electronics)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 123632字节
- validation: 30个样本, 641377字节
- test: 256个样本, 5717686字节
下载大小: 11602832字节
数据集大小: 6482695字节

能源与电力 (Energy_and_Power)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 105006字节
- validation: 30个样本, 1641935字节
- test: 432个样本, 14748428字节
下载大小: 35246567字节
数据集大小: 16495369字节

金融 (Finance)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 296124字节
- validation: 30个样本, 1071060字节
- test: 355个样本, 12065803字节
下载大小: 29551521字节
数据集大小: 13432987字节

地理 (Geography)

特征:
- id: string
- question: string
- options: string
- explanation: string
- image_1: image
- image_2: image
- image_3: image
- image_4: image
- image_5: image
- image_6: image
- image_7: image
- img_type: string
- answer: string
- topic_difficulty: string
- question_type: string
- subfield: string
分割:
- dev: 5个样本, 1494060字节
- validation: 30个样本, 6671316字节
- test: 565个样本, 1372

搜集汇总

数据集介绍

构建方式

在人工智能与多模态理解领域，大规模基准测试的构建是推动模型能力演进的关键基石。MMMU数据集由aslessor团队精心打造，旨在评估模型在跨学科视觉问答中的表现。该数据集采用多学科分组的模块化设计，涵盖了从会计、农业到数学、医学等数十个专业领域，每个领域作为一个独立的配置（config），包含问题、选项、解释、最多七张关联图像以及答案等字段。数据被划分为开发集、验证集和测试集，其中开发集和验证集各含少量样本用于调试与调优，测试集则包含数百个样本用于最终评估，整体规模介于10K至100K之间，确保了评测的全面性与挑战性。

特点

MMMU数据集的核心特点在于其深度与广度的完美融合。它横跨数十个学科，从基础科学到应用工程，从人文艺术到临床医学，每个领域的问题均附有详细的解释和主题难度标记，便于研究者分析模型在不同知识层次上的表现。每条数据可关联多达七张图像，支持复杂的多模态场景理解，同时通过question_type和subfield字段实现精细化的分类。这种结构不仅考验模型的视觉感知与语言推理能力，还要求其具备跨领域知识迁移的智慧，为多模态大模型的鲁棒性评估提供了前所未有的丰富素材。

使用方法

研究者可通过HuggingFace平台便捷地加载MMMU数据集。使用datasets库的load_dataset函数，指定参数为'aslessor/MMMU'，并选择目标学科配置（如'Accounting'或'Biology'）即可获取对应子集。数据以id、question、options、explanation、image_1至image_7、answer等字段组织，开发与验证集可用于模型微调与超参数优化，测试集则专用于最终性能评测。推荐在加载时利用split参数区分开发、验证与测试划分，并结合img_type与topic_difficulty进行针对性分析，以全面评估模型在多样化场景下的多模态问答能力。

背景与挑战

背景概述

MMMU（Massive Multi-discipline Multimodal Understanding）数据集由研究团队构建，旨在为多模态大模型提供一项涵盖广泛学科的高难度评测基准。该数据集创建于2023年前后，核心研究问题聚焦于评估模型在跨学科视觉问答任务中的表现，涉及会计、农业、艺术、医学、计算机科学等数十个专业领域。通过整合图像与文本交织的复杂问题，MMMU填补了现有基准对多学科融合理解能力评估的空白，成为检验模型在学术级知识推理、图表解析及专业图像识别方面能力的重要标尺。其影响力迅速扩展到人工智能领域，为多模态模型的性能比较与进化提供了标准化的挑战环境。

当前挑战

MMMU数据集所解决的领域问题在于，现有视觉问答基准往往局限于常识或单一学科，无法衡量模型在真实学术场景中处理多学科、多图像复杂问题的能力。构建过程中面临的挑战包括：1）从大学教材、考试及学术资料中收集并审核覆盖30余个学科的高质量题目，确保每道题涉及1至7张图像，且图像类型涵盖图表、照片、示意图等多种形式；2）设计包含解释字段（explanation）的标注机制，以支持模型推理过程的分析，同时平衡各学科样本数量与难度分布；3）控制图像质量与文本对齐的精度，避免歧义或信息缺失影响评测的公平性。

常用场景

经典使用场景

MMMU（Massive Multi-discipline Multimodal Understanding）数据集作为大规模多学科多模态理解基准，其经典使用场景在于评估视觉语言模型在跨学科知识图谱中的综合推理能力。该数据集涵盖了从会计学到动物学等30余个学科领域，每个样本均包含文本问题与多张关联图像，要求模型在图像理解与文本推理的交织中完成多项选择作答。研究者常利用该数据集检验模型在图表解析、示意图识别、实物图像分类等视觉任务与学科知识问答的协同表现，从而系统性地衡量模型在高等教育水平上的多模态认知水平。

解决学术问题

MMMU数据集的核心学术价值在于解决了多模态大模型评估中缺乏标准化、多学科、高难度基准的痛点。此前，视觉问答数据集多聚焦于常识或单一领域，难以反映模型在真实学术场景中的综合素养。MMMU通过构建包含不同难度层级与问题类型（如跨学科推理、图表分析）的样本，为学术界提供了衡量模型知识广度与深度的统一标尺。这一基准的建立，使得研究人员能够客观对比不同架构模型在专业学科上的短板，推动了多模态理解向精细化、学科化方向演进。

衍生相关工作

MMMU数据集催生了一系列围绕多模态模型评估与优化的衍生工作。研究者基于该基准提出了多种改进策略，如引入思维链提示（Chain-of-Thought）增强模型的分步推理能力，或设计专用视觉编码器以提升对学科图表的解析精度。在模型层面，MMMU被用于微调LLaVA、Qwen-VL等主流视觉语言模型，并催生了针对特定学科（如医学、工程）的领域专用多模态模型。这些工作不仅验证了MMMU作为评估框架的鲁棒性，也推动了多模态预训练范式的迭代创新。

以上内容由遇见数据集搜集并总结生成