2605.29833v1 May 28, 2026 cs.AI

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

Qian Tan
Qian Tan
Citations: 214
h-index: 3
Lei Bai
Lei Bai
Citations: 75
h-index: 4
Weida Wang
Weida Wang
Shanghai AI Laboratory
Citations: 189
h-index: 9
Zhuo Yang
Zhuo Yang
Citations: 13
h-index: 2
Jiaqing Xie
Jiaqing Xie
Citations: 54
h-index: 4
Tianfan Fu
Tianfan Fu
Citations: 7
h-index: 1
Yuqiang Li
Yuqiang Li
Citations: 29
h-index: 4
Lu Chen
Lu Chen
Citations: 460
h-index: 12
Wanli Ouyang
Wanli Ouyang
Citations: 4,084
h-index: 20
Wanhao Liu
Wanhao Liu
Citations: 119
h-index: 5
Ran Sun
Ran Sun
Citations: 297
h-index: 4
Jue Wang
Jue Wang
Citations: 160
h-index: 2
Xin Chen
Xin Chen
Citations: 178
h-index: 7

As multimodal language models play an increasingly important role in scientific research, materials science offers a critical testbed due to its interdisciplinary, multimodal, and application-driven nature. However, existing materials benchmarks mainly focus on property prediction, knowledge QA, or characterization understanding, leaving the broader reasoning process from materials knowledge to application underexplored. To fill this gap, we present OmniMatBench, a human-calibrated multimodal reasoning benchmark for materials science. OmniMatBench contains 3,171 expert-curated QA and calculation problems across 19 materials-science subfields, spanning fundamental materials knowledge, structural and engineering materials, materials processing and manufacturing, and functional and applied materials. We evaluate 13 open-source and closed-source MLLMs and find that the best model achieves only a 0.372 overall score, revealing a substantial gap in current materials-science reasoning. Further analysis shows strong variation across subfields, fixed reasoning heuristics, uneven materials knowledge, and limited high-level knowledge application under formula-, retrieval-, and code-assisted settings. OmniMatBench provides crucial insights into the capabilities and limitations of current MLLMs and establishes a foundation for reliable AI assistants in materials-science research.

0 Citations
0 Influential
10 Altmetric
50.0 Score
Original PDF

No Analysis Report Yet

This paper hasn't been analyzed by Gemini yet.

Log in to request an AI analysis.

댓글

댓글을 작성하려면 로그인하세요.

아직 댓글이 없습니다. 첫 번째 댓글을 남겨보세요!