Optimization modeling and verification from problem specifications using a multi-agent multi-stage LLM framework

This paper explores the use of Large Language Models (LLMs) in modeling real-world optimization problems. We concretely define the task of translating natural language descriptions into optimization models (NL2OPT) and provide criteria for classifying optimization problems for the NL2OPT task. Our novel multi-agent modeling framework leverages relations identifier agents and a multi-agent verification mechanism, eliminating the need for solver execution. Additionally, we introduce a straightforward and practical evaluation framework, offering a more effective assessment method compared to traditional execution-based evaluations. We have created a unique dataset tailored for optimization modeling, featuring Problem Specifications as a structured representation of optimization problems. Through comprehensive experiments, our study compares our modeling framework with existing LLM reasoning strategies, highlighting their relative effectiveness in optimization modeling tasks. We also perform ablation studies to explore the effect of different components of our modeling framework. Experimental results demonstrate that our multi-agent framework outperforms many common LLM prompting strategies.本论文探究了大语言模型（Large Language Models，LLMs）在现实世界优化问题建模中的应用。我们明确界定了将自然语言描述转换为优化模型的任务（Natural Language to Optimization，NL2OPT），并制定了针对NL2OPT任务的优化问题分类标准。我们提出的新型多智能体建模框架依托关系识别智能体与多智能体验证机制，无需借助求解器执行即可完成建模。此外，我们还引入了简洁实用的评估框架，相较传统基于执行的评估方法，该框架的评估效果更为优异。我们构建了专为优化建模打造的专属数据集，以问题规范（Problem Specifications）作为优化问题的结构化表示形式。通过全面的实验，本研究将所提出的建模框架与现有大语言模型推理策略进行了对比，凸显了二者在优化建模任务中的相对性能表现。我们还开展了消融实验，以探究建模框架各不同组件的作用效果。实验结果表明，所提出的多智能体框架优于诸多主流大语言模型提示策略。