AssetOpsBench

# AssetOpsBench **AssetOpsBench** is a specialized benchmark designed for evaluating Large Language Models (LLMs) and Multi-Agent systems in industrial operations. It focuses on the intersection of sensor data interpretation, maintenance logic, and **Prognostics and Health Management (PHM)**. The benchmark enables researchers to test how effectively AI agents can manage complex industrial assets, such as compressors and hydraulic pumps, by applying rule-based logic and diagnostic reasoning. ## 📂 Dataset Structure The dataset is divided into several configurations to allow for granular testing. Users can load data for a specific **Asset** type or **Task** type. ### Baseline Configurations (Data Center Infrastructure) This core set focuses on critical cooling systems within data center environments: * **Asset Coverage**: Includes data from 4 Chillers and 2 Air-Handling Units (AHUs). * **Lifecycle Tasks**: Benchmarks a model's ability to perform Anomaly Detection, Automated Sensor Mapping, and Work Order Generation. ### Asset Configurations Focus on hardware-specific sensor patterns and operational contexts: * **Compressor:** Data related to industrial air and gas compressors. * **Hydrolic Pump:** Data focusing on fluid power systems and pressure diagnostics. ### Task Configurations Focus on the reasoning and automation capabilities: * **PHM (Prognostics and Health Management):** Tasks centered on predicting Remaining Useful Life (RUL) and assessing State of Health (SoH). * **Rule Logic:** Evaluating the model's ability to trigger actions based on predefined industrial maintenance thresholds and logic. ## 🚀 Getting Started You can load the default scenario or a specific configuration using the Hugging Face `datasets` library. ### Loading the Default Scenarios ```python from datasets import load_dataset dataset = load_dataset("ibm-research/AssetOpsBench", "scenarios") ``` ### Loading a Specific Asset (e.g., Compressor) ```python from datasets import load_dataset dataset = load_dataset("ibm-research/AssetOpsBench", "compressor") ``` ## Cite this Dataset If you use our dataset in your paper, please cite our dataset by ``` @misc{patel2025assetopsbenchbenchmarkingaiagents, title={AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance}, author={Dhaval Patel and Shuxin Lin and James Rayfield and Nianjun Zhou and Roman Vaculin and Natalia Martinez and Fearghal O'donncha and Jayant Kalagnanam}, year={2025}, eprint={2506.03828}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2506.03828}, } ```# AssetOpsBench 本数据集包含以下内容： - 场景集（Scenarios）：由人工撰写的工业资产智能体（AI Agent）评估提示词。 本数据集用于AssetOpsBench基准测试：https://github.com/ibm-research/AssetOpsBench ## 数据集引用 若您在论文中使用本数据集，请按以下格式引用： @misc{patel2025assetopsbenchbenchmarkingaiagents, title={AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance}, author={Dhaval Patel and Shuxin Lin and James Rayfield and Nianjun Zhou and Roman Vaculin and Natalia Martinez and Fearghal O'donncha and Jayant Kalagnanam}, year={2025}, eprint={2506.03828}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2506.03828}, }

搜集汇总

背景与挑战

背景概述

AssetOpsBench是一个专门用于评估大语言模型和多智能体系统在工业运营中表现的基准数据集，重点关注传感器数据解释、维护逻辑与预测健康管理的交叉领域。它通过针对特定资产和任务的配置，测试AI代理在管理复杂工业设备（如压缩机和液压泵）时应用规则逻辑和诊断推理的能力。