umd-zhou-lab/TSRBench

--- language: - en task_categories: - question-answering size_categories: - 1K<n<10K dataset_info: features: - name: question dtype: string - name: answer dtype: string - name: domain dtype: string - name: name_of_series sequence: string - name: timeseries sequence: sequence: float64 - name: type dtype: string - name: task dtype: string - name: choices sequence: string splits: - name: Math_Reasoning num_bytes: 875777 num_examples: 550 download_size: 172660 dataset_size: 875777 configs: - config_name: default data_files: - split: Math_Reasoning path: data/Math_Reasoning-* --- # TSRBench [**📖 Paper**](https://huggingface.co/papers/2601.18744) | [**🏠 Project Page**](https://tsrbench.github.io/) | [**💻 GitHub**](https://github.com/tianyi-lab/TSRBench) TSRBench is a large-scale, comprehensive benchmark designed to stress-test the time series understanding and reasoning capabilities of generalist models (LLMs, VLMs, and TSLLMs). Time series data pervades real-world environments and underpins decision-making in high-stakes domains like finance, healthcare, and industrial systems. However, existing benchmarks often treat time series as isolated numerical sequences, stripping away the semantic context essential for complex problem-solving, or focusing solely on surface-level pattern recognition. TSRBench is more than a benchmark—it’s a multifaceted, standardized evaluation platform that not only uncovers the current challenges in time series reasoning but also provides actionable insights to push the boundaries of time series reasoning. It provides: - **More than 4,000 timeseries-text questions** covering diverse scenarios and practical challenges for generalist models. - **4 categories and 15 tasks** for various time series reasoning capabilities evaluation, including Perception (Perception, Reasoning, Prediction, Decision-Making), Reasoning (Abductive Reasoning, Numerical Reasoning, Deductive Reasoning, and more), Prediction (Time Series Forecasting, Event Prediction), and Decision-Making (Qualitative Decision-Making, Quantitative Decision-Making). ## 📃 Usage The data can be loaded via the Hugging Face `datasets` library: ```python from datasets import load_dataset tsrbench = load_dataset("umd-zhou-lab/TSRBench") ``` ## 📂 Dataset Description The dataset contains the following fields: | Field Name | Description | |------------|-----------------------------------------------------------------------------| | question | The natural language question associated with the task, describing what the model is required to answer or infer from the given time series. | | answer | The ground-truth answer to the question. Depending on the task, this can be a categorical label, a numerical value, or a textual response. | | domain |The application domain of the time series (e.g., finance, weather, healthcare), indicating the contextual background of the data. | | name_of_series | A sequence of names identifying each time series included in the sample. | | timeseries | The time series data represented as a nested sequence of floating-point values. Each inner sequence corresponds to the values of one time series over time. | | choices | A list of candidate answer options provided for multiple-choice questions. This field is empty for open-ended tasks. | ## Citation If you find TSRBench useful for your research, please cite: ```bibtex @article{yu2025tsrbench, title={TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models}, author={Fangxu Yu and Xingang Guo and Lingzhi Yuan and Haoqiang Kang and Hongyu Zhao and Lianhui Qin and Furong Huang and Bin Hu and Tianyi Zhou}, journal={arXiv preprint arXiv:2601.18744}, year={2025} } ```TSRBench is a large-scale, comprehensive benchmark designed to stress-test the time series understanding and reasoning capabilities of generalist models (LLMs, VLMs, and TSLLMs). Time series data pervades real-world environments and underpins decision-making in high-stakes domains like finance, healthcare, and industrial systems. However, existing benchmarks often treat time series as isolated numerical sequences, stripping away the semantic context essential for complex problem-solving, or focusing solely on surface-level pattern recognition. TSRBench is more than a benchmark—it’s a multifaceted, standardized evaluation platform that not only uncovers the current challenges in time series reasoning but also provides actionable insights to push the boundaries of time series reasoning. It provides more than 4,000 timeseries-text questions covering diverse scenarios and practical challenges for generalist models, and 4 categories and 15 tasks for various time series reasoning capabilities evaluation, including Perception (Perception, Reasoning, Prediction, Decision-Making), Reasoning (Abductive Reasoning, Numerical Reasoning, Deductive Reasoning, and more), Prediction (Time Series Forecasting, Event Prediction), and Decision-Making (Qualitative Decision-Making, Quantitative Decision-Making). The dataset fields include question, answer, domain, name_of_series, timeseries, and choices.