jhu-clsp/SciTaRC

---
homepage: https://github.com/JHU-CLSP/SciTaRC
repository: https://github.com/JHU-CLSP/SciTaRC
paper_url: https://arxiv.org/abs/[Your_ArXiv_ID]
language:
- en
license: cc-by-nc-4.0
task_categories:
- question-answering
- table-question-answering
tags:
- scientific-reasoning
- tabular-data
- complex-reasoning
- algorithmic-reasoning
- math
pretty_name: SciTaRC
size_categories:
- n<1K
configs:
- config_name: default
data_files:
- split: test
path: data/test-*
dataset_info:
features:
- name: paper
dtype: string
- name: relevant_tables
list:
list: string
- name: tables
list:
list: string
- name: fulltext
dtype: string
- name: question
dtype: string
- name: answer
dtype: string
- name: plan
dtype: string
splits:
- name: test
num_bytes: 48991529
num_examples: 371
download_size: 13748575
dataset_size: 48991529
---

# Dataset Card for SciTaRC

## Dataset Description

- **Paper:** [SciTaRC: Benchmarking QA on Scientific Tabular Data that Requires Language Reasoning and Complex Computation](https://arxiv.org/abs/[Your_ArXiv_ID])
- **Repository:** [JHU-CLSP/SciTaRC](https://github.com/JHU-CLSP/SciTaRC)

### Dataset Summary

**SciTaRC** (Scientific Table Reasoning and Computation) is an expert-authored benchmark designed to evaluate Large Language Models (LLMs) on complex question-answering tasks over real-world scientific tables.

Unlike existing benchmarks that focus on simple table-text integration or single-step operations, SciTaRC focuses on **composite reasoning**—requiring models to execute interdependent operations such as descriptive analysis, complex arithmetic, and ranking across detailed scientific tables. To facilitate granular diagnosis of model failures, every instance includes an expert-annotated **pseudo-code plan** that explicitly outlines the algorithmic reasoning steps required to reach the correct answer.

## Dataset Structure

The dataset is provided as a single `test` split containing 371 expert-annotated instances.

### Data Instances
A typical instance contains the question, the ground truth answer, the expert-authored pseudo-code plan, the LaTeX representations of the relevant tables, and the full text of the source paper.

### Data Fields
Each JSON object in the dataset contains the following fields:

- `paper` *(string)*: The arXiv ID of the source scientific paper (e.g., `"2401.06769"`).
- `question` *(string)*: The complex, multi-step question asked about the tabular data.
- `answer` *(string)*: The ground-truth answer.
- `plan` *(string)*: The expert-authored pseudo-code blueprint outlining logical and mathematical operations (e.g., `SELECT`, `LOOP`, `COMPUTE`).
- `relevant_tables` *(list of lists of strings)*: The exact LaTeX source code for the specific table(s) required to answer the question.
- `tables` *(list of lists of strings)*: The LaTeX source code for all tables and figures extracted from the paper.
- `fulltext` *(string)*: The complete LaTeX source text of the original scientific paper, providing full context.

## Citation

If you use this dataset, please cite our paper: