VideoFeedback2

[📃Paper](https://www.arxiv.org/abs/2509.22799) | [🌐Website](https://tiger-ai-lab.github.io/VideoScore2/) | [💻Code](https://github.com/TIGER-AI-Lab/VideoScore2) | [🛢️Dataset (VideoFeedback2)](https://huggingface.co/datasets/TIGER-Lab/VideoFeedback2) | [🤗Model (VideoScore2)](https://huggingface.co/TIGER-Lab/VideoScore2) | [🤗Space (VideoScore2)](https://huggingface.co/spaces/TIGER-Lab/VideoScore2) | [🤗50K videos cache](https://huggingface.co/datasets/hexuan21/VideoScore2_video_cache) ## Overview VideoFeedback2 is a large-scale, human-annotated dataset designed for training and evaluating multi-dimensional video evaluator [🤗Model (VideoScore2)](https://huggingface.co/TIGER-Lab/VideoScore2). It contains 27,168 AI-generated videos paired with **fine-grained human feedback scores** and **reasoning traces** across three evaluation dimensions: (1) Visual Quality; (2) Text Alignment; (3) Physical/Common-sense consistency. Prompt Collection: 2,933 unique text-to-video prompts sourced from both VidProM and Koala-36M datasets, supplemented with manually curated prompts emphasizing multi-action, OCR-text, and camera motion scenarios. Prompts underwent rule-based and LLM-based filtering to remove incoherent or underspecified cases. Video Collection: Videos were collected from 22 text-to-video (T2V) models, including diffusion-based and transformer-based systems such as ModelScope, VideoCrafter2, StepVideo-T2V, and Kling-1.6. Each prompt was rendered by 10 randomly selected models spanning four quality tiers (Poor → Modern), producing a balanced distribution of resolutions (256×256–1980×982), frame rates (8–30 fps), and durations (1–6 s). Annotation and Post-processing: please refer to our [paper](https://www.arxiv.org/abs/2509.22799) (Section3 and Appendix A) for more details. For training, see [VideoScore2/training](https://github.com/TIGER-AI-Lab/VideoScore2/tree/main/training) for details. . For evaluation, see [VideoScore2/evaluation](https://github.com/TIGER-AI-Lab/VideoScore2/tree/main/eval) for details ## Citation ```bibtex @misc{he2025videoscore2thinkscoregenerative, title={VideoScore2: Think before You Score in Generative Video Evaluation}, author={Xuan He and Dongfu Jiang and Ping Nie and Minghao Liu and Zhengxuan Jiang and Mingyi Su and Wentao Ma and Junru Lin and Chun Ye and Yi Lu and Keming Wu and Benjamin Schneider and Quy Duc Do and Zhuofeng Li and Yiming Jia and Yuxuan Zhang and Guo Cheng and Haozhe Wang and Wangchunshu Zhou and Qunshu Lin and Yuanxing Zhang and Ge Zhang and Wenhao Huang and Wenhu Chen}, year={2025}, eprint={2509.22799}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2509.22799}, } ```[📃论文](https://www.arxiv.org/abs/2509.22799) | [🌐项目主页](https://tiger-ai-lab.github.io/VideoScore2/) | [💻代码仓库](https://github.com/TIGER-AI-Lab/VideoScore2) | [🛢️数据集（VideoFeedback2）](https://huggingface.co/datasets/TIGER-Lab/VideoFeedback2) | [🤗模型（VideoScore2）](https://huggingface.co/TIGER-Lab/VideoScore2) | [🤗在线演示空间（VideoScore2）](https://huggingface.co/spaces/TIGER-Lab/VideoScore2) | [🤗5万视频缓存集](https://huggingface.co/datasets/hexuan21/VideoScore2_video_cache) ## 概述 VideoFeedback2是一个大规模人工标注数据集，专为训练与评估多维视频评估模型**VideoScore2**而构建。该数据集包含27168条AI生成视频，每条视频均配有**细粒度人类反馈评分**与**推理轨迹**，评估维度涵盖三大类：(1) 视觉质量；(2) 文本对齐度；(3) 物理/常识一致性。 ### 提示词采集 共收集2933条独特的文本到视频（text-to-video, T2V）提示词，数据来源包括VidProM与Koala-36M数据集，并补充了人工筛选的、侧重多动作、OCR文本与镜头运动场景的提示词。我们基于规则与大语言模型（Large Language Model, LLM）对提示词进行了过滤，移除了语义不通或描述模糊的样本。 ### 视频采集 视频来自22个文本到视频（T2V）模型，涵盖基于扩散模型与Transformer架构的系统，例如ModelScope、VideoCrafter2、StepVideo-T2V以及Kling-1.6。每条提示词均由10个随机选取的模型生成，这些模型覆盖了从"较差"到"现代级"的四个质量层级，最终生成的视频在分辨率（256×256~1980×982）、帧率（8~30 fps）与时长（1~6秒）上分布均衡。 ### 标注与后处理 标注与后处理流程的详细信息，请参阅我们的[论文](https://www.arxiv.org/abs/2509.22799)（第3节与附录A）。 训练相关细节请参阅[VideoScore2/training](https://github.com/TIGER-AI-Lab/VideoScore2/tree/main/training)；评估相关细节请参阅[VideoScore2/evaluation](https://github.com/TIGER-AI-Lab/VideoScore2/tree/main/eval)。 ## 引用格式 bibtex @misc{he2025videoscore2thinkscoregenerative, title={VideoScore2: Think before You Score in Generative Video Evaluation}, author={Xuan He and Dongfu Jiang and Ping Nie and Minghao Liu and Zhengxuan Jiang and Mingyi Su and Wentao Ma and Junru Lin and Chun Ye and Yi Lu and Keming Wu and Benjamin Schneider and Quy Duc Do and Zhuofeng Li and Yiming Jia and Yuxuan Zhang and Guo Cheng and Haozhe Wang and Wangchunshu Zhou and Qunshu Lin and Yuanxing Zhang and Ge Zhang and Wenhao Huang and Wenhu Chen}, year={2025}, eprint={2509.22799}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2509.22799}, }

搜集汇总

背景与挑战

背景概述

VideoFeedback2是一个大规模人工标注数据集，包含27,168个AI生成视频，每个视频都配有细粒度的人类反馈分数和推理轨迹，覆盖视觉质量、文本对齐和物理常识一致性三个评估维度。该数据集基于2,933个独特提示和22个文本到视频模型生成，旨在训练和评估多维视频评估模型VideoScore2。