CARP

书面故事与批评相结合是偏好学习的良好数据来源。批评可以作为一种非常丰富的信息度量，通过它来衡量故事内容的偏好。通过对比轶事评论预训练 (简称CARP)，我们介绍了通过/批评对的故事批评数据集 (即 “鹅比鸭子好”，“应该比不” 作为这种对的例子)，以及鲤鱼模型。CARP在故事批评数据集上进行了培训，以产生段落和评论的嵌入，其中段落和评论嵌入之间的高度相似性通常对应于适合给定段落的评论。通过衡量评论与故事的匹配程度，我们可以衡量故事在某种偏好下的表现。模型和检查点是公开可用的，数据集可以通过请求共享Combining written stories with critiques serves as an excellent data source for preference learning. Critiques can act as a highly informative metric for measuring preferences regarding story content. Introduced via Contrastive Anecdotal Review Pre-training (shortened as CARP), we present the Story Critique dataset consisting of story-critique pairs (e.g., "Geese are better than ducks" and "Should rather than not" as examples of such pairs) along with the CARP model. Trained on the Story Critique dataset, CARP generates embeddings for both story passages and critiques, where high similarity between the passage and critique embeddings typically indicates that the critique is suitable for the given passage. By measuring the degree of match between critiques and stories, we can evaluate how well a story performs under specific preferences. Both the model and its checkpoints are publicly available, and the dataset can be shared upon request.

搜集汇总

背景与挑战

背景概述

CARP是一个故事批评数据集，通过结合故事与批评对来支持偏好学习，训练模型生成嵌入以评估评论与故事的匹配程度。该数据集由EleutherAI于2021年发布，模型和检查点公开可用。