Dataset and Statistical Outputs for Structured Peer Review: Pilot results from 23 Elsevier Journals

Structured peer review consisting of 9 questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that in the first 2 months of the pilot received 2 reviewer reports. It was the goal of this pilot study to evaluate structured peer review implementation by: 1) exploring if and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions. As categories for outcome classification were developed throughout the data extraction process, the data file was constantly being updated. The final, cleaned dataset, is posted here alongside all statistical outputs of results. The original raw data, which contains proprietary Elsevier data, is available upon request from BM and will require a non-disclosure agreement.