Resources

Further reading on peer review research in ACL venues

On this page, you will find some resources related to collection of, and research on, peer review data in the ACL community. The initiative was pioneered by Gurevych, Kuznetsov and Dycke from UKP Lab, TU Darmstadt.

Description	Link
What can NLP do for peer review? (Kuznetsov et al. 2024) is a whitepaper originating from the Dagstuhl seminar on peer review. It includes a roadmap for future research, and an extensive literature list that is continually updated on Github.	Find the paper here
Donation-based Peer Review Data Collection at ARR (Dycke et al. 2023) is a blogpost describing the importance of peer review data collection and how people can contribute.	Read the blogpost
NLPeer: A Unified Resource for the Computational Study of Peer Review (Dycke et al. 2023) is the first ethically sourced, publically available multi-domain corpus of papers plus peer reviews from five different venues.	Find the paper here
Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond (Dycke et al. 2022) describes the workflow of peer review data collection as implemented in ARR.	Find the paper here
The original paper by Gurevych and Kuznetsov, proposing an initiative on collecting and publishing anonymized numerical data from the ACL conferences.	Get the PDF