Fall 2019

Tue, Sept 24th 2019, 1:30 PM, Venue: Panther Hollow, CIC (4101)
Ivan Stelmakh, Carnegie Mellon University

Title: PeerReview4All: improving fairness of peer review

Abstract: Peer review is the backbone of scholarly research and fairness of this process is crucial for successful development of academia. In this talk we will discuss our two recent works on fairness of peer review. In the first part of the talk, we will focus on automated assignment of papers to reviewers in conference peer review. We will show that assignment procedure currently employed by NeurIPS and ICML does not guarantee fairness and may discriminate against some submissions. In contrast, we will present the assignment algorithm that simultaneously ensures fairness and accuracy of the resulting allocation. In the second part of the talk, we will continue the long-lasting debate on single vs. double blind peer review. We will discuss prior works on testing for biases in single blind peer review and show that strict assumptions made there put these tests at risk of being unreliable. We will then present our novel test that accounts for various idiosyncrasies of peer review and provably controls for the Type-I error while having a non-trivial power.

Bio:
Ivan is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University, advised by Nihar Shah and Aarti Singh. His research interests lie in statistical learning theory and more specific in the field of learning from people. Before coming to CMU, he received a B.S. in Physics from Moscow Institute of Physics and Technology.
link to video