Abstract
Peer review is the widely accepted method of research validation. However, with the deluge of research paper submissions accompanied with the rising number venues, the paper vetting system has come under a lot of stress. Problems like dearth of adequate reviewers, finding appropriate expert reviewers, maintaining the quality of the reviews are steadily and strongly surfacing up. To ease the peer review workload to some extent, here we investigate how an Artificial Intelligence (AI)-powered review system would look like. We leverage on the paper-review interaction to predict the decision in the reviewing process. We do not envisage an AI reviewing papers in the near-future, but seek to explore a human-AI collaboration in the decision-making process where the AI would leverage on the human-written reviews and paper full-text to predict the fate of the paper. The idea is to have an assistive decision-making tool for the chairs/editors to help them with an additional layer of confidence, especially with borderline and contrastive reviews. We use cross-attention between the review text and paper full-text to learn the interactions and henceforth generate the decision. We also make use of sentiment information encoded within peer-review texts to guide the outcome. Our initial results show encouraging performance on a dataset of paper+peer reviews curated from the ICLR openreviews. We make our codes and dataset (https://github.com/PrabhatkrBharti/PEERAssist ) public for further explorations. We re-iterate that we are in an early stage of investigation and showcase our initial exciting results to justify our proposition.
Original language | English |
---|---|
Title of host publication | Towards Open and Trustworthy Digital Societies - 23rd International Conference on Asia-Pacific Digital Libraries, ICADL 2021, Proceedings |
Editors | Hao-Ren Ke, Chei Sian Lee, Kazunari Sugiyama |
Publisher | Springer Science and Business Media Deutschland GmbH |
Pages | 421-435 |
Number of pages | 15 |
ISBN (Print) | 9783030916688 |
DOIs | |
State | Published - 2021 |
Externally published | Yes |
Event | 23rd International Conference on Asia-Pacific Digital Libraries, ICADL 2021 - Virtual, Online Duration: Dec 1 2021 → Dec 3 2021 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 13133 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 23rd International Conference on Asia-Pacific Digital Libraries, ICADL 2021 |
---|---|
City | Virtual, Online |
Period | 12/1/21 → 12/3/21 |
Funding
Acknowledgement. Asif Ekbal is a recipient of the Visvesvaraya Young Faculty Award and acknowledges Digital India Corporation, Ministry of Electronics and Information Technology, Government of India for supporting this research.
Keywords
- Cross attention
- Decision prediction
- Deep neural network
- Peer reviews