DeepASPeer: Towards an aspect-level sentiment controllable framework for decision prediction from academic peer reviews

Sandeep Kumar, Hardik Arora, Tirthankar Ghosal, Asif Ekbal

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

10 Scopus citations

Abstract

Peer review is the widely accepted mechanism to determine the quality of scientific work. Even though peer-reviewing has been an integral part of academia since the 1600s, it frequently receives criticism for the lack of transparency and consistency. Even for humans, predicting the peer review outcome is a challenging task as there are many dimensions and human factors involved. However, Artificial Intelligence (AI) techniques can assist the editor/chair anticipate the final decision based on the reviews from the human reviewers. Peer review texts reflect the reviewers' opinions/ sentiments on various aspects (e.g., novelty, substance, soundness, etc.) of the paper concerning the research in the paper, which may be valuable to predict a manuscript's acceptance or rejection. The exact types and number of aspects could vary from one to the other venue (i.e., the conferences or journals). Peer review texts, however, which often contain rich sentiment information about the reviewers and, therefore, their overall opinion of the paper's research, can be useful in predicting a manuscript's acceptance or rejection. Here in this work, we study how we could take advantage of aspects and their corresponding sentiment to build a generic controllable system to assist the editor/chair in determining the outcome based on the reviews of a paper to make better editorial decisions. Our proposed deep neural architecture considers three information channels, including reviews, review aspect category, and its sentiment, to predict the final decision. Experimental results show that our model can achieve up to 76.67% accuracy on the ASAP-Review dataset (Aspect-enhanced Peer Review) consisting of ICLR and NIPS reviews considering the sentiment of the reviews. Empirical results also show an improvement of around 3.3 points while aspect information is added to the sentiment information1.

Original languageEnglish
Title of host publicationJCDL 2022 - Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781450393454
DOIs
StatePublished - Jun 20 2022
Externally publishedYes
Event22nd ACM/IEEE Joint Conference on Digital Libraries, JCDL 2022 - Virtual, Online, Germany
Duration: Jun 20 2022Jun 24 2022

Publication series

NameProceedings of the ACM/IEEE Joint Conference on Digital Libraries
ISSN (Print)1552-5996

Conference

Conference22nd ACM/IEEE Joint Conference on Digital Libraries, JCDL 2022
Country/TerritoryGermany
CityVirtual, Online
Period06/20/2206/24/22

Funding

Sandeep Kumar acknowledges the Prime Minister Research Fellowship (PMRF) program of the Govt of India for its support. Asif Ekbal acknowledges the Young Faculty Research Fellowship (YFRF), supported by Visvesvaraya PhD scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia). Tirthankar Ghosal acknowledges Cactus Communications, India.

Keywords

  • Aspect-based Sentiment Analysis
  • Decision Prediction
  • Deep Neural Network
  • Peer Reviews

Fingerprint

Dive into the research topics of 'DeepASPeer: Towards an aspect-level sentiment controllable framework for decision prediction from academic peer reviews'. Together they form a unique fingerprint.

Cite this