TY - GEN
T1 - Consistency Analysis of NLP Approaches for a Conference Reviewer-Manuscript Match-Making System
AU - Kotak, Nishith
AU - Roy, Anil K.
AU - Dasgupta, Sourish
AU - Ghosal, Tirthankar
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Peer-review process is an important part of scholarly communication. The quality of a conference also depends on its peer-review process. The selection of a competent reviewer to review a submitted manuscript in a conference, is a crucial facet of a peer-review process. This selection relies on the adopted match-making approach along with the constraint optimization reviewer allocation algorithm. The match-making approach needs to be consistent with its decision of selection and allocation of the reviewers. In this work, we proposed a framework for evaluating the consistency of various standard NLP approaches that are used for match-making process in a conference. The consistency analysis was performed over a real multi-tracked conference organized in 2019. We showed that the Contextual Neural Topic Modeling (CNTM) with word embedding technique was most consistence among all the 13 approaches that we chose to analyze.
AB - Peer-review process is an important part of scholarly communication. The quality of a conference also depends on its peer-review process. The selection of a competent reviewer to review a submitted manuscript in a conference, is a crucial facet of a peer-review process. This selection relies on the adopted match-making approach along with the constraint optimization reviewer allocation algorithm. The match-making approach needs to be consistent with its decision of selection and allocation of the reviewers. In this work, we proposed a framework for evaluating the consistency of various standard NLP approaches that are used for match-making process in a conference. The consistency analysis was performed over a real multi-tracked conference organized in 2019. We showed that the Contextual Neural Topic Modeling (CNTM) with word embedding technique was most consistence among all the 13 approaches that we chose to analyze.
KW - CNTM
KW - Consistency Analysis
KW - Keyword extraction approach
KW - NLP
KW - Reviewer-Manuscript Match-Making
KW - Semantics Analysis
KW - Topic Modeling approach
KW - Transformer based embedding approach
UR - http://www.scopus.com/inward/record.url?scp=85126394763&partnerID=8YFLogxK
U2 - 10.1109/INDICON52576.2021.9691571
DO - 10.1109/INDICON52576.2021.9691571
M3 - Conference contribution
AN - SCOPUS:85126394763
T3 - Proceedings of the 2021 IEEE 18th India Council International Conference, INDICON 2021
BT - Proceedings of the 2021 IEEE 18th India Council International Conference, INDICON 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 18th IEEE India Council International Conference, INDICON 2021
Y2 - 19 December 2021 through 21 December 2021
ER -