TAP-DLND 1.0: A corpus for document level novelty detection

Tirthankar Ghosal; Amitra Salam; Swati Tiwari; Asif Ekbal; Pushpak Bhattacharyya

TAP-DLND 1.0: A corpus for document level novelty detection

Tirthankar Ghosal, Amitra Salam, Swati Tiwari, Asif Ekbal, Pushpak Bhattacharyya

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

Detecting novelty of an entire document is an Artificial Intelligence (AI) frontier problem. This has immense importance in widespread Natural Language Processing (NLP) applications ranging from extractive text document summarization to tracking development of news events to predicting impact of scholarly articles. Although a very relevant problem in the present context of exponential data duplication, we are unaware of any document level dataset that correctly addresses the evaluation of automatic novelty detection techniques in a classification framework. To bridge this relative gap, here in this work, we present a resource for benchmarking the techniques for document level novelty detection. We create the resource via topic-specific crawling of news documents across several domains in a periodic manner. We release the annotated corpus with necessary statistics and show its use with a developed system for the problem in concern.

Original language	English
Title of host publication	LREC 2018 - 11th International Conference on Language Resources and Evaluation
Editors	Hitoshi Isahara, Bente Maegaard, Stelios Piperidis, Christopher Cieri, Thierry Declerck, Koiti Hasida, Helene Mazo, Khalid Choukri, Sara Goggi, Joseph Mariani, Asuncion Moreno, Nicoletta Calzolari, Jan Odijk, Takenobu Tokunaga
Publisher	European Language Resources Association (ELRA)
Pages	3541-3547
Number of pages	7
ISBN (Electronic)	9791095546009
State	Published - 2019
Externally published	Yes
Event	11th International Conference on Language Resources and Evaluation, LREC 2018 - Miyazaki, Japan Duration: May 7 2018 → May 12 2018

Publication series

Name	LREC 2018 - 11th International Conference on Language Resources and Evaluation

Conference

Conference	11th International Conference on Language Resources and Evaluation, LREC 2018
Country/Territory	Japan
City	Miyazaki
Period	05/7/18 → 05/12/18

Funding

Tirthankar Ghosal and Asif Ekbal gratefully acknowledge Visvesvaraya PhD scheme for Electronics and Information Technology, an initiative of Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia). Tirthankar is a PhD scholar under Visves-varaya PhD scheme and Asif is the recipient of Sir Visves-varaya Young Faculty Research Fellow Award. The work is also generously supported by Elsevier Centre of Excellence for Natural Language Processing, Department of Computer Science and Engineering, Indian Institute of Technology Patna. We would also like to thank the anonymous reviewers for their valuable suggestions in improving this work.

Keywords

Classification
Corpus
Document level novelty detection
Web crawling

Cite this

Ghosal, T., Salam, A., Tiwari, S., Ekbal, A., & Bhattacharyya, P. (2019). TAP-DLND 1.0: A corpus for document level novelty detection. In H. Isahara, B. Maegaard, S. Piperidis, C. Cieri, T. Declerck, K. Hasida, H. Mazo, K. Choukri, S. Goggi, J. Mariani, A. Moreno, N. Calzolari, J. Odijk, & T. Tokunaga (Eds.), LREC 2018 - 11th International Conference on Language Resources and Evaluation (pp. 3541-3547). (LREC 2018 - 11th International Conference on Language Resources and Evaluation). European Language Resources Association (ELRA).

Ghosal, Tirthankar ; Salam, Amitra ; Tiwari, Swati et al. / TAP-DLND 1.0 : A corpus for document level novelty detection. LREC 2018 - 11th International Conference on Language Resources and Evaluation. editor / Hitoshi Isahara ; Bente Maegaard ; Stelios Piperidis ; Christopher Cieri ; Thierry Declerck ; Koiti Hasida ; Helene Mazo ; Khalid Choukri ; Sara Goggi ; Joseph Mariani ; Asuncion Moreno ; Nicoletta Calzolari ; Jan Odijk ; Takenobu Tokunaga. European Language Resources Association (ELRA), 2019. pp. 3541-3547 (LREC 2018 - 11th International Conference on Language Resources and Evaluation).

@inproceedings{720e144b787c49c9a6289188e531281f,

title = "TAP-DLND 1.0: A corpus for document level novelty detection",

abstract = "Detecting novelty of an entire document is an Artificial Intelligence (AI) frontier problem. This has immense importance in widespread Natural Language Processing (NLP) applications ranging from extractive text document summarization to tracking development of news events to predicting impact of scholarly articles. Although a very relevant problem in the present context of exponential data duplication, we are unaware of any document level dataset that correctly addresses the evaluation of automatic novelty detection techniques in a classification framework. To bridge this relative gap, here in this work, we present a resource for benchmarking the techniques for document level novelty detection. We create the resource via topic-specific crawling of news documents across several domains in a periodic manner. We release the annotated corpus with necessary statistics and show its use with a developed system for the problem in concern.",

keywords = "Classification, Corpus, Document level novelty detection, Web crawling",

author = "Tirthankar Ghosal and Amitra Salam and Swati Tiwari and Asif Ekbal and Pushpak Bhattacharyya",

note = "Publisher Copyright: {\textcopyright} LREC 2018 - 11th International Conference on Language Resources and Evaluation. All rights reserved.; 11th International Conference on Language Resources and Evaluation, LREC 2018 ; Conference date: 07-05-2018 Through 12-05-2018",

year = "2019",

language = "English",

series = "LREC 2018 - 11th International Conference on Language Resources and Evaluation",

publisher = "European Language Resources Association (ELRA)",

pages = "3541--3547",

editor = "Hitoshi Isahara and Bente Maegaard and Stelios Piperidis and Christopher Cieri and Thierry Declerck and Koiti Hasida and Helene Mazo and Khalid Choukri and Sara Goggi and Joseph Mariani and Asuncion Moreno and Nicoletta Calzolari and Jan Odijk and Takenobu Tokunaga",

booktitle = "LREC 2018 - 11th International Conference on Language Resources and Evaluation",

}

Ghosal, T, Salam, A, Tiwari, S, Ekbal, A & Bhattacharyya, P 2019, TAP-DLND 1.0: A corpus for document level novelty detection. in H Isahara, B Maegaard, S Piperidis, C Cieri, T Declerck, K Hasida, H Mazo, K Choukri, S Goggi, J Mariani, A Moreno, N Calzolari, J Odijk & T Tokunaga (eds), LREC 2018 - 11th International Conference on Language Resources and Evaluation. LREC 2018 - 11th International Conference on Language Resources and Evaluation, European Language Resources Association (ELRA), pp. 3541-3547, 11th International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, 05/7/18.

TAP-DLND 1.0: A corpus for document level novelty detection. / Ghosal, Tirthankar; Salam, Amitra; Tiwari, Swati et al.
LREC 2018 - 11th International Conference on Language Resources and Evaluation. ed. / Hitoshi Isahara; Bente Maegaard; Stelios Piperidis; Christopher Cieri; Thierry Declerck; Koiti Hasida; Helene Mazo; Khalid Choukri; Sara Goggi; Joseph Mariani; Asuncion Moreno; Nicoletta Calzolari; Jan Odijk; Takenobu Tokunaga. European Language Resources Association (ELRA), 2019. p. 3541-3547 (LREC 2018 - 11th International Conference on Language Resources and Evaluation).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - TAP-DLND 1.0

T2 - 11th International Conference on Language Resources and Evaluation, LREC 2018

AU - Ghosal, Tirthankar

AU - Salam, Amitra

AU - Tiwari, Swati

AU - Ekbal, Asif

AU - Bhattacharyya, Pushpak

PY - 2019

Y1 - 2019

N2 - Detecting novelty of an entire document is an Artificial Intelligence (AI) frontier problem. This has immense importance in widespread Natural Language Processing (NLP) applications ranging from extractive text document summarization to tracking development of news events to predicting impact of scholarly articles. Although a very relevant problem in the present context of exponential data duplication, we are unaware of any document level dataset that correctly addresses the evaluation of automatic novelty detection techniques in a classification framework. To bridge this relative gap, here in this work, we present a resource for benchmarking the techniques for document level novelty detection. We create the resource via topic-specific crawling of news documents across several domains in a periodic manner. We release the annotated corpus with necessary statistics and show its use with a developed system for the problem in concern.

AB - Detecting novelty of an entire document is an Artificial Intelligence (AI) frontier problem. This has immense importance in widespread Natural Language Processing (NLP) applications ranging from extractive text document summarization to tracking development of news events to predicting impact of scholarly articles. Although a very relevant problem in the present context of exponential data duplication, we are unaware of any document level dataset that correctly addresses the evaluation of automatic novelty detection techniques in a classification framework. To bridge this relative gap, here in this work, we present a resource for benchmarking the techniques for document level novelty detection. We create the resource via topic-specific crawling of news documents across several domains in a periodic manner. We release the annotated corpus with necessary statistics and show its use with a developed system for the problem in concern.

KW - Classification

KW - Corpus

KW - Document level novelty detection

KW - Web crawling

UR - http://www.scopus.com/inward/record.url?scp=85059899170&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85059899170

T3 - LREC 2018 - 11th International Conference on Language Resources and Evaluation

SP - 3541

EP - 3547

BT - LREC 2018 - 11th International Conference on Language Resources and Evaluation

A2 - Isahara, Hitoshi

A2 - Maegaard, Bente

A2 - Piperidis, Stelios

A2 - Cieri, Christopher

A2 - Declerck, Thierry

A2 - Hasida, Koiti

A2 - Mazo, Helene

A2 - Choukri, Khalid

A2 - Goggi, Sara

A2 - Mariani, Joseph

A2 - Moreno, Asuncion

A2 - Calzolari, Nicoletta

A2 - Odijk, Jan

A2 - Tokunaga, Takenobu

PB - European Language Resources Association (ELRA)

Y2 - 7 May 2018 through 12 May 2018

ER -

Ghosal T, Salam A, Tiwari S, Ekbal A, Bhattacharyya P. TAP-DLND 1.0: A corpus for document level novelty detection. In Isahara H, Maegaard B, Piperidis S, Cieri C, Declerck T, Hasida K, Mazo H, Choukri K, Goggi S, Mariani J, Moreno A, Calzolari N, Odijk J, Tokunaga T, editors, LREC 2018 - 11th International Conference on Language Resources and Evaluation. European Language Resources Association (ELRA). 2019. p. 3541-3547. (LREC 2018 - 11th International Conference on Language Resources and Evaluation).

TAP-DLND 1.0: A corpus for document level novelty detection

Abstract

Publication series

Conference

Funding

Keywords

Other files and links

Fingerprint

Cite this