Automated Indexing of Structured Scientific Metadata Using Apache Solr

Kavya Guntupally; Kyle Dumas; Wade Darnell; Michael Crow; Ranjeet Devarakonda; Prakash Giri

doi:10.1109/BigData50022.2020.9378448

Automated Indexing of Structured Scientific Metadata Using Apache Solr

Kavya Guntupally, Kyle Dumas, Wade Darnell, Michael Crow, Ranjeet Devarakonda, Prakash Giri

Earth System Data Management Operations

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Scientific datasets are continuously growing with the amount of raw data being collected worldwide. This amount of data poses the biggest challenge to web search engines on how to retrieve them efficiently. This paper discusses how major scientific data centers are using popular open-source search platforms such as Solr [1] to retrieve structured data stored in data sources such as relational database management systems using its import handler mechanisms [2]. Additionally, we will also focus on how we can configure Solr to serve advanced full-text, faceted search capabilities, along with its key features, which simplify representing and delivering better performance to the scientific search interfaces.

Original language	English
Title of host publication	Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020
Editors	Xintao Wu, Chris Jermaine, Li Xiong, Xiaohua Tony Hu, Olivera Kotevska, Siyuan Lu, Weijia Xu, Srinivas Aluru, Chengxiang Zhai, Eyhab Al-Masri, Zhiyuan Chen, Jeff Saltz
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	5685-5687
Number of pages	3
ISBN (Electronic)	9781728162515
DOIs	https://doi.org/10.1109/BigData50022.2020.9378448
State	Published - Dec 10 2020
Event	8th IEEE International Conference on Big Data, Big Data 2020 - Virtual, Atlanta, United States Duration: Dec 10 2020 → Dec 13 2020

Publication series

Name	Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

Conference

Conference	8th IEEE International Conference on Big Data, Big Data 2020
Country/Territory	United States
City	Virtual, Atlanta
Period	12/10/20 → 12/13/20

Funding

ACKNOWLEDGEMENT Oak Ridge National Laboratory is managed by the UT-Battelle, LLC, for the U.S. Department of Energy under contract DEAC05-00OR22725.

Keywords

Apache Lucene
Apache Solr
Data Import Handler
data indexing
faceting search

Access to Document

10.1109/BigData50022.2020.9378448

Cite this

Guntupally, K., Dumas, K., Darnell, W., Crow, M., Devarakonda, R., & Giri, P. (2020). Automated Indexing of Structured Scientific Metadata Using Apache Solr. In X. Wu, C. Jermaine, L. Xiong, X. T. Hu, O. Kotevska, S. Lu, W. Xu, S. Aluru, C. Zhai, E. Al-Masri, Z. Chen, & J. Saltz (Eds.), Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020 (pp. 5685-5687). Article 9378448 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData50022.2020.9378448

Guntupally, Kavya ; Dumas, Kyle ; Darnell, Wade et al. / Automated Indexing of Structured Scientific Metadata Using Apache Solr. Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. editor / Xintao Wu ; Chris Jermaine ; Li Xiong ; Xiaohua Tony Hu ; Olivera Kotevska ; Siyuan Lu ; Weijia Xu ; Srinivas Aluru ; Chengxiang Zhai ; Eyhab Al-Masri ; Zhiyuan Chen ; Jeff Saltz. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 5685-5687 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020).

@inproceedings{bb013c79c4954cac9fd44b50b9b4a120,

title = "Automated Indexing of Structured Scientific Metadata Using Apache Solr",

abstract = "Scientific datasets are continuously growing with the amount of raw data being collected worldwide. This amount of data poses the biggest challenge to web search engines on how to retrieve them efficiently. This paper discusses how major scientific data centers are using popular open-source search platforms such as Solr [1] to retrieve structured data stored in data sources such as relational database management systems using its import handler mechanisms [2]. Additionally, we will also focus on how we can configure Solr to serve advanced full-text, faceted search capabilities, along with its key features, which simplify representing and delivering better performance to the scientific search interfaces.",

keywords = "Apache Lucene, Apache Solr, Data Import Handler, data indexing, faceting search",

author = "Kavya Guntupally and Kyle Dumas and Wade Darnell and Michael Crow and Ranjeet Devarakonda and Prakash Giri",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 8th IEEE International Conference on Big Data, Big Data 2020 ; Conference date: 10-12-2020 Through 13-12-2020",

year = "2020",

month = dec,

day = "10",

doi = "10.1109/BigData50022.2020.9378448",

language = "English",

series = "Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "5685--5687",

editor = "Xintao Wu and Chris Jermaine and Li Xiong and Hu, {Xiaohua Tony} and Olivera Kotevska and Siyuan Lu and Weijia Xu and Srinivas Aluru and Chengxiang Zhai and Eyhab Al-Masri and Zhiyuan Chen and Jeff Saltz",

booktitle = "Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020",

}

Guntupally, K, Dumas, K, Darnell, W, Crow, M, Devarakonda, R & Giri, P 2020, Automated Indexing of Structured Scientific Metadata Using Apache Solr. in X Wu, C Jermaine, L Xiong, XT Hu, O Kotevska, S Lu, W Xu, S Aluru, C Zhai, E Al-Masri, Z Chen & J Saltz (eds), Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020., 9378448, Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020, Institute of Electrical and Electronics Engineers Inc., pp. 5685-5687, 8th IEEE International Conference on Big Data, Big Data 2020, Virtual, Atlanta, United States, 12/10/20. https://doi.org/10.1109/BigData50022.2020.9378448

Automated Indexing of Structured Scientific Metadata Using Apache Solr. / Guntupally, Kavya; Dumas, Kyle; Darnell, Wade et al.
Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. ed. / Xintao Wu; Chris Jermaine; Li Xiong; Xiaohua Tony Hu; Olivera Kotevska; Siyuan Lu; Weijia Xu; Srinivas Aluru; Chengxiang Zhai; Eyhab Al-Masri; Zhiyuan Chen; Jeff Saltz. Institute of Electrical and Electronics Engineers Inc., 2020. p. 5685-5687 9378448 (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automated Indexing of Structured Scientific Metadata Using Apache Solr

AU - Guntupally, Kavya

AU - Dumas, Kyle

AU - Darnell, Wade

AU - Crow, Michael

AU - Devarakonda, Ranjeet

AU - Giri, Prakash

PY - 2020/12/10

Y1 - 2020/12/10

N2 - Scientific datasets are continuously growing with the amount of raw data being collected worldwide. This amount of data poses the biggest challenge to web search engines on how to retrieve them efficiently. This paper discusses how major scientific data centers are using popular open-source search platforms such as Solr [1] to retrieve structured data stored in data sources such as relational database management systems using its import handler mechanisms [2]. Additionally, we will also focus on how we can configure Solr to serve advanced full-text, faceted search capabilities, along with its key features, which simplify representing and delivering better performance to the scientific search interfaces.

AB - Scientific datasets are continuously growing with the amount of raw data being collected worldwide. This amount of data poses the biggest challenge to web search engines on how to retrieve them efficiently. This paper discusses how major scientific data centers are using popular open-source search platforms such as Solr [1] to retrieve structured data stored in data sources such as relational database management systems using its import handler mechanisms [2]. Additionally, we will also focus on how we can configure Solr to serve advanced full-text, faceted search capabilities, along with its key features, which simplify representing and delivering better performance to the scientific search interfaces.

KW - Apache Lucene

KW - Apache Solr

KW - Data Import Handler

KW - data indexing

KW - faceting search

UR - http://www.scopus.com/inward/record.url?scp=85103819215&partnerID=8YFLogxK

U2 - 10.1109/BigData50022.2020.9378448

DO - 10.1109/BigData50022.2020.9378448

M3 - Conference contribution

AN - SCOPUS:85103819215

T3 - Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

SP - 5685

EP - 5687

BT - Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

A2 - Wu, Xintao

A2 - Jermaine, Chris

A2 - Xiong, Li

A2 - Hu, Xiaohua Tony

A2 - Kotevska, Olivera

A2 - Lu, Siyuan

A2 - Xu, Weijia

A2 - Aluru, Srinivas

A2 - Zhai, Chengxiang

A2 - Al-Masri, Eyhab

A2 - Chen, Zhiyuan

A2 - Saltz, Jeff

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 8th IEEE International Conference on Big Data, Big Data 2020

Y2 - 10 December 2020 through 13 December 2020

ER -

Guntupally K, Dumas K, Darnell W, Crow M, Devarakonda R, Giri P. Automated Indexing of Structured Scientific Metadata Using Apache Solr. In Wu X, Jermaine C, Xiong L, Hu XT, Kotevska O, Lu S, Xu W, Aluru S, Zhai C, Al-Masri E, Chen Z, Saltz J, editors, Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 5685-5687. 9378448. (Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020). doi: 10.1109/BigData50022.2020.9378448

Automated Indexing of Structured Scientific Metadata Using Apache Solr

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this