Distributed-Memory Parallel JointNMF

Srinivas Eswar; Benjamin Cobb; Koby Hayashi; Ramakrishnan Kannan; Grey Ballard; Richard Vuduc; Haesun Park

doi:10.1145/3577193.3593733

Distributed-Memory Parallel JointNMF

Srinivas Eswar, Benjamin Cobb, Koby Hayashi, Ramakrishnan Kannan, Grey Ballard, Richard Vuduc, Haesun Park

Discrete Algorithms

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

Original language	English
Title of host publication	ACM ICS 2023 - Proceedings of the International Conference on Supercomputing
Publisher	Association for Computing Machinery
Pages	301-312
Number of pages	12
ISBN (Electronic)	9798400700569
DOIs	https://doi.org/10.1145/3577193.3593733
State	Published - Jun 21 2023
Event	37th ACM International Conference on Supercomputing, ICS 2023 - Orlando, United States Duration: Jun 21 2023 → Jun 23 2023

Publication series

Name	Proceedings of the International Conference on Supercomputing

Conference

Conference	37th ACM International Conference on Supercomputing, ICS 2023
Country/Territory	United States
City	Orlando
Period	06/21/23 → 06/23/23

Funding

This material is based upon work supported by the National Science Foundation (NSF) under Grant Nos. OAC-2106920 and CCF-1942892. It is also based upon work supported by the U.S. Department of Energy, Office of Science under Contract DE-AC02-06CH11357 at Argonne National Laboratory and UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 at Oak Ridge National Laboratory. Additionally, it is based upon work supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research program under Award Number DE-SC-0023296. Koby Hayashi acknowledges support from the United States Department of Energy through the Computational Sciences Graduate Fellowship (DOE CSGF) under grant number: DE-SC0020347. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DOE or NSF.

Keywords

high performance computing
multimodal inputs
nonnegative matrix factorization

Access to Document

10.1145/3577193.3593733

Cite this

@inproceedings{567156607ebc4ad09fbd757088ed276d,

title = "Distributed-Memory Parallel JointNMF",

abstract = "Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.",

keywords = "high performance computing, multimodal inputs, nonnegative matrix factorization",

author = "Srinivas Eswar and Benjamin Cobb and Koby Hayashi and Ramakrishnan Kannan and Grey Ballard and Richard Vuduc and Haesun Park",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; 37th ACM International Conference on Supercomputing, ICS 2023 ; Conference date: 21-06-2023 Through 23-06-2023",

year = "2023",

month = jun,

day = "21",

doi = "10.1145/3577193.3593733",

language = "English",

series = "Proceedings of the International Conference on Supercomputing",

publisher = "Association for Computing Machinery",

pages = "301--312",

booktitle = "ACM ICS 2023 - Proceedings of the International Conference on Supercomputing",

}

Eswar, S, Cobb, B, Hayashi, K, Kannan, R, Ballard, G, Vuduc, R & Park, H 2023, Distributed-Memory Parallel JointNMF. in ACM ICS 2023 - Proceedings of the International Conference on Supercomputing. Proceedings of the International Conference on Supercomputing, Association for Computing Machinery, pp. 301-312, 37th ACM International Conference on Supercomputing, ICS 2023, Orlando, United States, 06/21/23. https://doi.org/10.1145/3577193.3593733

Distributed-Memory Parallel JointNMF. / Eswar, Srinivas; Cobb, Benjamin; Hayashi, Koby et al.
ACM ICS 2023 - Proceedings of the International Conference on Supercomputing. Association for Computing Machinery, 2023. p. 301-312 (Proceedings of the International Conference on Supercomputing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Distributed-Memory Parallel JointNMF

AU - Eswar, Srinivas

AU - Cobb, Benjamin

AU - Hayashi, Koby

AU - Kannan, Ramakrishnan

AU - Ballard, Grey

AU - Vuduc, Richard

AU - Park, Haesun

PY - 2023/6/21

Y1 - 2023/6/21

N2 - Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

AB - Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

KW - high performance computing

KW - multimodal inputs

KW - nonnegative matrix factorization

UR - http://www.scopus.com/inward/record.url?scp=85168425207&partnerID=8YFLogxK

U2 - 10.1145/3577193.3593733

DO - 10.1145/3577193.3593733

M3 - Conference contribution

AN - SCOPUS:85168425207

T3 - Proceedings of the International Conference on Supercomputing

SP - 301

EP - 312

BT - ACM ICS 2023 - Proceedings of the International Conference on Supercomputing

PB - Association for Computing Machinery

T2 - 37th ACM International Conference on Supercomputing, ICS 2023

Y2 - 21 June 2023 through 23 June 2023

ER -

Distributed-Memory Parallel JointNMF

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this