Scalable balanced training of conditional generative adversarial neural networks on image data

Massimiliano Lupo Pasini; Vittorio Gabbi; Junqi Yin; Simona Perotto; Nouamane Laanait

doi:10.1007/s11227-021-03808-2

Scalable balanced training of conditional generative adversarial neural networks on image data

Massimiliano Lupo Pasini, Vittorio Gabbi, Junqi Yin, Simona Perotto, Nouamane Laanait

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

Original language	English
Pages (from-to)	13358-13384
Number of pages	27
Journal	Journal of Supercomputing
Volume	77
Issue number	11
DOIs	https://doi.org/10.1007/s11227-021-03808-2
State	Published - Nov 2021

Funding

Research completed through the Artificial Intelligence Initiative sponsored by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U. S. Department of Energy. This work used resources of the Oak Ridge Leadership Computing Facility, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This manuscript has been authored in part by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan ( http://energy.gov/downloads/doe-public-access-plan ).

Funders	Funder number
Laboratory Directed Research
U.S. Department of Energy	DE-AC05-00OR22725
Office of Science
Oak Ridge National Laboratory

Keywords

Computer vision
Deep learning
Generative adversarial neural networks
Supercomputing

Access to Document

10.1007/s11227-021-03808-2

Cite this

@article{de39635cd320478298cdf62a49811db5,

title = "Scalable balanced training of conditional generative adversarial neural networks on image data",

abstract = "We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fr{\'e}chet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.",

keywords = "Computer vision, Deep learning, Generative adversarial neural networks, Supercomputing",

author = "{Lupo Pasini}, Massimiliano and Vittorio Gabbi and Junqi Yin and Simona Perotto and Nouamane Laanait",

note = "Publisher Copyright: {\textcopyright} 2021, This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply.",

year = "2021",

month = nov,

doi = "10.1007/s11227-021-03808-2",

language = "English",

volume = "77",

pages = "13358--13384",

journal = "Journal of Supercomputing",

issn = "0920-8542",

publisher = "Springer",

number = "11",

}

TY - JOUR

T1 - Scalable balanced training of conditional generative adversarial neural networks on image data

AU - Lupo Pasini, Massimiliano

AU - Gabbi, Vittorio

AU - Yin, Junqi

AU - Perotto, Simona

AU - Laanait, Nouamane

PY - 2021/11

Y1 - 2021/11

N2 - We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

AB - We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

KW - Computer vision

KW - Deep learning

KW - Generative adversarial neural networks

KW - Supercomputing

UR - http://www.scopus.com/inward/record.url?scp=85105312271&partnerID=8YFLogxK

U2 - 10.1007/s11227-021-03808-2

DO - 10.1007/s11227-021-03808-2

M3 - Article

AN - SCOPUS:85105312271

SN - 0920-8542

VL - 77

SP - 13358

EP - 13384

JO - Journal of Supercomputing

JF - Journal of Supercomputing

IS - 11

ER -

Scalable balanced training of conditional generative adversarial neural networks on image data

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this