Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks

Catherine D. Schuman; Grant Bruer; Aaron R. Young; Mark Dean; James S. Plank

doi:10.1109/IJCNN.2018.8489149

Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks

Catherine D. Schuman, Grant Bruer, Aaron R. Young, Mark Dean, James S. Plank

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

Original language	English
Title of host publication	2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781509060146
DOIs	https://doi.org/10.1109/IJCNN.2018.8489149
State	Published - Oct 10 2018
Externally published	Yes
Event	2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil Duration: Jul 8 2018 → Jul 13 2018

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks
Volume	2018-July

Conference

Conference	2018 International Joint Conference on Neural Networks, IJCNN 2018
Country/Territory	Brazil
City	Rio de Janeiro
Period	07/8/18 → 07/13/18

Funding

Notice: This manuscript has been authored in part by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Funders	Funder number
DOE Office of Science	DE-AC05-00OR22725
U. S. Department of Energy
Oak Ridge National Laboratory
Air Force Research Laboratory	FA8750-16-1-0065

Access to Document

10.1109/IJCNN.2018.8489149

Cite this

Schuman, C. D., Bruer, G., Young, A. R., Dean, M., & Plank, J. S. (2018). Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings Article 8489149 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2018-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2018.8489149

@inproceedings{73e222fe03924d2893cabfad43c5e37d,

title = "Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks",

abstract = "Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.",

author = "Schuman, {Catherine D.} and Grant Bruer and Young, {Aaron R.} and Mark Dean and Plank, {James S.}",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 International Joint Conference on Neural Networks, IJCNN 2018 ; Conference date: 08-07-2018 Through 13-07-2018",

year = "2018",

month = oct,

day = "10",

doi = "10.1109/IJCNN.2018.8489149",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings",

}

Schuman, CD, Bruer, G, Young, AR, Dean, M & Plank, JS 2018, Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks. in 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings., 8489149, Proceedings of the International Joint Conference on Neural Networks, vol. 2018-July, Institute of Electrical and Electronics Engineers Inc., 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, 07/8/18. https://doi.org/10.1109/IJCNN.2018.8489149

Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks. / Schuman, Catherine D.; Bruer, Grant; Young, Aaron R. et al.
2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. 8489149 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2018-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks

AU - Schuman, Catherine D.

AU - Bruer, Grant

AU - Young, Aaron R.

AU - Dean, Mark

AU - Plank, James S.

PY - 2018/10/10

Y1 - 2018/10/10

N2 - Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

AB - Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

UR - http://www.scopus.com/inward/record.url?scp=85056500375&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2018.8489149

DO - 10.1109/IJCNN.2018.8489149

M3 - Conference contribution

AN - SCOPUS:85056500375

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 International Joint Conference on Neural Networks, IJCNN 2018

Y2 - 8 July 2018 through 13 July 2018

ER -

Schuman CD, Bruer G, Young AR, Dean M, Plank JS. Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2018. 8489149. (Proceedings of the International Joint Conference on Neural Networks). doi: 10.1109/IJCNN.2018.8489149

Understanding Selection and Diversity for Evolution of Spiking Recurrent Neural Networks

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this