Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Catherine D. Schuman; J. Parker Mitchell; J. Travis Johnston; Maryam Parsa; Bill Kay; Prasanna Date; Robert M. Patton

doi:10.1109/IJCNN48605.2020.9207560

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Catherine D. Schuman, J. Parker Mitchell, J. Travis Johnston, Maryam Parsa, Bill Kay, Prasanna Date, Robert M. Patton

Learning Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

17 Scopus citations

Abstract

Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algo-rithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.

Original language	English
Title of host publication	2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728169262
DOIs	https://doi.org/10.1109/IJCNN48605.2020.9207560
State	Published - Jul 2020
Event	2020 International Joint Conference on Neural Networks, IJCNN 2020 - Virtual, Glasgow, United Kingdom Duration: Jul 19 2020 → Jul 24 2020

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks

Conference

Conference	2020 International Joint Conference on Neural Networks, IJCNN 2020
Country/Territory	United Kingdom
City	Virtual, Glasgow
Period	07/19/20 → 07/24/20

Funding

This material is based in part upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under contract number DE-AC05-00OR22725, and in part by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC. Notice: This manuscript has been authored in part by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Keywords

neuromorphic computing
resilience
spiking neural net-works

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/IJCNN48605.2020.9207560

Cite this

Schuman, C. D., Parker Mitchell, J., Johnston, J. T., Parsa, M., Kay, B., Date, P., & Patton, R. M. (2020). Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems. In 2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings Article 9207560 (Proceedings of the International Joint Conference on Neural Networks). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN48605.2020.9207560

@inproceedings{de421153d3184fddbac256f4b94557f9,

title = "Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems",

abstract = "Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algo-rithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.",

keywords = "neuromorphic computing, resilience, spiking neural net-works",

author = "Schuman, {Catherine D.} and {Parker Mitchell}, J. and Johnston, {J. Travis} and Maryam Parsa and Bill Kay and Prasanna Date and Patton, {Robert M.}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 International Joint Conference on Neural Networks, IJCNN 2020 ; Conference date: 19-07-2020 Through 24-07-2020",

year = "2020",

month = jul,

doi = "10.1109/IJCNN48605.2020.9207560",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings",

}

Schuman, CD, Parker Mitchell, J, Johnston, JT, Parsa, M, Kay, B, Date, P & Patton, RM 2020, Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems. in 2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings., 9207560, Proceedings of the International Joint Conference on Neural Networks, Institute of Electrical and Electronics Engineers Inc., 2020 International Joint Conference on Neural Networks, IJCNN 2020, Virtual, Glasgow, United Kingdom, 07/19/20. https://doi.org/10.1109/IJCNN48605.2020.9207560

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems. / Schuman, Catherine D.; Parker Mitchell, J.; Johnston, J. Travis et al.
2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. 9207560 (Proceedings of the International Joint Conference on Neural Networks).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

AU - Schuman, Catherine D.

AU - Parker Mitchell, J.

AU - Johnston, J. Travis

AU - Parsa, Maryam

AU - Kay, Bill

AU - Date, Prasanna

AU - Patton, Robert M.

PY - 2020/7

Y1 - 2020/7

N2 - Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algo-rithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.

AB - Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algo-rithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.

KW - neuromorphic computing

KW - resilience

KW - spiking neural net-works

UR - http://www.scopus.com/inward/record.url?scp=85093840314&partnerID=8YFLogxK

U2 - 10.1109/IJCNN48605.2020.9207560

DO - 10.1109/IJCNN48605.2020.9207560

M3 - Conference contribution

AN - SCOPUS:85093840314

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - 2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 International Joint Conference on Neural Networks, IJCNN 2020

Y2 - 19 July 2020 through 24 July 2020

ER -

Schuman CD, Parker Mitchell J, Johnston JT, Parsa M, Kay B, Date P et al. Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems. In 2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2020. 9207560. (Proceedings of the International Joint Conference on Neural Networks). doi: 10.1109/IJCNN48605.2020.9207560

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Abstract

Publication series

Conference

Funding

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this