Single-iteration training algorithm for multi-layer feed-forward neural networks

J. Barhen; R. Cogswell; V. Protopopescu

doi:10.1023/A:1009682730770

Single-iteration training algorithm for multi-layer feed-forward neural networks

J. Barhen, R. Cogswell, V. Protopopescu

Computational Sciences & Engineering Div

Research output: Contribution to journal › Article › peer-review

8 Scopus citations

Abstract

A new methodology for neural learning is presented. Only a single iteration is needed to train a feed-forward network with near-optimal results. This is achieved by introducing a key modification to the conventional multi-layer architecture. A virtual input layer is implemented, which is connected to the nominal input layer by a special nonlinear transfer function, and to the first hidden layer by regular (linear) synapses. A sequence of alternating direction singular value decompositions is then used to determine precisely the inter-layer synaptic weights. This computational paradigm exploits the known separability of the linear (inter-layer propagation) and nonlinear (neuron activation) aspects of information transfer within a neural network. Examples show that the trained neural networks generalize well.

Original language	English
Pages (from-to)	113-129
Number of pages	17
Journal	Neural Processing Letters
Volume	11
Issue number	2
DOIs	https://doi.org/10.1023/A:1009682730770
State	Published - 2000

Funding

The research described herein was performed at the Center for Engineering Science Advanced Research, Oak Ridge National Laboratory. Primary funding was provided by the DeepLook Consortium under Agreement Number ERD-97-1506 with Lockheed Martin Energy Research Corporation. Additional funding was provided by the Engineering Research Program of the Office of Basic Energy Sciences under contract DE-AC05-96OR22464 with Lockheed Martin Energy Research Corporation. R. Cogswell's participation was supported by the Great Lakes Colleges Association/Associated Colleges of the Midwest Oak Ridge Science Semester, sponsored by the Office of University Science Education, Oak Ridge National Laboratory. * Research supported by the Engineering Research Program, Office of Basic Energy Sciences, of the U.S. Department of Energy, under contract No. DE-AC05-96OR22464 with Lockheed Martin Energy Research Corporation. The U.S. Government's right to a non-exclusive, royalty-free license in and to any copyright is acknowledged.

Funders	Funder number
DeepLook Consortium	ERD-97-1506
Lockheed Martin Energy Research Corporation
Midwest Oak Ridge Science Semester
Office of University
U.S. Department of Energy
Basic Energy Sciences	DE-AC05-96OR22464
Oak Ridge National Laboratory

Access to Document

10.1023/A:1009682730770

Cite this

@article{b41e0b5ddfaa484a82a3d561bffb3b35,

title = "Single-iteration training algorithm for multi-layer feed-forward neural networks",

abstract = "A new methodology for neural learning is presented. Only a single iteration is needed to train a feed-forward network with near-optimal results. This is achieved by introducing a key modification to the conventional multi-layer architecture. A virtual input layer is implemented, which is connected to the nominal input layer by a special nonlinear transfer function, and to the first hidden layer by regular (linear) synapses. A sequence of alternating direction singular value decompositions is then used to determine precisely the inter-layer synaptic weights. This computational paradigm exploits the known separability of the linear (inter-layer propagation) and nonlinear (neuron activation) aspects of information transfer within a neural network. Examples show that the trained neural networks generalize well.",

author = "J. Barhen and R. Cogswell and V. Protopopescu",

year = "2000",

doi = "10.1023/A:1009682730770",

language = "English",

volume = "11",

pages = "113--129",

journal = "Neural Processing Letters",

issn = "1370-4621",

publisher = "Springer",

number = "2",

}

TY - JOUR

T1 - Single-iteration training algorithm for multi-layer feed-forward neural networks

AU - Barhen, J.

AU - Cogswell, R.

AU - Protopopescu, V.

PY - 2000

Y1 - 2000

N2 - A new methodology for neural learning is presented. Only a single iteration is needed to train a feed-forward network with near-optimal results. This is achieved by introducing a key modification to the conventional multi-layer architecture. A virtual input layer is implemented, which is connected to the nominal input layer by a special nonlinear transfer function, and to the first hidden layer by regular (linear) synapses. A sequence of alternating direction singular value decompositions is then used to determine precisely the inter-layer synaptic weights. This computational paradigm exploits the known separability of the linear (inter-layer propagation) and nonlinear (neuron activation) aspects of information transfer within a neural network. Examples show that the trained neural networks generalize well.

AB - A new methodology for neural learning is presented. Only a single iteration is needed to train a feed-forward network with near-optimal results. This is achieved by introducing a key modification to the conventional multi-layer architecture. A virtual input layer is implemented, which is connected to the nominal input layer by a special nonlinear transfer function, and to the first hidden layer by regular (linear) synapses. A sequence of alternating direction singular value decompositions is then used to determine precisely the inter-layer synaptic weights. This computational paradigm exploits the known separability of the linear (inter-layer propagation) and nonlinear (neuron activation) aspects of information transfer within a neural network. Examples show that the trained neural networks generalize well.

UR - http://www.scopus.com/inward/record.url?scp=0033747328&partnerID=8YFLogxK

U2 - 10.1023/A:1009682730770

DO - 10.1023/A:1009682730770

M3 - Article

AN - SCOPUS:0033747328

SN - 1370-4621

VL - 11

SP - 113

EP - 129

JO - Neural Processing Letters

JF - Neural Processing Letters

IS - 2

ER -

Single-iteration training algorithm for multi-layer feed-forward neural networks

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Cite this