Learning algorithms for feedforward networks based on finite samples

Nageswara S.V. Rao; Vladimir Protopopescu; Reinhold C. Mann; E. M. Oblow; S. Sitharama Iyengar

doi:10.1109/72.508936

Learning algorithms for feedforward networks based on finite samples

Nageswara S.V. Rao, Vladimir Protopopescu, Reinhold C. Mann, E. M. Oblow, S. Sitharama Iyengar

Research output: Contribution to journal › Article › peer-review

18 Scopus citations

Abstract

We present two classes of convergent algorithms for learning continuous functions and regressions that are approximated by feedforward networks. The first class of algorithms, applicable to networks with unknown weights located only in the output layer, is obtained by utilizing the potential function methods of Aizerman et al. The second class, applicable to general feedforward networks, is obtained by utilizing the classical Robbins-Monro style stochastic approximation methods. Conditions relating the sample sizes to the error bounds are derived for both classes of algorithms using martingale-type inequalities. For concreteness, the discussion is presented in terms of neural networks, but the results are applicable to general feedforward networks, in particular to wavelet networks. The algorithms can be directly adapted to concept learning problems.

Original language	English
Pages (from-to)	926-940
Number of pages	15
Journal	IEEE Transactions on Neural Networks
Volume	7
Issue number	4
DOIs	https://doi.org/10.1109/72.508936
State	Published - 1996

Funding

Manuscript received September 19, 1994; revised February 3, 1996. This work was sponsored by the Engineering Research Program of the Office of Basic Energy Sciences of the US. Department of Energy, under Contract DE-AC05-960R22464 with Lockheed Martin Energy Research Corp. N. S. V. Rao, V. Protopopescu, R. C. Mann, and E. M. Oblow are with the Center for Engineering Systems Advanced Research, Oak Ridge National Laboratory, Oak Ridge, TN 37831-6364 USA. S. S. Iyengar is with the Department of Computer Science, Louisiana State University, Baton Rouge, LA 70803 USA. Publisher Item Identifier S 1045-9227(96)04395-0.

Funders	Funder number
Office of Basic Energy Sciences
U.S. Department of Energy	DE-AC05-960R22464

Access to Document

10.1109/72.508936

Cite this

@article{7b718ecfc19f49be8f73c63d78172396,

title = "Learning algorithms for feedforward networks based on finite samples",

abstract = "We present two classes of convergent algorithms for learning continuous functions and regressions that are approximated by feedforward networks. The first class of algorithms, applicable to networks with unknown weights located only in the output layer, is obtained by utilizing the potential function methods of Aizerman et al. The second class, applicable to general feedforward networks, is obtained by utilizing the classical Robbins-Monro style stochastic approximation methods. Conditions relating the sample sizes to the error bounds are derived for both classes of algorithms using martingale-type inequalities. For concreteness, the discussion is presented in terms of neural networks, but the results are applicable to general feedforward networks, in particular to wavelet networks. The algorithms can be directly adapted to concept learning problems.",

author = "Rao, {Nageswara S.V.} and Vladimir Protopopescu and Mann, {Reinhold C.} and Oblow, {E. M.} and Iyengar, {S. Sitharama}",

year = "1996",

doi = "10.1109/72.508936",

language = "English",

volume = "7",

pages = "926--940",

journal = "IEEE Transactions on Neural Networks",

issn = "1045-9227",

number = "4",

}

TY - JOUR

T1 - Learning algorithms for feedforward networks based on finite samples

AU - Rao, Nageswara S.V.

AU - Protopopescu, Vladimir

AU - Mann, Reinhold C.

AU - Oblow, E. M.

AU - Iyengar, S. Sitharama

PY - 1996

Y1 - 1996

N2 - We present two classes of convergent algorithms for learning continuous functions and regressions that are approximated by feedforward networks. The first class of algorithms, applicable to networks with unknown weights located only in the output layer, is obtained by utilizing the potential function methods of Aizerman et al. The second class, applicable to general feedforward networks, is obtained by utilizing the classical Robbins-Monro style stochastic approximation methods. Conditions relating the sample sizes to the error bounds are derived for both classes of algorithms using martingale-type inequalities. For concreteness, the discussion is presented in terms of neural networks, but the results are applicable to general feedforward networks, in particular to wavelet networks. The algorithms can be directly adapted to concept learning problems.

AB - We present two classes of convergent algorithms for learning continuous functions and regressions that are approximated by feedforward networks. The first class of algorithms, applicable to networks with unknown weights located only in the output layer, is obtained by utilizing the potential function methods of Aizerman et al. The second class, applicable to general feedforward networks, is obtained by utilizing the classical Robbins-Monro style stochastic approximation methods. Conditions relating the sample sizes to the error bounds are derived for both classes of algorithms using martingale-type inequalities. For concreteness, the discussion is presented in terms of neural networks, but the results are applicable to general feedforward networks, in particular to wavelet networks. The algorithms can be directly adapted to concept learning problems.

UR - http://www.scopus.com/inward/record.url?scp=0030190723&partnerID=8YFLogxK

U2 - 10.1109/72.508936

DO - 10.1109/72.508936

M3 - Article

AN - SCOPUS:0030190723

SN - 1045-9227

VL - 7

SP - 926

EP - 940

JO - IEEE Transactions on Neural Networks

JF - IEEE Transactions on Neural Networks

IS - 4

ER -

Learning algorithms for feedforward networks based on finite samples

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Cite this