A BACKWARD SDE METHOD FOR UNCERTAINTY QUANTIFICATION IN DEEP LEARNING

Richard Archibald; Feng Bao; Yanzhao Cao; He Zhang

doi:10.3934/dcdss.2022062

A BACKWARD SDE METHOD FOR UNCERTAINTY QUANTIFICATION IN DEEP LEARNING

Richard Archibald, Feng Bao, Yanzhao Cao, He Zhang

Data Analysis and Machine Learning

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

We develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

Original language	English
Pages (from-to)	2807-2835
Number of pages	29
Journal	Discrete and Continuous Dynamical Systems - Series S
Volume	15
Issue number	10
DOIs	https://doi.org/10.3934/dcdss.2022062
State	Published - Oct 2022

Funding

2020 Mathematics Subject Classification. 660H35, 68T07, 93E20. Key words and phrases. Probabilistic machine learning, stochastic neural networks, stochastic optimal control, stochastic gradient descent. The second and third authors are partially supported by U.S. Department of Energy under grant numbers DE-SC0022297 and DE-SC0022253, the last author is supported by NSFC12071175 and Science and Technology Development of Jilin Province, China no. 201902013020. ∗ Corresponding author: He Zhang.

Funders	Funder number
NSFC12071175 and Science and Technology Development of Jilin Province	201902013020
U.S. Department of Energy	DE-SC0022253, DE-SC0022297

Keywords

Probabilistic machine learning
stochastic gradient descent
stochastic neural networks
stochastic optimal control

Access to Document

10.3934/dcdss.2022062

Cite this

@article{f3e8100495f14e9a861e6a542b65bdc5,

title = "A BACKWARD SDE METHOD FOR UNCERTAINTY QUANTIFICATION IN DEEP LEARNING",

abstract = "We develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.",

keywords = "Probabilistic machine learning, stochastic gradient descent, stochastic neural networks, stochastic optimal control",

author = "Richard Archibald and Feng Bao and Yanzhao Cao and He Zhang",

year = "2022",

month = oct,

doi = "10.3934/dcdss.2022062",

language = "English",

volume = "15",

pages = "2807--2835",

journal = "Discrete and Continuous Dynamical Systems - Series S",

issn = "1937-1632",

publisher = "American Institute of Mathematical Sciences (AIMS)",

number = "10",

}

TY - JOUR

T1 - A BACKWARD SDE METHOD FOR UNCERTAINTY QUANTIFICATION IN DEEP LEARNING

AU - Archibald, Richard

AU - Bao, Feng

AU - Cao, Yanzhao

AU - Zhang, He

PY - 2022/10

Y1 - 2022/10

N2 - We develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

AB - We develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

KW - Probabilistic machine learning

KW - stochastic gradient descent

KW - stochastic neural networks

KW - stochastic optimal control

UR - http://www.scopus.com/inward/record.url?scp=85135476233&partnerID=8YFLogxK

U2 - 10.3934/dcdss.2022062

DO - 10.3934/dcdss.2022062

M3 - Article

AN - SCOPUS:85135476233

SN - 1937-1632

VL - 15

SP - 2807

EP - 2835

JO - Discrete and Continuous Dynamical Systems - Series S

JF - Discrete and Continuous Dynamical Systems - Series S

IS - 10

ER -

A BACKWARD SDE METHOD FOR UNCERTAINTY QUANTIFICATION IN DEEP LEARNING

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this