A stochastic gradient descent approach for stochastic optimal control

Richard Archibald; Feng Bao; Jiongmin Yong

doi:10.4208/eajam.190420.200420

A stochastic gradient descent approach for stochastic optimal control

Richard Archibald, Feng Bao, Jiongmin Yong

Data Analysis and Machine Learning

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

In this work, we introduce a stochastic gradient descent approach to solve the stochastic optimal control problem through stochastic maximum principle. The motivation that drives our method is the gradient of the cost functional in the stochastic optimal control problem is under expectation, and numerical calculation of such an expectation requires fully computation of a system of forward backward stochastic differential equations, which is computationally expensive. By evaluating the expectation with single-sample representation as suggested by the stochastic gradient descent type optimisation, we could save computational efforts in solving FBSDEs and only focus on the optimisation task which aims to determine the optimal control process.

Original language	English
Pages (from-to)	635-658
Number of pages	24
Journal	East Asian Journal on Applied Mathematics
Volume	10
Issue number	4
DOIs	https://doi.org/10.4208/eajam.190420.200420
State	Published - Nov 2020

Funding

This work is partially supported by the Scientific Discovery through Advanced Computing (SciDAC) program funded by U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research through FASTMath Institute and CompFUSE project. The second author also acknowledges support by U.S. National Science Foundation under Contract DMS-1720222. The third author acknowledges the partial support by NSF grant DMS-1812921.

Funders	Funder number
FASTMath Institute
U.S. National Science Foundation
National Science Foundation	DMS-1720222, DMS-1812921
U.S. Department of Energy
Office of Science

Keywords

Forward backward stochastic differential equations
Maximum principle
Stochastic gradient descent
Stochastic optimal control

Access to Document

10.4208/eajam.190420.200420

Cite this

@article{ebfbd362cd054cb6ae45c033f00f2743,

title = "A stochastic gradient descent approach for stochastic optimal control",

abstract = "In this work, we introduce a stochastic gradient descent approach to solve the stochastic optimal control problem through stochastic maximum principle. The motivation that drives our method is the gradient of the cost functional in the stochastic optimal control problem is under expectation, and numerical calculation of such an expectation requires fully computation of a system of forward backward stochastic differential equations, which is computationally expensive. By evaluating the expectation with single-sample representation as suggested by the stochastic gradient descent type optimisation, we could save computational efforts in solving FBSDEs and only focus on the optimisation task which aims to determine the optimal control process.",

keywords = "Forward backward stochastic differential equations, Maximum principle, Stochastic gradient descent, Stochastic optimal control",

author = "Richard Archibald and Feng Bao and Jiongmin Yong",

note = "Publisher Copyright: {\textcopyright} 2020 Global-Science Press",

year = "2020",

month = nov,

doi = "10.4208/eajam.190420.200420",

language = "English",

volume = "10",

pages = "635--658",

journal = "East Asian Journal on Applied Mathematics",

issn = "2079-7362",

number = "4",

}

TY - JOUR

T1 - A stochastic gradient descent approach for stochastic optimal control

AU - Archibald, Richard

AU - Bao, Feng

AU - Yong, Jiongmin

PY - 2020/11

Y1 - 2020/11

N2 - In this work, we introduce a stochastic gradient descent approach to solve the stochastic optimal control problem through stochastic maximum principle. The motivation that drives our method is the gradient of the cost functional in the stochastic optimal control problem is under expectation, and numerical calculation of such an expectation requires fully computation of a system of forward backward stochastic differential equations, which is computationally expensive. By evaluating the expectation with single-sample representation as suggested by the stochastic gradient descent type optimisation, we could save computational efforts in solving FBSDEs and only focus on the optimisation task which aims to determine the optimal control process.

AB - In this work, we introduce a stochastic gradient descent approach to solve the stochastic optimal control problem through stochastic maximum principle. The motivation that drives our method is the gradient of the cost functional in the stochastic optimal control problem is under expectation, and numerical calculation of such an expectation requires fully computation of a system of forward backward stochastic differential equations, which is computationally expensive. By evaluating the expectation with single-sample representation as suggested by the stochastic gradient descent type optimisation, we could save computational efforts in solving FBSDEs and only focus on the optimisation task which aims to determine the optimal control process.

KW - Forward backward stochastic differential equations

KW - Maximum principle

KW - Stochastic gradient descent

KW - Stochastic optimal control

UR - http://www.scopus.com/inward/record.url?scp=85090887950&partnerID=8YFLogxK

U2 - 10.4208/eajam.190420.200420

DO - 10.4208/eajam.190420.200420

M3 - Article

AN - SCOPUS:85090887950

SN - 2079-7362

VL - 10

SP - 635

EP - 658

JO - East Asian Journal on Applied Mathematics

JF - East Asian Journal on Applied Mathematics

IS - 4

ER -

A stochastic gradient descent approach for stochastic optimal control

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this