Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to engine emission control

Peter Shih, Brian C. Kaul, Sarangapani Jagannathan, James A. Drallmeier

Research output: Contribution to journalArticlepeer-review

38 Scopus citations

Abstract

A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradientdescent- based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NOx) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NOx's are reduced by over 80% compared with stoichiometric levels.

Original languageEnglish
Pages (from-to)1162-1179
Number of pages18
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume39
Issue number5
DOIs
StatePublished - 2009
Externally publishedYes

Funding

Manuscript received February 7, 2007; revised October 24, 2007, February 18, 2008, and March 22, 2008. First published March 24, 2009; current version published September 16, 2009. This work was supported in part by the National Science Foundation (NSF) under Grant ECCS 0327877 and Grant ECCS 0621924, NSF I/UCRC Award for IMS, GAANN Program, and the Intelligent Systems Center. This paper was recommended by Associate Editor F. L. Lewis.

Keywords

  • Adaptive critic
  • Discrete-time system
  • Engine emission control
  • Nonstrict nonlinear output feedback
  • Reinforcement learning control

Fingerprint

Dive into the research topics of 'Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to engine emission control'. Together they form a unique fingerprint.

Cite this