Abstract
A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradientdescent- based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NOx) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NOx's are reduced by over 80% compared with stoichiometric levels.
Original language | English |
---|---|
Pages (from-to) | 1162-1179 |
Number of pages | 18 |
Journal | IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics |
Volume | 39 |
Issue number | 5 |
DOIs | |
State | Published - 2009 |
Externally published | Yes |
Funding
Manuscript received February 7, 2007; revised October 24, 2007, February 18, 2008, and March 22, 2008. First published March 24, 2009; current version published September 16, 2009. This work was supported in part by the National Science Foundation (NSF) under Grant ECCS 0327877 and Grant ECCS 0621924, NSF I/UCRC Award for IMS, GAANN Program, and the Intelligent Systems Center. This paper was recommended by Associate Editor F. L. Lewis.
Keywords
- Adaptive critic
- Discrete-time system
- Engine emission control
- Nonstrict nonlinear output feedback
- Reinforcement learning control