随机动态系统的学习和控制问题研究
摘要
随着现代科学技术和生产技术的发展,控制系统的日益复杂化,复杂系统的建模与控制成为控制界研究的热点问题之一。由于实际中普遍存在着不同程度的系统不确性,控制问题都不能用简单的确定性模型加以描述,所以需要把控制系统与学习系统不确定性结合起来作为一个问题考虑。学习和控制的本质为：一方面,控制信号使得系统输出趋向期望的目标(称为控制作用)；另一方面,控制信号还要减小系统参数的不确定性(称为对参数不确定性的学习作用)。两种作用在控制律的实现中是矛盾的,前者要求控制信号的变化趋向平缓,而后者要求维持一定幅度的激励,所以需要进行权衡。
本文主要从线性、非线性随机系统和多模型非线性随机系统进行了研究。
(1)对于具有未知参数的高斯白噪声随机线性系统,利用“效用函数”提出了一个的学习和控制权衡的控制策略。控制器既能够控制系统跟踪期望输出,又能对未知参数进行学习。通过仿真,验证了控制方法的有效性。
(2)对具有未知参数的非线性的随机系统的控制问题,提出了一种学习和控制权衡优化算法。控制一方面能使输出跟踪期望输出；另一方面能用RBF网络在线学习非线性系统中未知参数。并通过仿真比较,表明了算法的优越性。
(3)对于一类未知参数的非线性多模型随机系统,利用RBF网络在线学习非线性函数,运用Bayes后验概率对模型进行估计,最后根据代价函数得出控制信号。通过仿真,此算法能够较准确的得出系统切换的时间,能够准确跟踪系统变化。
With the development of modern science and technology and production technology, control system become increasingly complex, and complex system modeling and control become one of hot researches. As in practice the systems have uncertainties, the control problem can not use a simple deterministic model to describe, and it need to combine controlling a system and learning a system uncertainty to an issue. The nature of control is that: on the one hand, the control signal can make the system output towards the desired goal (called control action); the other hand, the control signal can reduce the uncertainty of system parameters (called learning the parameter uncertainty). However, two hands are contradictory, and the former requires the control signal to change smoothly, while the latter wants to maintain certain amplitude of motivation, so control need trade-off.
In this paper, linear, nonlinear stochastic system and multi-model nonlinear stochastic systems is studied.
(1) For the unknown parameters with Gaussian white noise stochastic linear systems, using "utility function" it is presented a trade-off of learning and controlling of the control strategy. Controller. on the one hand, can control the system toward the desired output:on the other hand, can learn the unknown linear parameters. Simulation results show the validation of such approach.
(2) With unknown parameters of the non-linear stochastic systems control problem, it is proposed a learning and controlling optimization algorithm. Controller can not only control the output to track the desired output, but also can using RBF networks online learn unknown parameters of nonlinear systems. Simulation results show superiority of the algorithm.
(3) For a class of unknown parameters of nonlinear multi-model switching stochastic systems, it is proposed an algorithm, which use RBF network to learn nonlinear functions online, and Bayes posterior probability to estimate the model, according to the cost function obtain the control signal. By simulation, the algorithm can obtain a switching time of system exactly and can accurately track changes in the system.
本文主要从线性、非线性随机系统和多模型非线性随机系统进行了研究。
(1)对于具有未知参数的高斯白噪声随机线性系统,利用“效用函数”提出了一个的学习和控制权衡的控制策略。控制器既能够控制系统跟踪期望输出,又能对未知参数进行学习。通过仿真,验证了控制方法的有效性。
(2)对具有未知参数的非线性的随机系统的控制问题,提出了一种学习和控制权衡优化算法。控制一方面能使输出跟踪期望输出；另一方面能用RBF网络在线学习非线性系统中未知参数。并通过仿真比较,表明了算法的优越性。
(3)对于一类未知参数的非线性多模型随机系统,利用RBF网络在线学习非线性函数,运用Bayes后验概率对模型进行估计,最后根据代价函数得出控制信号。通过仿真,此算法能够较准确的得出系统切换的时间,能够准确跟踪系统变化。
With the development of modern science and technology and production technology, control system become increasingly complex, and complex system modeling and control become one of hot researches. As in practice the systems have uncertainties, the control problem can not use a simple deterministic model to describe, and it need to combine controlling a system and learning a system uncertainty to an issue. The nature of control is that: on the one hand, the control signal can make the system output towards the desired goal (called control action); the other hand, the control signal can reduce the uncertainty of system parameters (called learning the parameter uncertainty). However, two hands are contradictory, and the former requires the control signal to change smoothly, while the latter wants to maintain certain amplitude of motivation, so control need trade-off.
In this paper, linear, nonlinear stochastic system and multi-model nonlinear stochastic systems is studied.
(1) For the unknown parameters with Gaussian white noise stochastic linear systems, using "utility function" it is presented a trade-off of learning and controlling of the control strategy. Controller. on the one hand, can control the system toward the desired output:on the other hand, can learn the unknown linear parameters. Simulation results show the validation of such approach.
(2) With unknown parameters of the non-linear stochastic systems control problem, it is proposed a learning and controlling optimization algorithm. Controller can not only control the output to track the desired output, but also can using RBF networks online learn unknown parameters of nonlinear systems. Simulation results show superiority of the algorithm.
(3) For a class of unknown parameters of nonlinear multi-model switching stochastic systems, it is proposed an algorithm, which use RBF network to learn nonlinear functions online, and Bayes posterior probability to estimate the model, according to the cost function obtain the control signal. By simulation, the algorithm can obtain a switching time of system exactly and can accurately track changes in the system.
引文
[1]黄琳,段志生.控制科学中的复杂性[J].自动化学报,2003,29(5)：748-754.
[2]孙优贤.工业过程控制技术：应用篇[M].北京：化学工业出版社，2006.
[3]朱豫才.过程控制的多变量系统辨识[M].长沙：国防科技大学出版社，2005.
[4]冯纯伯.复杂系统的控制问题-试谈控制科学的发展[J].控制理论与应用：2004,21(6)：855-857.
[5]Y C Zhu. System identification for process control:recent experiences and outlook [J]. International Journal of Modeling, Identification and Control,2009,6(2):89-103.
[6]Schrama.R.J.P. Accurate identification for control:the necessity of an iterative scheme [J]. IEEE Trans.on Automatic Control,1992,37(7):991-994.
[7]Feldbaum A A. Dual control theory:Ⅰ [J]. Automation and Remote Control,1960,21:874-880.
[8]Feldbaum A A. Dual control theory:Ⅱ [J]. Automation and Remote Control,1960,21:1033-1039.
[9]Feldbaum A A. Dual control theory:Ⅲ [J]. Automation and Remote Control,1961,22:1-12.
[10]Feldbaum A A. Dual control theory:Ⅳ [J]. Automation and Remote Control,1961,22:109-121.
[11]Astrom K J, Wittenmark B. 自适应控制 [M]. 北京：科学出版社,1992.
[12]梁军.对偶自适应控制[J].控制理论与应用，1997,14(3)：297-305.
[13]W.J. Murphy. Optimal stochastic control of discrete time systems with unknown gain [J]. IEEE Trans.on Automatic Control,1968,13(4):338-342.
[14]Alspach.D.L. Dual controls based on approximate a posterior density function [J].IEEE Trans. on Automatic Control,1972,17(5):689-692.
[15]Moreno.L. Acosta.L. and Sanchez.J.L. Design of algorithms for spatial-time reduction complexity of dynamic programming [J]. IEEE Proc. D. Control Theory Application.1992,139(2):172-180.
[16]E.Tse. B.Shalom and L.Meier. Widen-sense adaptive dual controller nonlinear stochastic system [J]. IEEE Trans. on Automatic Control,1973,18(2):98-108.
[17]R.Milito, C.S.Padilla, R.A.Padilla and D.Cadorin. An innovation approach to dual control [J]. IEEE Trans. on Automatic Control,1982,27(1):132-137.
[18]A.L.Maitelli and T.Yoneyama.Two-stage suboptimal dual controller for systems with stochastic parameters using optimal predictors [J]. IEEE Proc. D. Control Theory Application,1994, 141(4):253-260.
[19]H.F.Chen and L.Guo. Convergence rate of LS identification and adaptive control for stochastic systems [J]. Int. J of Control,1986,44(5):1459-1476.
[20]H.F.Chen and L.Guo. A robust stochastic adaptive controller [J]. IEEE Trans. on Automatic Control,1988,33(11):1035-1043.
[21]D.Li, F.C.Qian and P.L.Fu.Variance Minimization approach for a class of dual control problems [J]. IEEE Trans. on Automatic Control,002,47(12):2010-2021.
[22]P.L.Fu, D.Li and F.C.Qian. Active dual control for linear-quadratic Gaussian system with unknown parameters [C]. The 15th IFAC World Congress, Barcelona, Spain,2002:21-26.
[23]钱富才,宋俐,陈小可.基于滚动优化的对偶控制策略[J].控制理论与应用,2005，12(6)：855-860.
[24]高振斌,钱富才,刘丁.参数未知随机系统的最小方差对偶自适应控[J].自动化学报,2007，29(10)：1709-1713.
[25]郑大钟.线性系统理论[M].北京：清华大学出版社,2002.
[26]Alexander Lanzon and George Papageorgiou. Distance measures for uncertain linear systems:A general theory [J]. IEEE Trans, on Automatic Control, 2009,57(12):1532-1547.
[27]Jian Chen, Behal, A. and Dawson, D.M. Robust feedback control for a class of uncertain MIMO nonlinear systems [J]. IEEE Trans. on Automatic Control,2008,53(3):591-596.
[28]B. Carew and P. R. Belaxger. Identification of optimum filter stead-state gain for systems with unknown noise covariance [J]. IEEE Trans. on Automatic Control,1973,18(6):582-587.
[29]Ian R. Petersen. Robust unobservability for uncertain linear systems with structured uncertainty [J]. IEEE Trans. on Automatic Control,2007,52(8):1461-1470.
[30]Mark French. Adaptive control and robustness in the gap metric [J]. IEEE Trans. on Automatic Control,2008,53(2):461-478.
[31]Duan Li, Fucai Qian and Peilin Fu. Optimal nominal dual control for discrete-time LQG problem with unknown parameters [J]. Automatica,2008,44(1):119-127.
[32]D. Li, F.C. Qian and P.L. Fu. Variance minimization in stochastic systems[C]. In X Y, Zhou (Ed.). Stochastic Modeling and Control, Springer, New York,2002.
[33]Duan Li, Fucai Qian and Peilin Fu. Research on dual control[J]. Acta Automatica Sinica,2005, 31(1):32-42.
[34]Duan Li, Peilin Fu and Fucai Qian. Guaranteed performance for discrete-time LQG problem performance [C]. The 16th IFAC World Congress, Prague, Czech Republic,2005,4-8.
[35]岳超源.决策理论与方法[M].北京：科学出版社,2003.
[36]Reid, R. W. Citron, S. J. On noninferior performance index vectors [J]. J of Optimization and Applications,1971,7(1):11-28.
[37]侯媛彬,汪梅,王立琦.系统辨识及其matlab仿真[M].北京：科学出版社,2004.
[38]孙增圻.智能控制理论与技术[M].北京：清华大学出版社,2003.
[39]朱全民.非线性系统辨识[J].控制理论与应用,1994,11(6)：641-652.
[40]FAFRI-S.KADIRKAMANATHAN V. Dual Adaptive Control of Nonlinear Stochastic Systems using Neural Network [J]. Automatica,1998,134(2):245-253.
[41]钱富才,刘丁,李云霞.基于两级算法的对偶控制[J].控制理论与应用,2004,21(1)：89-93.
[42]高隽.人工神经网络原理及仿真实例[M].北京：机械工业出版社.2007.
[43]Lainiotis D.G. Deshpande J.G. and Upadhyay T.N. Optimal adaptive control:a nonlinear separation theorem [J]. International Journal of Control.1972,15(5):877-888.
[44]Narendra K.S. Balakrishnan J. Adaptive control using multiple models [J]. IEEE Trans. on Automatic Control.1997.42(2):171-187.
[45]Miyazawa Y. Robust flight control system design with multiple model approach [J]. Journal of Guidance Control and Dynamics,1992.15(3):785-788.
[46]Athans M. Castanon D. Dunn K P.t al. The stochastic control of the F-8C aircraft using a multiple model adaptive control (MMAC) method-part I:equilibrium flight [J]. IEEE Trans. on Automatic Control.1997.22(5):768-780.
[47]Boskovic J.D. Mehra R.K. Stable multiple model adaptive flight control for accommodation of a large control effecter failure [C]. Proceedings of the American Control Conference. American. 1999.1920-1924.
[48]Ciliz M.K. Narendra K.S. Intelligent control of robotic manipulators:a multiple models based approach [J]. Proceedings of IEEE International Conference on Intelligent Robots and Systems. American,1995,422-427.
[49]Karimi A. Landau I. Robust adaptive control of s flexible transmission system using multiple models [J]. IEEE Trans. on Control Systems Technology.2002.8(2):321-331.
[50]Yu C. Roy R.J. Multiple-model adaptive predictive control of mean arterial pressure and cardiac output [J]. IEEE Trans. on Biomedical Engineering.1992,39(8):765-778.
[51]He W.G. Kaufman H. Roy R. Multiple-model adaptive control procedure for blood pressure control [J]. IEEE Trans. on Biomedical Engineering.1986.33(1):10-19.
[52]Zheng H, Zhu K Y, Zhang D G. Design of an adaptive drug delivery control system[C]. Proceedings of 2005 International Conference on Control and Automation. American,2005, 846-851.
[53]韩曾晋.自适应控制[M].北京：清华大学出版社.1995.
[54]Filatov. N.M. Survey of adaptive dual control methods [J]. IEEE Proceedings:Control Theory and Applications,2000.147(1):118-128.
[2]孙优贤.工业过程控制技术：应用篇[M].北京：化学工业出版社，2006.
[3]朱豫才.过程控制的多变量系统辨识[M].长沙：国防科技大学出版社，2005.
[4]冯纯伯.复杂系统的控制问题-试谈控制科学的发展[J].控制理论与应用：2004,21(6)：855-857.
[5]Y C Zhu. System identification for process control:recent experiences and outlook [J]. International Journal of Modeling, Identification and Control,2009,6(2):89-103.
[6]Schrama.R.J.P. Accurate identification for control:the necessity of an iterative scheme [J]. IEEE Trans.on Automatic Control,1992,37(7):991-994.
[7]Feldbaum A A. Dual control theory:Ⅰ [J]. Automation and Remote Control,1960,21:874-880.
[8]Feldbaum A A. Dual control theory:Ⅱ [J]. Automation and Remote Control,1960,21:1033-1039.
[9]Feldbaum A A. Dual control theory:Ⅲ [J]. Automation and Remote Control,1961,22:1-12.
[10]Feldbaum A A. Dual control theory:Ⅳ [J]. Automation and Remote Control,1961,22:109-121.
[11]Astrom K J, Wittenmark B. 自适应控制 [M]. 北京：科学出版社,1992.
[12]梁军.对偶自适应控制[J].控制理论与应用，1997,14(3)：297-305.
[13]W.J. Murphy. Optimal stochastic control of discrete time systems with unknown gain [J]. IEEE Trans.on Automatic Control,1968,13(4):338-342.
[14]Alspach.D.L. Dual controls based on approximate a posterior density function [J].IEEE Trans. on Automatic Control,1972,17(5):689-692.
[15]Moreno.L. Acosta.L. and Sanchez.J.L. Design of algorithms for spatial-time reduction complexity of dynamic programming [J]. IEEE Proc. D. Control Theory Application.1992,139(2):172-180.
[16]E.Tse. B.Shalom and L.Meier. Widen-sense adaptive dual controller nonlinear stochastic system [J]. IEEE Trans. on Automatic Control,1973,18(2):98-108.
[17]R.Milito, C.S.Padilla, R.A.Padilla and D.Cadorin. An innovation approach to dual control [J]. IEEE Trans. on Automatic Control,1982,27(1):132-137.
[18]A.L.Maitelli and T.Yoneyama.Two-stage suboptimal dual controller for systems with stochastic parameters using optimal predictors [J]. IEEE Proc. D. Control Theory Application,1994, 141(4):253-260.
[19]H.F.Chen and L.Guo. Convergence rate of LS identification and adaptive control for stochastic systems [J]. Int. J of Control,1986,44(5):1459-1476.
[20]H.F.Chen and L.Guo. A robust stochastic adaptive controller [J]. IEEE Trans. on Automatic Control,1988,33(11):1035-1043.
[21]D.Li, F.C.Qian and P.L.Fu.Variance Minimization approach for a class of dual control problems [J]. IEEE Trans. on Automatic Control,002,47(12):2010-2021.
[22]P.L.Fu, D.Li and F.C.Qian. Active dual control for linear-quadratic Gaussian system with unknown parameters [C]. The 15th IFAC World Congress, Barcelona, Spain,2002:21-26.
[23]钱富才,宋俐,陈小可.基于滚动优化的对偶控制策略[J].控制理论与应用,2005，12(6)：855-860.
[24]高振斌,钱富才,刘丁.参数未知随机系统的最小方差对偶自适应控[J].自动化学报,2007，29(10)：1709-1713.
[25]郑大钟.线性系统理论[M].北京：清华大学出版社,2002.
[26]Alexander Lanzon and George Papageorgiou. Distance measures for uncertain linear systems:A general theory [J]. IEEE Trans, on Automatic Control, 2009,57(12):1532-1547.
[27]Jian Chen, Behal, A. and Dawson, D.M. Robust feedback control for a class of uncertain MIMO nonlinear systems [J]. IEEE Trans. on Automatic Control,2008,53(3):591-596.
[28]B. Carew and P. R. Belaxger. Identification of optimum filter stead-state gain for systems with unknown noise covariance [J]. IEEE Trans. on Automatic Control,1973,18(6):582-587.
[29]Ian R. Petersen. Robust unobservability for uncertain linear systems with structured uncertainty [J]. IEEE Trans. on Automatic Control,2007,52(8):1461-1470.
[30]Mark French. Adaptive control and robustness in the gap metric [J]. IEEE Trans. on Automatic Control,2008,53(2):461-478.
[31]Duan Li, Fucai Qian and Peilin Fu. Optimal nominal dual control for discrete-time LQG problem with unknown parameters [J]. Automatica,2008,44(1):119-127.
[32]D. Li, F.C. Qian and P.L. Fu. Variance minimization in stochastic systems[C]. In X Y, Zhou (Ed.). Stochastic Modeling and Control, Springer, New York,2002.
[33]Duan Li, Fucai Qian and Peilin Fu. Research on dual control[J]. Acta Automatica Sinica,2005, 31(1):32-42.
[34]Duan Li, Peilin Fu and Fucai Qian. Guaranteed performance for discrete-time LQG problem performance [C]. The 16th IFAC World Congress, Prague, Czech Republic,2005,4-8.
[35]岳超源.决策理论与方法[M].北京：科学出版社,2003.
[36]Reid, R. W. Citron, S. J. On noninferior performance index vectors [J]. J of Optimization and Applications,1971,7(1):11-28.
[37]侯媛彬,汪梅,王立琦.系统辨识及其matlab仿真[M].北京：科学出版社,2004.
[38]孙增圻.智能控制理论与技术[M].北京：清华大学出版社,2003.
[39]朱全民.非线性系统辨识[J].控制理论与应用,1994,11(6)：641-652.
[40]FAFRI-S.KADIRKAMANATHAN V. Dual Adaptive Control of Nonlinear Stochastic Systems using Neural Network [J]. Automatica,1998,134(2):245-253.
[41]钱富才,刘丁,李云霞.基于两级算法的对偶控制[J].控制理论与应用,2004,21(1)：89-93.
[42]高隽.人工神经网络原理及仿真实例[M].北京：机械工业出版社.2007.
[43]Lainiotis D.G. Deshpande J.G. and Upadhyay T.N. Optimal adaptive control:a nonlinear separation theorem [J]. International Journal of Control.1972,15(5):877-888.
[44]Narendra K.S. Balakrishnan J. Adaptive control using multiple models [J]. IEEE Trans. on Automatic Control.1997.42(2):171-187.
[45]Miyazawa Y. Robust flight control system design with multiple model approach [J]. Journal of Guidance Control and Dynamics,1992.15(3):785-788.
[46]Athans M. Castanon D. Dunn K P.t al. The stochastic control of the F-8C aircraft using a multiple model adaptive control (MMAC) method-part I:equilibrium flight [J]. IEEE Trans. on Automatic Control.1997.22(5):768-780.
[47]Boskovic J.D. Mehra R.K. Stable multiple model adaptive flight control for accommodation of a large control effecter failure [C]. Proceedings of the American Control Conference. American. 1999.1920-1924.
[48]Ciliz M.K. Narendra K.S. Intelligent control of robotic manipulators:a multiple models based approach [J]. Proceedings of IEEE International Conference on Intelligent Robots and Systems. American,1995,422-427.
[49]Karimi A. Landau I. Robust adaptive control of s flexible transmission system using multiple models [J]. IEEE Trans. on Control Systems Technology.2002.8(2):321-331.
[50]Yu C. Roy R.J. Multiple-model adaptive predictive control of mean arterial pressure and cardiac output [J]. IEEE Trans. on Biomedical Engineering.1992,39(8):765-778.
[51]He W.G. Kaufman H. Roy R. Multiple-model adaptive control procedure for blood pressure control [J]. IEEE Trans. on Biomedical Engineering.1986.33(1):10-19.
[52]Zheng H, Zhu K Y, Zhang D G. Design of an adaptive drug delivery control system[C]. Proceedings of 2005 International Conference on Control and Automation. American,2005, 846-851.
[53]韩曾晋.自适应控制[M].北京：清华大学出版社.1995.
[54]Filatov. N.M. Survey of adaptive dual control methods [J]. IEEE Proceedings:Control Theory and Applications,2000.147(1):118-128.