未知环境中无人驾驶船舶智能避碰决策方法
|
摘要
[目的]为了实现无人驾驶船舶在未知环境下的智能避障功能,[方法]首先,建立一种基于深度强化学习(DRL)技术的无人驾驶船舶智能避碰决策模型,分析无人驾驶船舶智能避碰决策面临的问题,提出智能避碰决策的设计准则。然后,在此基础上,建立基于Markov决策方法(MDP)的智能避碰决策模型,通过值函数求解决策模型中的最优策略,使无人驾驶船舶状态对行为映射中的回报最大,并专门设计由接近目标、偏离航线和安全性组成的激励函数。最后,分别在静态、动态障碍环境下进行仿真实验。[结果]结果表明,该智能决策方法可以有效避让障碍物,保障无人驾驶船舶在未知水域中的航行安全,[结论]所提方法可为无人驾驶船舶的自主航行提供理论参考。
[Objectives]In order to realize intelligent obstacle avoidance of unmanned vessel in unknown waters,[Methods]an intelligent obstacle avoidance decision-making model of the unmanned vesselbased on Deep Reinforcement Learning(DRL)is established. Here we analyze the problems encounteredin the unmanned vessel's intelligent obstacle avoidance decision-making,propose the design criteria ofthe intelligent obstacle avoidance decision-making,and then accordingly establish a decision-makingmodel based on Markov Decision Process(MDP),through which obtain the optimal strategy by valuefunction to make the maximum returns in behavior mapping of the unmanned vessel status and to design anexcitation function specially composed of target approaching,off course and safety. Finally,carry out thesimulation tests respectively in static and dynamic waters.[Results]The results show that the proposedintelligent decision-making method can effectively avoid obstacles,and ensure the safe navigation of theunmanned vessel in unknown waters. [Conclusions]The proposed method can provide a theoretical reference for autonomous navigation of the unmanned vessel.
[Objectives]In order to realize intelligent obstacle avoidance of unmanned vessel in unknown waters,[Methods]an intelligent obstacle avoidance decision-making model of the unmanned vesselbased on Deep Reinforcement Learning(DRL)is established. Here we analyze the problems encounteredin the unmanned vessel's intelligent obstacle avoidance decision-making,propose the design criteria ofthe intelligent obstacle avoidance decision-making,and then accordingly establish a decision-makingmodel based on Markov Decision Process(MDP),through which obtain the optimal strategy by valuefunction to make the maximum returns in behavior mapping of the unmanned vessel status and to design anexcitation function specially composed of target approaching,off course and safety. Finally,carry out thesimulation tests respectively in static and dynamic waters.[Results]The results show that the proposedintelligent decision-making method can effectively avoid obstacles,and ensure the safe navigation of theunmanned vessel in unknown waters. [Conclusions]The proposed method can provide a theoretical reference for autonomous navigation of the unmanned vessel.
引文
[1]赵忆文,谈大龙.基于速度矢量可行度的移动机器人多行为综合决策方法[J].信息与控制,2001,30(1):72-75.ZHAO Y W,TAN D L. Multi-behavior integrated-decision method based on feasibility of velocity vectors[J].Information and Control,2001,30(1):72-75(in Chinese).
[2]张晓东,童剑,郭敏,等.舰船物资转运方案计算机辅助决策算法研究[J].中国舰船研究,2011,6(4):104-110.ZHANG X D,TONG J,GUO M,et al. Computer aided decision-making method for material transfer planning on ship deck[J]. Chinese Journal of Ship Research,2011,6(4):104-110(in Chinese).
[3]王斌明.基于多传感器信息融合的移动机器人避障研究[D].南京:南京理工大学,2006.
[4] TANAKA Y,JI Y,YAMASHITA A,et al. Adaptive traversability analysis and behavior generation method for mobile robots on rough terrain[J]. Journal of the Japan Society for Precision Engineering,2015,81(12):1119-1126.
[5]陈雪梅,田赓,苗一松,等.城市环境下无人驾驶车辆驾驶规则获取及决策算法[J].北京理工大学学报,2017,37(5):491-496.CHEN X M,TIAN G,MIAO Y S,et al. Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic[J]. Transactions of Beijing Institute of Technology,2017,37(5):491-496(in Chinese).
[6]杜明博.基于人类驾驶行为的无人驾驶车辆行为决策与运动规划方法研究[D].合肥:中国科学技术大学,2016.
[7]田赓.复杂动态城市环境下无人驾驶车辆仿生换道决策模型研究[D].北京:北京理工大学,2016.
[8] TEMIZER S,KOCHENDERFER M J,KAELBLING L P,et al. Collision avoidance for unmanned aircraft using Markov decision processes[C]//AIAA Guidance,Navigation,and Control Conference. Toronto,Ontario,Canada:AIAA,2010.
[9] FURDA A,VLACIC L. Enabling safe autonomous driving in real-world city traffic using multiple criteria decision making[J]. IEEE Intelligent Transportation Systems Magazine,2011,3(1):4-17.
[10] LI X,XU X,ZUO L. Reinforcement learning based overtaking decision-making for highway autonomous driving[C]//6th International Conference on Intelligent Control and Information Processing. Wuhan,China:IEEE,2016:336-342.
[11] ZHENG R,LIU C M,GUO Q. A decision-making method for autonomous vehicles based on simulation and reinforcement learning[C]//2013 International Conference on Machine Learning and Cybernetics.Tianjin,China:IEEE,2013:362-369.
[12]刘全,翟建伟,章宗长,等.深度强化学习综述[J].计算机学报,2018,41(1):1-27.LIU Q,ZHAI J W,ZHANG Z Z,et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers,2018,41(1):1-27(in Chinese).
[13]赵冬斌,邵坤,朱圆恒,等.深度强化学习综述:兼论计算机围棋的发展[J].控制理论与应用,2016,33(6):701-717.ZHAO D B,SHAO K,ZHU Y H,et al. Review of deep reinforcement learning and discussions on the development of computer Go[J]. Control Theory&Applications,2016,33(6):701-717(in Chinese).
[14] SZEPESVARI C. Algorithms for reinforcement learning[J]. Synthesis Lectures on Artificial Intelligence and Machine Learning,2009,4(1):632-636.
[15] VAN HASSELT H,GUEZ A,SILVER D. Deep reinforcement learning with double q-learning[J]. Computer Science,2015,3(3):6461-6474.
[16] CHENG Y,ZHANG WD. Concise deep reinforcement learning obstacle avoidance for under actuated unmanned marine vessels[J]. Neurocomputing,2017,272:63-73.
[17]郑睿.基于增强学习的无人车辆智能决策方法研究[D].长沙:国防科学技术大学,2013.
[18]刘虎,张仲明.航行船舶安全会遇距离[J].水运管理,2015,37(10):12-13.
[2]张晓东,童剑,郭敏,等.舰船物资转运方案计算机辅助决策算法研究[J].中国舰船研究,2011,6(4):104-110.ZHANG X D,TONG J,GUO M,et al. Computer aided decision-making method for material transfer planning on ship deck[J]. Chinese Journal of Ship Research,2011,6(4):104-110(in Chinese).
[3]王斌明.基于多传感器信息融合的移动机器人避障研究[D].南京:南京理工大学,2006.
[4] TANAKA Y,JI Y,YAMASHITA A,et al. Adaptive traversability analysis and behavior generation method for mobile robots on rough terrain[J]. Journal of the Japan Society for Precision Engineering,2015,81(12):1119-1126.
[5]陈雪梅,田赓,苗一松,等.城市环境下无人驾驶车辆驾驶规则获取及决策算法[J].北京理工大学学报,2017,37(5):491-496.CHEN X M,TIAN G,MIAO Y S,et al. Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic[J]. Transactions of Beijing Institute of Technology,2017,37(5):491-496(in Chinese).
[6]杜明博.基于人类驾驶行为的无人驾驶车辆行为决策与运动规划方法研究[D].合肥:中国科学技术大学,2016.
[7]田赓.复杂动态城市环境下无人驾驶车辆仿生换道决策模型研究[D].北京:北京理工大学,2016.
[8] TEMIZER S,KOCHENDERFER M J,KAELBLING L P,et al. Collision avoidance for unmanned aircraft using Markov decision processes[C]//AIAA Guidance,Navigation,and Control Conference. Toronto,Ontario,Canada:AIAA,2010.
[9] FURDA A,VLACIC L. Enabling safe autonomous driving in real-world city traffic using multiple criteria decision making[J]. IEEE Intelligent Transportation Systems Magazine,2011,3(1):4-17.
[10] LI X,XU X,ZUO L. Reinforcement learning based overtaking decision-making for highway autonomous driving[C]//6th International Conference on Intelligent Control and Information Processing. Wuhan,China:IEEE,2016:336-342.
[11] ZHENG R,LIU C M,GUO Q. A decision-making method for autonomous vehicles based on simulation and reinforcement learning[C]//2013 International Conference on Machine Learning and Cybernetics.Tianjin,China:IEEE,2013:362-369.
[12]刘全,翟建伟,章宗长,等.深度强化学习综述[J].计算机学报,2018,41(1):1-27.LIU Q,ZHAI J W,ZHANG Z Z,et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers,2018,41(1):1-27(in Chinese).
[13]赵冬斌,邵坤,朱圆恒,等.深度强化学习综述:兼论计算机围棋的发展[J].控制理论与应用,2016,33(6):701-717.ZHAO D B,SHAO K,ZHU Y H,et al. Review of deep reinforcement learning and discussions on the development of computer Go[J]. Control Theory&Applications,2016,33(6):701-717(in Chinese).
[14] SZEPESVARI C. Algorithms for reinforcement learning[J]. Synthesis Lectures on Artificial Intelligence and Machine Learning,2009,4(1):632-636.
[15] VAN HASSELT H,GUEZ A,SILVER D. Deep reinforcement learning with double q-learning[J]. Computer Science,2015,3(3):6461-6474.
[16] CHENG Y,ZHANG WD. Concise deep reinforcement learning obstacle avoidance for under actuated unmanned marine vessels[J]. Neurocomputing,2017,272:63-73.
[17]郑睿.基于增强学习的无人车辆智能决策方法研究[D].长沙:国防科学技术大学,2013.
[18]刘虎,张仲明.航行船舶安全会遇距离[J].水运管理,2015,37(10):12-13.