基于CNN-LSTM的机器人触觉识别与自适应抓取控制
|
摘要
基于触觉进行物体识别对于机器人实现精细操作、人机交互有着重要意义。结合深度学习理论,提出了一种基于卷积神经网络(CNN)和长短期记忆(LSTM)神经网络的融合模型的机器人触觉序列识别方法,使用14种实验样品组建的触觉数据库进行了十四分类和四分类测试,分别达到了94.2%和95.0%的识别正确率;在此基础上搭建了一套结合物体在线识别的稳定抓取系统,有效地改善了机器人灵巧手的抓握效果。实验表明,对比基本卷积神经网络模型和简单长短期记忆神经网络模型,提出的融合模型对于触觉序列有更好的识别能力,并且能够实际应用于物体在线识别和稳定抓取控制。
Object recognition based on tactile is of great significance to the robotic fine operation and man-machine interaction. Combining deep learning theory, a robot tactile sequence recognition method is proposed based on the fusion model of convolutional neural network(CNN) and long short term memory(LSTM) neural network. Fourteen classification test and four classification test are conducted with the tactile dataset constructed with 14 kinds of experiment samples, and the correct recognition rates of 94.2% and 95.0% are achieved, respectively. On this basis, a stable grasping system combining object online recognition is built, which effectively improves the grasping effect of the robot dexterous hand. The experiment results show that compared with the basic convolution neural network model and simple long short term memory neural network model, the proposed fusion model has better recognition capability for the tactile sequence and can be applied to the object on-line recognition and stable grasping control practically.
Object recognition based on tactile is of great significance to the robotic fine operation and man-machine interaction. Combining deep learning theory, a robot tactile sequence recognition method is proposed based on the fusion model of convolutional neural network(CNN) and long short term memory(LSTM) neural network. Fourteen classification test and four classification test are conducted with the tactile dataset constructed with 14 kinds of experiment samples, and the correct recognition rates of 94.2% and 95.0% are achieved, respectively. On this basis, a stable grasping system combining object online recognition is built, which effectively improves the grasping effect of the robot dexterous hand. The experiment results show that compared with the basic convolution neural network model and simple long short term memory neural network model, the proposed fusion model has better recognition capability for the tactile sequence and can be applied to the object on-line recognition and stable grasping control practically.
引文
[1] 秦欢欢, 宋爱国, 莫依婷, 等. 带有双手力觉反馈的人机交互系统设计[J]. 仪器仪表学报,2018,39(7):66-73.QIN H H, SONG AI G, MO Y T, et al. Design of human-machine system with two-handed force feedback[J]. Chinese Journal of Scientific Instrument, 2018,39(7):66-73.
[2] 顾海巍. 机器人灵巧手的物体触摸识别及自适应抓取研究[D]. 哈尔滨:哈尔滨工业大学,2017.GU H W. Research on object touch recognition and adaptive grasping by robot dexterous hand[D]. Harbin: Harbin Institute of Technology, 2017.
[3] SCHNEIDER A, STURM J, STACHNISS C, et al. Object identification with tactile sensors using bag-of-features[C]. International Conference on Intelligent Robots and Systems, 2009:243- 248.
[4] 萧伟, 孙富春, 刘华平. 机器人灵巧手的触觉分析与建模[J]. 机器人, 2013, 35(4):394- 401.XIAO W, SUN F CH, LIU H P. Tactile analysis and modeling of dexterous robotic hand[J]. Robot, 2013, 35(4):394- 401.
[5] MADRY M, BO L, KRAGIC D, et al. ST-HMP: Unsupervised spatio-temporal feature learning for tactile data[C]. International Conference on Robotics and Automation,2014:2262- 2269.
[6] AGGARWAL A, KAMPMANN P, LEMBURG J, et al. Haptic object recognition in underwater and deep-sea environments[J]. Journal of Field Robotics, 2015, 32(1):167-185.
[7] WETTRLS N, LOEB G E. Haptic feature extraction from a biomimetic tactile sensor: Force, contact location and curvature[C]. International Conference on Robotics and Biomimetics,2012:2471- 2478.
[8] CHU V, MCMAHON I, RIANO L, et al. Robotic learning of haptic adjectives through physical interaction[J]. Robotics & Autonomous Systems, 2015, 63(P3):279- 292.
[9] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278- 2324.
[10] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[11] 王洋, 余祖俊, 朱力强, 等. 基于CNN的高速铁路侵限异物特征快速提取算法[J]. 仪器仪表学报, 2017, 38(5):1267-1275.WANG Y, YU Z J, ZHUO L Q, et al. Fast feature extraction algorithm for high-speed railway clearance intruding objects based on CNN[J]. Chinese Journal of Scientific Instrument, 2017, 38(5):1267-1275.
[12] 夏飞, 罗志疆, 张浩, 等. 混合神经网络在变压器故障诊断中的应用[J]. 电子测量与仪器学报, 2017, 31(1): 118-124.XIA F, LUO ZH J, ZHANG H, et al. Application of mixed neural network in transformer fault diagnosis[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31(1):118-124.
[13] 周晓彦, 王珂, 李凌燕. 基于深度学习的目标检测算法综述[J]. 电子测量技术, 2017,40(11):89-93.ZHOU X Y, WANG K, LI L Y. Review of object detection based on deep learning[J]. Electronic Measurement Technology, 2017,40(11):89-93.
[14] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Computer Science, 2015, arxiv:1511.08308.
[15] 谢逸, 饶文碧, 段鹏飞, 等. 基于CNN和LSTM混合模型的中文词性标注[J]. 武汉大学学报(理学版), 2017,63(3):246- 250.XIE Y, RAO W B, DUAN P F. A Chinese POS tagging approach using CNN and LSTM-based hybrid model[J]. Journal of Wuhan University (Natural Science Edition), 2017,63(3):246- 250.
[16] SATT A, ROZENBERG S, HOORY R. Efficient emotion recognition from speech using deep learning on spectrograms[C]. Interspeech,2017:1089-1093.
[17] NG Y H, HAUSKNECHT M, VIJAYANARASIMHAN S, et al. Beyond short snippets: Deep networks for video classification[C]. Computer Vision and Pattern Recognition, 2015:4694- 4702.
[18] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[19] 段艳杰, 吕宜生, 张杰,等. 深度学习在控制领域的研究现状与展望[J]. 自动化学报, 2016, 42(5):643- 654.DUAN Y J, LV Y SH, ZHANG J, et al. Deep learning for control: The state of the art and prospects[J]. Acta Automatica Sinica, 2016, 42(5):643- 654.
[20] 王新, 候风艳. 基于改进的PSO-BP神经网络的无刷直流电机控制[J]. 电子测量技术, 2017, 40(2):10-14.WANG X, HOU F Y. Control of brushless DC motor based on improved PSO-BP neural network[J]. Electronic Measurement Technology, 2017, 40(2):10-14.
[21] 张达敏, 林辉品, 林智勇, 等. 基于神经网络预测控制的节能电梯能量管理[J]. 仪器仪表学报, 2017,38(12): 3137-3142.ZHANG D M, LIN H P, LIN ZH Y, et al. Energy management of energy-saving elevator based on neural network predictive control[J]. Chinese Journal of Scientific Instrument, 2017,38(12):3137-3142.
[2] 顾海巍. 机器人灵巧手的物体触摸识别及自适应抓取研究[D]. 哈尔滨:哈尔滨工业大学,2017.GU H W. Research on object touch recognition and adaptive grasping by robot dexterous hand[D]. Harbin: Harbin Institute of Technology, 2017.
[3] SCHNEIDER A, STURM J, STACHNISS C, et al. Object identification with tactile sensors using bag-of-features[C]. International Conference on Intelligent Robots and Systems, 2009:243- 248.
[4] 萧伟, 孙富春, 刘华平. 机器人灵巧手的触觉分析与建模[J]. 机器人, 2013, 35(4):394- 401.XIAO W, SUN F CH, LIU H P. Tactile analysis and modeling of dexterous robotic hand[J]. Robot, 2013, 35(4):394- 401.
[5] MADRY M, BO L, KRAGIC D, et al. ST-HMP: Unsupervised spatio-temporal feature learning for tactile data[C]. International Conference on Robotics and Automation,2014:2262- 2269.
[6] AGGARWAL A, KAMPMANN P, LEMBURG J, et al. Haptic object recognition in underwater and deep-sea environments[J]. Journal of Field Robotics, 2015, 32(1):167-185.
[7] WETTRLS N, LOEB G E. Haptic feature extraction from a biomimetic tactile sensor: Force, contact location and curvature[C]. International Conference on Robotics and Biomimetics,2012:2471- 2478.
[8] CHU V, MCMAHON I, RIANO L, et al. Robotic learning of haptic adjectives through physical interaction[J]. Robotics & Autonomous Systems, 2015, 63(P3):279- 292.
[9] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278- 2324.
[10] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[11] 王洋, 余祖俊, 朱力强, 等. 基于CNN的高速铁路侵限异物特征快速提取算法[J]. 仪器仪表学报, 2017, 38(5):1267-1275.WANG Y, YU Z J, ZHUO L Q, et al. Fast feature extraction algorithm for high-speed railway clearance intruding objects based on CNN[J]. Chinese Journal of Scientific Instrument, 2017, 38(5):1267-1275.
[12] 夏飞, 罗志疆, 张浩, 等. 混合神经网络在变压器故障诊断中的应用[J]. 电子测量与仪器学报, 2017, 31(1): 118-124.XIA F, LUO ZH J, ZHANG H, et al. Application of mixed neural network in transformer fault diagnosis[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31(1):118-124.
[13] 周晓彦, 王珂, 李凌燕. 基于深度学习的目标检测算法综述[J]. 电子测量技术, 2017,40(11):89-93.ZHOU X Y, WANG K, LI L Y. Review of object detection based on deep learning[J]. Electronic Measurement Technology, 2017,40(11):89-93.
[14] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Computer Science, 2015, arxiv:1511.08308.
[15] 谢逸, 饶文碧, 段鹏飞, 等. 基于CNN和LSTM混合模型的中文词性标注[J]. 武汉大学学报(理学版), 2017,63(3):246- 250.XIE Y, RAO W B, DUAN P F. A Chinese POS tagging approach using CNN and LSTM-based hybrid model[J]. Journal of Wuhan University (Natural Science Edition), 2017,63(3):246- 250.
[16] SATT A, ROZENBERG S, HOORY R. Efficient emotion recognition from speech using deep learning on spectrograms[C]. Interspeech,2017:1089-1093.
[17] NG Y H, HAUSKNECHT M, VIJAYANARASIMHAN S, et al. Beyond short snippets: Deep networks for video classification[C]. Computer Vision and Pattern Recognition, 2015:4694- 4702.
[18] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[19] 段艳杰, 吕宜生, 张杰,等. 深度学习在控制领域的研究现状与展望[J]. 自动化学报, 2016, 42(5):643- 654.DUAN Y J, LV Y SH, ZHANG J, et al. Deep learning for control: The state of the art and prospects[J]. Acta Automatica Sinica, 2016, 42(5):643- 654.
[20] 王新, 候风艳. 基于改进的PSO-BP神经网络的无刷直流电机控制[J]. 电子测量技术, 2017, 40(2):10-14.WANG X, HOU F Y. Control of brushless DC motor based on improved PSO-BP neural network[J]. Electronic Measurement Technology, 2017, 40(2):10-14.
[21] 张达敏, 林辉品, 林智勇, 等. 基于神经网络预测控制的节能电梯能量管理[J]. 仪器仪表学报, 2017,38(12): 3137-3142.ZHANG D M, LIN H P, LIN ZH Y, et al. Energy management of energy-saving elevator based on neural network predictive control[J]. Chinese Journal of Scientific Instrument, 2017,38(12):3137-3142.