基于KF-LSTM模型的手写数字轨迹的sEMG重建算法
|
摘要
为了从神经肌肉活动中有效地重建出手写轨迹,提出一种卡尔曼滤波器与长短期记忆网络深度融合的混合模型(KF-LSTM),对手写数字轨迹坐标映射的表面肌电(sEMG)信号进行训练与解码.招募5名被试,设计了组间实验和组内实验方案,同步采集手写过程中的sEMG和轨迹坐标,构建基于KF-LSTM的手写轨迹预测模型;以决定系数和主观可辨认度作为评价指标,分别与LSTM模型、浅层神经网络(NN)模型以及KF模型的重建结果进行比较.实验结果表明,KF-LSTM模型在组间实验及组内实验中的表现均高于其他3种方法,能有效地提升重建精度,提高重建轨迹的光顺度.
For the purpose of reconstructing handwriting traces from neuromuscular activities effectively, a well-integrated Kalman filter modified long-short-term memory network hybrid method(KF-LSTM) is proposed, which can train and decode the sEMG(surface electromyography) signals to the corresponding coordinates of handwriting numeral traces. Five participants were recruited for the between-group and within-group experiments. After synchronously collecting the sEMG signals and coordinates in the handwriting process, the KF-LSTM prediction models were constructed. The decision coefficient and the subjective identifiability were calculated as the evaluation indices. The performance of the KF-LSTM models was compared with the LSTM models, the NN(neural network) models and the KF(Kalman filter) models. The experiment results show that the proposed KF-LSTM method perform better than the other 3 methods, improve the reconstruction accuracies and make reconstructed traces much smoother.
For the purpose of reconstructing handwriting traces from neuromuscular activities effectively, a well-integrated Kalman filter modified long-short-term memory network hybrid method(KF-LSTM) is proposed, which can train and decode the sEMG(surface electromyography) signals to the corresponding coordinates of handwriting numeral traces. Five participants were recruited for the between-group and within-group experiments. After synchronously collecting the sEMG signals and coordinates in the handwriting process, the KF-LSTM prediction models were constructed. The decision coefficient and the subjective identifiability were calculated as the evaluation indices. The performance of the KF-LSTM models was compared with the LSTM models, the NN(neural network) models and the KF(Kalman filter) models. The experiment results show that the proposed KF-LSTM method perform better than the other 3 methods, improve the reconstruction accuracies and make reconstructed traces much smoother.
引文
[1]Engels F.The role of labor in the process if transition from ape to man[M]//Cao Baohua,Yu Guangyuan.Beijing:People’s Publishing House,1949(in Chinese)(Engels F.劳动在从猿到人转变过程中的作用[M].曹葆华,于光远,译.北京:人民出版社,1949)
[2]Chen Zhongwei.Reconstruction of hand defects[J].Chinese Journal of Orthopedic Trauma,2002,4(4):241-243(in Chinese)(陈中伟.手缺损的重建[J].中华创伤骨科杂志,2002,4(4):241-243)
[3]Zhang Xu.Body gesture recognition and interaction based on surface electromyogram[D].Hefei:University of Science and Technology of China.Department of Electronic Science and Technology,2010(in Chinese)(张旭.基于表面肌电信号的人体动作识别与交互[D].合肥:中国科学技术大学电子科学与技术系,2010)
[4]Wang Jie,Han Ding,Gao Zhongbao,et al.Role of electronic pen in diagnosis of Parkinson’s disease and assessment of early and intermediate motor symptom[J].Chinese Journal of Geriatric Heart Brain and Vessel Diseases,2017,19(5):496-502(in Chinese)(王洁,韩丁,高中宝,等.手写笔在帕金森病诊断及早中期运动障碍评估中的作用研究[J].中华老年心脑血管病杂志,2017,19(5):496-502)
[5]Caligiuri M P,Teulings H L,Dean C E,et al.Handwriting movement kinematics for quantifying extrapyramidal side effects in patients treated with atypical antipsychotics[J].Psychiatry Research,2010,177:77-83
[6]Chowdhury A,Ramadas R,Karmakar S.Muscle computer interface:a review[C]//Proceedings of ICoRD’13.Heidelberg:Springer,2013:411-421
[7]Jin Lixin,Yu Fangheng,Shen Haibin.Posture correction algorithm in air handwriting recognition[J].Electronic Technology,2015,44(3):9-12(in Chinese)(金立新,余昉恒,沈海斌.空中手写识别的姿态矫正算法研究[J].电子技术,2015,44(3):9-12)
[8]Farina D,Jiang N,Rehbaum H,et al.The extraction of neural information from the surface EMG for the control of upper-limb prostheses:emerging avenues and challenges[J].IEEETransactions on Neural Systems and Rehabilitation Engineering,2014,22(4):797-809
[9]Okorokova E,Lebedev M,Linderman M,et al.A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings[J].Frontiers in Neuroscience,2015,9:Article No.389
[10]Chen Y M,Yang Z L.A novel hybrid model for drawing trace reconstruction from multichannel surface electromyographic activity[J].Frontiers in Neuroscience,2017,11:Article No.61
[11]Yang Zhongliang,Wang Jian,Chen Yumiao.Surface EMGbased emotion recognition model for body language of head movements[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(9):1396-1402(in Chinese)(杨钟亮,王健,陈育苗.基于表面肌电的头部运动体态语言情感识别模型[J].计算机辅助设计与图形学学报,2014,26(9):1396-1402)
[12]Ren Tingting.Research on the methods and application of handwriting movement analysis in children[D].Hefei:University of Science and Technology of China.School of Nuclear Science and Technology,2017(in Chinese)(任亭亭.儿童手写运动分析方法及其应用研究[D].合肥:中国科学技术大学核科学技术学院,2017)
[13]Ellis A W.Normal writing processes and peripheral acquired dysgraphias[J].Language and Cognitive Processes,1988,3(2):99-127
[14]Sueiras J,Ruiz V,Sanchez A,et al.Offline continuous handwriting recognition using sequence to sequence neural networks[J].Neurocomputing,2018,289:119-128
[15]Simonnet D,Anquetil E,Bouillon M.Multi-criteria handwriting quality analysis with online fuzzy models[J].Pattern Recognition,2017,69:310-324
[16]Xin Yizhong,Jiang Xinhui,Li Yang,et al.Multi-input of touch,pen and mid-air gesture[J].Journal of Computer-Aided Design&Computer Graphics,2018,30(5):957-967(in Chinese)(辛义忠,姜欣慧,李洋,等.触控、笔和空间手势的混合输入[J].计算机辅助设计与图形学学报,2018,30(5):957-967)
[17]Linderman M,Lebedev M A,Erlichman J S.Recognition of handwriting from electromyography[J].PLoS ONE,2009,4(8):e6791
[18]Yang Z L,Chen Y M.Surface EMG-based sketching recognition using two analysis windows and gene expression programming[J].Frontiers in Neuroscience,2016,10:Article No.445
[19]Chen Y M,Yang Z L,Gong H,et al.Recognition of sketching from surface electromyography[J].Neural Computing and Applications,2018,30(9):2725-2737
[20]Xia P,Hu J,Peng Y H.EMG-based estimation of limb movement using deep learning with recurrent convolutional neural networks[J].Artificial Organs,2018,42(5):E67-E77
[21]Chen J C,Zhang X D,Cheng Y,et al.Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks[J].Biomedical Signal Processing and Control,2018,40:335-342
[22]Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780
[23]Kalman R E,Bucy R S.New results in linear filtering and prediction theory[J].Journal of Basic Engineering,1961,83(1):95-108
[24]Turing A M.Computing machinery and intelligence[J].Mind,1950,59(236):433-460
[25]Yang Z L,Wen Y L,Chen Y M.sEMG-based drawing trace reconstruction:a novel hybrid algorithm fusing gene expression programming into Kalman filter[J].Sensors,2018,18:Article No.3296
[2]Chen Zhongwei.Reconstruction of hand defects[J].Chinese Journal of Orthopedic Trauma,2002,4(4):241-243(in Chinese)(陈中伟.手缺损的重建[J].中华创伤骨科杂志,2002,4(4):241-243)
[3]Zhang Xu.Body gesture recognition and interaction based on surface electromyogram[D].Hefei:University of Science and Technology of China.Department of Electronic Science and Technology,2010(in Chinese)(张旭.基于表面肌电信号的人体动作识别与交互[D].合肥:中国科学技术大学电子科学与技术系,2010)
[4]Wang Jie,Han Ding,Gao Zhongbao,et al.Role of electronic pen in diagnosis of Parkinson’s disease and assessment of early and intermediate motor symptom[J].Chinese Journal of Geriatric Heart Brain and Vessel Diseases,2017,19(5):496-502(in Chinese)(王洁,韩丁,高中宝,等.手写笔在帕金森病诊断及早中期运动障碍评估中的作用研究[J].中华老年心脑血管病杂志,2017,19(5):496-502)
[5]Caligiuri M P,Teulings H L,Dean C E,et al.Handwriting movement kinematics for quantifying extrapyramidal side effects in patients treated with atypical antipsychotics[J].Psychiatry Research,2010,177:77-83
[6]Chowdhury A,Ramadas R,Karmakar S.Muscle computer interface:a review[C]//Proceedings of ICoRD’13.Heidelberg:Springer,2013:411-421
[7]Jin Lixin,Yu Fangheng,Shen Haibin.Posture correction algorithm in air handwriting recognition[J].Electronic Technology,2015,44(3):9-12(in Chinese)(金立新,余昉恒,沈海斌.空中手写识别的姿态矫正算法研究[J].电子技术,2015,44(3):9-12)
[8]Farina D,Jiang N,Rehbaum H,et al.The extraction of neural information from the surface EMG for the control of upper-limb prostheses:emerging avenues and challenges[J].IEEETransactions on Neural Systems and Rehabilitation Engineering,2014,22(4):797-809
[9]Okorokova E,Lebedev M,Linderman M,et al.A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings[J].Frontiers in Neuroscience,2015,9:Article No.389
[10]Chen Y M,Yang Z L.A novel hybrid model for drawing trace reconstruction from multichannel surface electromyographic activity[J].Frontiers in Neuroscience,2017,11:Article No.61
[11]Yang Zhongliang,Wang Jian,Chen Yumiao.Surface EMGbased emotion recognition model for body language of head movements[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(9):1396-1402(in Chinese)(杨钟亮,王健,陈育苗.基于表面肌电的头部运动体态语言情感识别模型[J].计算机辅助设计与图形学学报,2014,26(9):1396-1402)
[12]Ren Tingting.Research on the methods and application of handwriting movement analysis in children[D].Hefei:University of Science and Technology of China.School of Nuclear Science and Technology,2017(in Chinese)(任亭亭.儿童手写运动分析方法及其应用研究[D].合肥:中国科学技术大学核科学技术学院,2017)
[13]Ellis A W.Normal writing processes and peripheral acquired dysgraphias[J].Language and Cognitive Processes,1988,3(2):99-127
[14]Sueiras J,Ruiz V,Sanchez A,et al.Offline continuous handwriting recognition using sequence to sequence neural networks[J].Neurocomputing,2018,289:119-128
[15]Simonnet D,Anquetil E,Bouillon M.Multi-criteria handwriting quality analysis with online fuzzy models[J].Pattern Recognition,2017,69:310-324
[16]Xin Yizhong,Jiang Xinhui,Li Yang,et al.Multi-input of touch,pen and mid-air gesture[J].Journal of Computer-Aided Design&Computer Graphics,2018,30(5):957-967(in Chinese)(辛义忠,姜欣慧,李洋,等.触控、笔和空间手势的混合输入[J].计算机辅助设计与图形学学报,2018,30(5):957-967)
[17]Linderman M,Lebedev M A,Erlichman J S.Recognition of handwriting from electromyography[J].PLoS ONE,2009,4(8):e6791
[18]Yang Z L,Chen Y M.Surface EMG-based sketching recognition using two analysis windows and gene expression programming[J].Frontiers in Neuroscience,2016,10:Article No.445
[19]Chen Y M,Yang Z L,Gong H,et al.Recognition of sketching from surface electromyography[J].Neural Computing and Applications,2018,30(9):2725-2737
[20]Xia P,Hu J,Peng Y H.EMG-based estimation of limb movement using deep learning with recurrent convolutional neural networks[J].Artificial Organs,2018,42(5):E67-E77
[21]Chen J C,Zhang X D,Cheng Y,et al.Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks[J].Biomedical Signal Processing and Control,2018,40:335-342
[22]Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780
[23]Kalman R E,Bucy R S.New results in linear filtering and prediction theory[J].Journal of Basic Engineering,1961,83(1):95-108
[24]Turing A M.Computing machinery and intelligence[J].Mind,1950,59(236):433-460
[25]Yang Z L,Wen Y L,Chen Y M.sEMG-based drawing trace reconstruction:a novel hybrid algorithm fusing gene expression programming into Kalman filter[J].Sensors,2018,18:Article No.3296