基于B-Spline曲线的流式数据事件模板构建方法
|
摘要
流式数据处理系统中常常要提取出事件的模板,进而针对将来发生在流式数据上的事件做预测分析处理.针对目前的流式数据系统中存在的事件模板的构建过程计算量过大,使用的数据节点较多,误差较大等问题,文本提出一种基于B-Spline曲线的流式数据事件模板构建方法.该方法首先给出了流式数据上的事件和事件模板的定义,在此基础上确定了基本尺度事件,基于该事件给出了基于线性变换变的流式数据事件的归一化处理方法.其次,本文提出使用B-Spline曲线来进行事件模板的拟合,采用均匀的节点矢量,通过遗传算法求解B-Spline的控制节点.实验表明,本文提出的方法能有效的减小事件模板构建过程中存在的计算量过大,使用数据节点较多、误差较大等问题,具有较高的可用性.
Streaming data processing systems often have to extract event templates,and then perform predictive analysis on future events that occur on streaming data. For the current streaming data system,there are some problems such as too much computation in the construction process of the event template,many data nodes used,and large errors. The page proposes a B-Spline curve-based flow big data event template construction method. The method first gives the definition of event and event templates on streaming data. On this basis,the basic scale events are determined,and a normalized processing method for streaming big data events is given based on this event. Finally,this paper proposes the use of B-Spline curve to fit the event template,adopts uniform node vector,and solves the control node of B-Spline through genetic algorithm. The Experiments show that the method proposed in this paper can effectively reduce the problem of excessive computation,more data nodes used,larger errors in the process of event template construction and higher availability.
Streaming data processing systems often have to extract event templates,and then perform predictive analysis on future events that occur on streaming data. For the current streaming data system,there are some problems such as too much computation in the construction process of the event template,many data nodes used,and large errors. The page proposes a B-Spline curve-based flow big data event template construction method. The method first gives the definition of event and event templates on streaming data. On this basis,the basic scale events are determined,and a normalized processing method for streaming big data events is given based on this event. Finally,this paper proposes the use of B-Spline curve to fit the event template,adopts uniform node vector,and solves the control node of B-Spline through genetic algorithm. The Experiments show that the method proposed in this paper can effectively reduce the problem of excessive computation,more data nodes used,larger errors in the process of event template construction and higher availability.
引文
[1]Zhang Xiang-yang,Xie Guang-xiang.Analysis and discussion on influencing factors of coal mine dynamic disaster[J].Safety in Coal M ines,2010,41(11):108-110.
[2]Dou L,Mu Z,Li,et al.Research progress of monitoring,forecasting,and prevention of rockburst in underground coal mining in China[J].International Journal of Coal Science&Technology,2014,1(3):278-288.
[3]Xue Guang-hui,Zhao Xin-ying,Liu Ermeng,et al.Time-domain characteristic extraction of coal and rock vibration signal in fullymechanized top coal caving face[J].Coal Science and Technology,2015,43(12):92-97.
[4]Chai Yu-mei,Zhou Chi,Wang Li-ming.Detecting concept drift and classifying data streams[J].Journal of Chinese Computer Systems,2011,32(3):421-425.
[5]Li Yan-hong,Mu Guo-wang,Guo zeng.Application of genetic algorithm to determine shape parameters ofβ-spline curves[J].Computer Engineering and Applications,2011,47(9):175-177.
[6]Mu Guo-wang,Zang Ting,Zhao Gang.Using the modified genetic algorithm to determine the knots of B-spline curves[J].Computer Engineering and Applications,2006,42(11):88-90.
[7]Zhou Ming,Sun Shu-dong.Genetic algorithms:theory and applications[M].Beijing:National Defense Industry Press,1999.
[8]Han Jiang,Jiang Ben-chi,Xia Lian,et al.A B-Spline Curve fitting algorithm based on control key points[J].Applied M athematics and M echanics,2015,36(4):423-431.
[9]Xu Zhen-jia.Research on parallel clustering algorithm for streaming data[D].Qufu:Qufu Normal University,2015.
[10]Hu Yu,Yan Qiao-mei.Optimal algorithm of data streams clustering on sliding windowmodel[J].Computer Applications,2008,28(6):1414-1416.
[11]Hou Dong-feng,Liu Qing-bao,Zhang Wei-ming,et al.Adaptive method of computing data stream aggregation[J].Computer Science,2010,37(3):152-155.
[12]Wu Guo-ping.Research on real-time anomaly detection of massive log streams based on DM E cluster analysis model[D].Hangzhou:Hangzhou Dianzi University,2017.
[13]Sun Yue-hong,Wei Jian-xiang,Xia De-shen.Parameter optimization for B-spline curve fitting based on adaptive genetic algorithm[J].Journal of Computer Applications,2010,30(7):1878-1882.
[14]Ren Zhong,Yang Can-jun,Chen Ying.B-spline interpolation algorithm in motion planning[J].M echanical&Electrical Engineering M agazine,2001,18(5):38-39.
[15]JIANG An-nan.Forecast of rock burst based on pattern recognition by least square support vector machine[C].Chinese Journal of Rock M echanics and Engineering,2005.
[16]Ge Zhi-hui,Xiao Fu-kun,Duan Li-qun,et al.Prediction of risk grade of impulsion pressure[J].Coal Technology.2010,29(11):100-102.
[17]Jiang Yao-dong,Pan Yi-shan,Jiang Fu-xing,et al.State of the art reviewon mechanism and prevention of coal bumps in China[J].Journal of China Coal Society,2014,39(2):205-213.
[18]Dou Lin-ming,Gong Si-yuan,Liu Peng,et al.Remote online early warning platform of mine seismic and bump disaster[J].Coal Science and Technology,2015,43(6):48-53.
[19]Jiang Shao-jun.Machine learning algorithm supported by large data[J].Computer knowledge and Technology,2018,14(7):164-165.
[20]Ding Shi-fei,Qi Bing-juan,Tan Hong-yan.An overviewon theory and algorithm of support vector machine[J].Journal of University of Electronic Science and Technology in China.2011,40(1):2-10.
[21]Ping Yuan.Research on clustering and text classification based on support vector machine[D].Beijing University of Posts and Telecomunications,2012.
[5]李彦红,穆国旺,郭增.用遗传算法确定三次β样条曲线的形状参数[J].计算机工程与应用,2011,47(9):175-177.
[6]穆国旺,臧婷,赵罡.用改进遗传算法确定B-SPLINE曲线的节点矢量[J].计算机工程与应用,2006,42(11):88-90.
[7]周明,孙树栋编著,遗传算法原理及应用[M].北京:国防工业出版社,1999.
[8]韩江,江本赤,夏链,李大柱.基于轮廓关键点的B-SPLINE曲线拟合算法[J].应用数学和力学,2015,36(4):423-431.
[9]许振佳.流式数据的并行聚类算法研究[D].曲阜:曲阜师范大学,2015.
[10]胡彧,闫巧梅.滑动窗口模型下的优化数据流聚类算法[J].计算机应用,2008,28(6):1414-1416.
[11]侯东风,刘青宝,张维明,等.一种适应性的流式数据聚集计算方法[J].计算机科学,2010,37(3):152-155.
[12]武国平.基于DME聚类分析模型的海量日志流实时异常检测研究[D].杭州:杭州电子科技大学,2017.
[13]孙越泓,魏建香,夏德深.基于自适应遗传算法的B-SPLINE曲线拟合的参数优化[J].计算机应用,2010,30(7):1878-1882.
[14]任重,杨灿军,陈鹰.轨迹规划中的B-SPLINE插值算法[J].机电工程,2001,18(5):38-39.
[15]姜谙男.冲击地压危险性预测的最小二乘支持向量机模式识别[C].第九届全国岩石动力学学术会议论文集,2005.
[16]葛志会,肖福坤,段立群,等.冲击矿压危险等级预测[J].煤炭技术,2010,29(11):100-102.
[17]姜耀东,潘一山,姜福兴,等.我国煤炭开采中的冲击地压机理和防治[J].煤炭学报,2014,39(2):205-213.
[18]窦林名,巩思园,刘鹏,等.矿震冲击灾害远程在线预警平台[J].煤炭科学技术,2015,43(6):48-53.
[19]姜绍俊.大数据支撑下的机器学习算法[J].电脑知识与技术,2018,14(7):164-165.
[20]丁世飞,齐丙娟,谭红艳.支持向量机理论与算法研究综述[J].电子科技大学学报,2011,40(1):2-10.
[21]平源.基于支持向量机的聚类及文本分类研究[D].北京:北京邮电大学,2012.
[2]Dou L,Mu Z,Li,et al.Research progress of monitoring,forecasting,and prevention of rockburst in underground coal mining in China[J].International Journal of Coal Science&Technology,2014,1(3):278-288.
[3]Xue Guang-hui,Zhao Xin-ying,Liu Ermeng,et al.Time-domain characteristic extraction of coal and rock vibration signal in fullymechanized top coal caving face[J].Coal Science and Technology,2015,43(12):92-97.
[4]Chai Yu-mei,Zhou Chi,Wang Li-ming.Detecting concept drift and classifying data streams[J].Journal of Chinese Computer Systems,2011,32(3):421-425.
[5]Li Yan-hong,Mu Guo-wang,Guo zeng.Application of genetic algorithm to determine shape parameters ofβ-spline curves[J].Computer Engineering and Applications,2011,47(9):175-177.
[6]Mu Guo-wang,Zang Ting,Zhao Gang.Using the modified genetic algorithm to determine the knots of B-spline curves[J].Computer Engineering and Applications,2006,42(11):88-90.
[7]Zhou Ming,Sun Shu-dong.Genetic algorithms:theory and applications[M].Beijing:National Defense Industry Press,1999.
[8]Han Jiang,Jiang Ben-chi,Xia Lian,et al.A B-Spline Curve fitting algorithm based on control key points[J].Applied M athematics and M echanics,2015,36(4):423-431.
[9]Xu Zhen-jia.Research on parallel clustering algorithm for streaming data[D].Qufu:Qufu Normal University,2015.
[10]Hu Yu,Yan Qiao-mei.Optimal algorithm of data streams clustering on sliding windowmodel[J].Computer Applications,2008,28(6):1414-1416.
[11]Hou Dong-feng,Liu Qing-bao,Zhang Wei-ming,et al.Adaptive method of computing data stream aggregation[J].Computer Science,2010,37(3):152-155.
[12]Wu Guo-ping.Research on real-time anomaly detection of massive log streams based on DM E cluster analysis model[D].Hangzhou:Hangzhou Dianzi University,2017.
[13]Sun Yue-hong,Wei Jian-xiang,Xia De-shen.Parameter optimization for B-spline curve fitting based on adaptive genetic algorithm[J].Journal of Computer Applications,2010,30(7):1878-1882.
[14]Ren Zhong,Yang Can-jun,Chen Ying.B-spline interpolation algorithm in motion planning[J].M echanical&Electrical Engineering M agazine,2001,18(5):38-39.
[15]JIANG An-nan.Forecast of rock burst based on pattern recognition by least square support vector machine[C].Chinese Journal of Rock M echanics and Engineering,2005.
[16]Ge Zhi-hui,Xiao Fu-kun,Duan Li-qun,et al.Prediction of risk grade of impulsion pressure[J].Coal Technology.2010,29(11):100-102.
[17]Jiang Yao-dong,Pan Yi-shan,Jiang Fu-xing,et al.State of the art reviewon mechanism and prevention of coal bumps in China[J].Journal of China Coal Society,2014,39(2):205-213.
[18]Dou Lin-ming,Gong Si-yuan,Liu Peng,et al.Remote online early warning platform of mine seismic and bump disaster[J].Coal Science and Technology,2015,43(6):48-53.
[19]Jiang Shao-jun.Machine learning algorithm supported by large data[J].Computer knowledge and Technology,2018,14(7):164-165.
[20]Ding Shi-fei,Qi Bing-juan,Tan Hong-yan.An overviewon theory and algorithm of support vector machine[J].Journal of University of Electronic Science and Technology in China.2011,40(1):2-10.
[21]Ping Yuan.Research on clustering and text classification based on support vector machine[D].Beijing University of Posts and Telecomunications,2012.
[5]李彦红,穆国旺,郭增.用遗传算法确定三次β样条曲线的形状参数[J].计算机工程与应用,2011,47(9):175-177.
[6]穆国旺,臧婷,赵罡.用改进遗传算法确定B-SPLINE曲线的节点矢量[J].计算机工程与应用,2006,42(11):88-90.
[7]周明,孙树栋编著,遗传算法原理及应用[M].北京:国防工业出版社,1999.
[8]韩江,江本赤,夏链,李大柱.基于轮廓关键点的B-SPLINE曲线拟合算法[J].应用数学和力学,2015,36(4):423-431.
[9]许振佳.流式数据的并行聚类算法研究[D].曲阜:曲阜师范大学,2015.
[10]胡彧,闫巧梅.滑动窗口模型下的优化数据流聚类算法[J].计算机应用,2008,28(6):1414-1416.
[11]侯东风,刘青宝,张维明,等.一种适应性的流式数据聚集计算方法[J].计算机科学,2010,37(3):152-155.
[12]武国平.基于DME聚类分析模型的海量日志流实时异常检测研究[D].杭州:杭州电子科技大学,2017.
[13]孙越泓,魏建香,夏德深.基于自适应遗传算法的B-SPLINE曲线拟合的参数优化[J].计算机应用,2010,30(7):1878-1882.
[14]任重,杨灿军,陈鹰.轨迹规划中的B-SPLINE插值算法[J].机电工程,2001,18(5):38-39.
[15]姜谙男.冲击地压危险性预测的最小二乘支持向量机模式识别[C].第九届全国岩石动力学学术会议论文集,2005.
[16]葛志会,肖福坤,段立群,等.冲击矿压危险等级预测[J].煤炭技术,2010,29(11):100-102.
[17]姜耀东,潘一山,姜福兴,等.我国煤炭开采中的冲击地压机理和防治[J].煤炭学报,2014,39(2):205-213.
[18]窦林名,巩思园,刘鹏,等.矿震冲击灾害远程在线预警平台[J].煤炭科学技术,2015,43(6):48-53.
[19]姜绍俊.大数据支撑下的机器学习算法[J].电脑知识与技术,2018,14(7):164-165.
[20]丁世飞,齐丙娟,谭红艳.支持向量机理论与算法研究综述[J].电子科技大学学报,2011,40(1):2-10.
[21]平源.基于支持向量机的聚类及文本分类研究[D].北京:北京邮电大学,2012.