窗口竞争性自适应重加权采样策略的近红外特征变量选择方法
|
摘要
通过消除光谱中的冗余信息变量,挑选出代表样品性质的特征变量代替全谱建立定量模型,可以提高近红外分析结果的准确性。基于进化论中适者生存原理的竞争性自适应重加权采样(CARS)算法因具有计算速度快、筛选得到的特征波长少等优点,在近红外特征变量筛选方面得到了广泛的应用。然而该方法在计算过程中容易出现校正集和验证集结果不一致情况。这是因为算法过于强调校正集交叉验证结果,且并未考虑相邻变量之间的协同作用。为了建立更加稳健的变量筛选方法,通过结合"窗口"以及CARS算法的优势,提出了一种基于窗口竞争性自适应重加权采样(WCARS)策略的近红外特征变量筛选方法,并将其应用于复杂植物样品近红外光谱与其化学成分含量之间的建模分析。采用WCARS方法可以实现准确定量分析,且通过与竞争性自适应重加权采样(CARS)方法结果相比较, WCARS方法得到的校正集和预测集结果一致,在一定程度上减少了过拟合问题的出现。该策略能有效增强特征变量选择的稳健性,提高了定量模型的可信度,具有一定的应用价值。
Variable selection plays an important role in the quantitative analysis of near infrared spectra. The accuracy of near infrared spectroscopy can be improved by eliminating the redundant variables and selecting the characteristic variables. Competitive adaptive reweighted sampling(CARS) method is a newly developed strategy for wavelength selection by employing the principle "survival of the fittest" on which Darwin's Evolution Theory is based. The number of selected wavelengths by CARS is much smaller than those of other methods with fast calculating speed and high accuracy. However, it is easy to get inconsistent results between the calibration and validation set due to the excessive attention on the cross validation results. In order to develop a robust variable selection method, by combining the advantages of CARS and "window", a new tactic called window competitive adaptive reweighted sampling(WCARS) is employed to select characteristic variables and applied to the analysis of the near infrared spectra of the complex plant samples and the contents of the chemical components. Compared with the results of CARS method, accurate quantitative results can be obtained by the WCARS method. Furthermore, the results of correction set are consistent with those of the prediction set, and the problem of overfitting can be avoided. The results show that WCARS tactic can efficiently improve the accuracy and stability of variables selection and optimize the precision of prediction model, which has a certain application value.
Variable selection plays an important role in the quantitative analysis of near infrared spectra. The accuracy of near infrared spectroscopy can be improved by eliminating the redundant variables and selecting the characteristic variables. Competitive adaptive reweighted sampling(CARS) method is a newly developed strategy for wavelength selection by employing the principle "survival of the fittest" on which Darwin's Evolution Theory is based. The number of selected wavelengths by CARS is much smaller than those of other methods with fast calculating speed and high accuracy. However, it is easy to get inconsistent results between the calibration and validation set due to the excessive attention on the cross validation results. In order to develop a robust variable selection method, by combining the advantages of CARS and "window", a new tactic called window competitive adaptive reweighted sampling(WCARS) is employed to select characteristic variables and applied to the analysis of the near infrared spectra of the complex plant samples and the contents of the chemical components. Compared with the results of CARS method, accurate quantitative results can be obtained by the WCARS method. Furthermore, the results of correction set are consistent with those of the prediction set, and the problem of overfitting can be avoided. The results show that WCARS tactic can efficiently improve the accuracy and stability of variables selection and optimize the precision of prediction model, which has a certain application value.
引文
[1] ZHANG Yi-ting,WANG Cui-cui,FAN Meng-li,et al(张伊挺,王翠翠,樊梦丽,等).Spectroscopy and Spectral Analysis (光谱学与光谱分析),2017,36(12):4100.
[2] Wang C C,Cai W S,Shao X G.Analytical Letters,2017,51(4):537.
[3] Wei H Y,Li H,Liu P,et al.Spectroscopy Letters,2017,50(7):470.
[4] CHU Xiao-li,YUAN Hong-fu,LU Wan-zhen(褚小立,袁洪福,陆婉珍).Progress Chemistry(化学进展),2004,16(4):528.
[5] Tan Z,Lou T T,Huang Z X,et al.Journal of Agricultural and Food Chemistry,2017,65:6274.
[6] Han X,Huang Z X,Chen X D,et al.Fuel,2017,207:146.
[7] Zhang C X,Bian X H,Liu P,et al.Chemometrics and Intelligent Laboratory Systems,2016,161:43.
[8] Saudland A,Wagner J,Nielsen J P,et al.Applied Spectroscopy,2007,54(3):413.
[9] Ranzan C,Trierweiler L F,Hitzmann B,et al.Chemometrics and Intelligent Laboratory Systems,2015,142:78.
[10] Li H D,Liang Y Z,Xu Q S,et al.Analytica Chimica Acta,2009,648(1):77.
[11] Cai W S,Li Y K,Shao X G.Chemometrics and Intelligent Laboratory Systems,2008,90(2):188.
[12] Xu H,Liu Z C,Cai W S,et al.Chemometrics and Intelligent Laboratory Systems,2009,97(2):189.
[13] Han X,Tan Z,Huang Z X,et al.Analytical Methods 2017,9:3720.
[14] Nie L X,Dai Z,Ma S C.Analytical Letters,2016,49(14):2259.
[15] Wu L J,Wang B X,Yin Y F,et al.Analytical Letters,2016,49(14):2290.
[16] Li H D,Xu Q S,Liang Y Z.Chemometrics and Intelligent Laboratory Systems,2018,176:34.
[2] Wang C C,Cai W S,Shao X G.Analytical Letters,2017,51(4):537.
[3] Wei H Y,Li H,Liu P,et al.Spectroscopy Letters,2017,50(7):470.
[4] CHU Xiao-li,YUAN Hong-fu,LU Wan-zhen(褚小立,袁洪福,陆婉珍).Progress Chemistry(化学进展),2004,16(4):528.
[5] Tan Z,Lou T T,Huang Z X,et al.Journal of Agricultural and Food Chemistry,2017,65:6274.
[6] Han X,Huang Z X,Chen X D,et al.Fuel,2017,207:146.
[7] Zhang C X,Bian X H,Liu P,et al.Chemometrics and Intelligent Laboratory Systems,2016,161:43.
[8] Saudland A,Wagner J,Nielsen J P,et al.Applied Spectroscopy,2007,54(3):413.
[9] Ranzan C,Trierweiler L F,Hitzmann B,et al.Chemometrics and Intelligent Laboratory Systems,2015,142:78.
[10] Li H D,Liang Y Z,Xu Q S,et al.Analytica Chimica Acta,2009,648(1):77.
[11] Cai W S,Li Y K,Shao X G.Chemometrics and Intelligent Laboratory Systems,2008,90(2):188.
[12] Xu H,Liu Z C,Cai W S,et al.Chemometrics and Intelligent Laboratory Systems,2009,97(2):189.
[13] Han X,Tan Z,Huang Z X,et al.Analytical Methods 2017,9:3720.
[14] Nie L X,Dai Z,Ma S C.Analytical Letters,2016,49(14):2259.
[15] Wu L J,Wang B X,Yin Y F,et al.Analytical Letters,2016,49(14):2290.
[16] Li H D,Xu Q S,Liang Y Z.Chemometrics and Intelligent Laboratory Systems,2018,176:34.