显隐信息协同的多视角极限学习模糊系统
|
摘要
多视角数据正在越来越多地应用于各种建模任务,但当前的多视角模糊系统建模方法,主要集中于实现各个显性视角的合作,还未能充分探讨和利用各视角间共享的隐信息。针对此,对如何引入各个显性视角共享的隐空间信息来实现显隐视角协同的模糊系统建模进行了研究。具体地,基于岭回归极限学习模糊系统(ridge regression extreme learning fuzzy system,RR-EL-FS)模型,引入隐空间信息实现显隐视角协同学习来对RR-EL-FS进行学习,最终开发出具有显隐视角协同功能的岭回归极限学习模糊系统预测模型(ridgeregression extreme learning fuzzy system with cooperation between visible and hidden views,RR-EL-FS-CVH)。该方法较之以往相关的多视角建模方法能更好地利用隐空间的有效信息,从而能够进一步提高受训模型的泛化性能。大量的实验结果亦验证了所提方法的有效性。
Multi-view datasets are frequently encountered in learning tasks. However, existing multi-view approaches mainly focus on the visible views and ignore the hidden information shared by the visible views. To address this problem, a multi-view fuzzy system(FS) which utilizes the hidden information shared by the multiple visible views is proposed in this paper. More specifically, using the ridge regression extreme learning fuzzy system(RR-EL-FS)as the base model, an approach with the cooperation of visible and hidden views(RR-EL-FS-CVH) is proposed for multi-view learning. This method can make better use of the effective information of the hidden space than the previous multi-view methods, and thus can further improve the generalization performance of the trained model.Extensive experiments are conducted to validate its effectiveness.
Multi-view datasets are frequently encountered in learning tasks. However, existing multi-view approaches mainly focus on the visible views and ignore the hidden information shared by the visible views. To address this problem, a multi-view fuzzy system(FS) which utilizes the hidden information shared by the multiple visible views is proposed in this paper. More specifically, using the ridge regression extreme learning fuzzy system(RR-EL-FS)as the base model, an approach with the cooperation of visible and hidden views(RR-EL-FS-CVH) is proposed for multi-view learning. This method can make better use of the effective information of the hidden space than the previous multi-view methods, and thus can further improve the generalization performance of the trained model.Extensive experiments are conducted to validate its effectiveness.
引文
[1] Deng Z H, Jiang Y Z, Choi K S, et al. Knowledge-leveragebased TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks&Learning Systems, 2013, 24(8):1200-1212.
[2] Blum A, Mitchell T M. Combining labeled and unlabeled data with co-training[C]//Proceedings of the 11th Annual Conference on Computational Learning Theory, Madison,Jul 24-26, 1998. New York:ACM, 1998:92-100.
[3] Cleuziou G, Exbrayat M, Martin L, et al. CoFKM:a centralized method for multiple-view clustering[C]//Proceedings of the 9th IEEE International Conference on Data Mining,Miami, Dec 6-9, 2009. Washington:IEEE Computer Society,2004:752-757.
[4] Pan S J, Yang Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge&Data Engineering, 2010, 22(10):1345-1359.
[5] Li G X, Chang K Y, Hoi S C H. Multiview semi-supervised learning with consensus[J]. IEEE Transactions on Knowledge&Data Engineering, 2012, 24(11):2040-2051.
[6] Sun S L. Multi-view Laplacian support vector machines[C]//LNCS 7121:Proceedings of the 7th International Conference on Advanced Data Mining and Applications, Beijing, Dec17-19, 2011. Berlin, Heidelberg:Springer, 2011:209-222.
[7] Farquhar J D R, Hardoon D R, Meng H Y, et al. Two view learning:SVM-2K, theory and practice[C]//Proceedings of the International Conference on Neural Information Processing Systems, Vancouver, Dec 5-8, 2005. Cambridge:MIT Press,2005:355-362.
[8] Chen Q N, Sun S L. Hierarchical multi-view fisher discriminant analysis[C]//LNCS 5864:Proceedings of the International Conference on Neural Information Processing, Bangkok,Dec 1-5, 2009. Berlin, Heidelberg:Springer, 2009:289-298.
[9] Pedrycz W. Collaborative fuzzy clustering[J]. Pattern Recognition Letters, 2002, 23(14):1675-1686.
[10] Tzortzis G, Likas A. Kernel-based weighted multi-view clustering[C]//Proceedings of the 12th IEEE International Conference on Data Mining, Brussels, Dec 10-13, 2012.Washington:IEEE Computer Society, 2012:675-684.
[11] Yamanishi Y, Vert J P, Kanehisa M. Protein network inference from multiple genomic data:a supervised approach[J]. Bioinformatics, 2004, 20(1):363-370.
[12] Chaudhuri K, Kakade S M, Livescu K, et al. Multi-view clustering via canonical correlation analysis[C]//Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Jun 14-18, 2009. New York:ACM, 2009:129-136.
[13] Kakade S M, Foster D P. Multi-view regression via canonical correlation analysis[C]//LNCS 4539:Proceedings of the 20th Annual Conference on Learning Theory, San Diego, Jun 13-15, 2007. Berlin, Heidelberg:Springer, 2007:82-96.
[14] Jiang Y Z, Chung F L, Wang S T, et al. Collaborative fuzzy clustering from multiple weighted views[J]. IEEE Transactions on Cybernetics, 2015, 45(4):688-701.
[15] Wang Y T, Chen L H. Multi-view fuzzy clustering with minimax optimization for effective clustering of data from multiple sources[J]. Expert Systems with Applications,2017, 72:457-466.
[16] Jiang Y Z, Deng Z H, Chung F L, et al. Realizing two-view TSK fuzzy classification system by using collaborative learning[J]. IEEE Transactions on Systems, Man&Cybernetics:Systems, 2017, 47(1):145-160.
[17] Jiang Y Z, Deng Z H, Chung F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1):3-20.
[18] Zhang T, Deng Z H, Choi K S, et al. Robust extreme learning fuzzy systems using ridge regression for small and noisy datasets[C]//Proceedings of the IEEE International Conference on Fuzzy Systems, Naples, Jul 9-12, 2017. Piscataway:IEEE,2017:1-7.
[19] Takagi T, Sugeno M. Fuzzy identification of systems and its applications to modeling and control[J]. Readings in Fuzzy Sets for Intelligent Systems, 1993, 15(1):387-403.
[20] Mamdani E H. Application of fuzzy logic to approximate reasoning using linguistic synthesis[C]//Proceedings of the International Symposium on Multiple-Valued Logic, Logan,May 25-28, 1976. Washington:IEEE Computer Society, 1976:196-202.
[21] Azeem M F, Hanmandlu M, Ahmad N. Generalization of adaptive neuro-fuzzy inference systems[J]. IEEE Transactions on Neural Networks, 2000, 11(6):1332-1346.
[22] Wong S Y, Yap K S, Yap H J, et al. On equivalence of FIS and ELM for interpretable rule-based knowledge representation[J]. IEEE Transactions on Neural Networks&Learning Systems, 2015, 26(7):1417-1430.
[23] Yap K S, Lim C P, Au M T. Improved GART neural network model for pattern classification and rule extraction with application to power systems[J]. IEEE Transactions on Neural Networks, 2011, 22(12):2310-2323.
[24] Tan S C, Lim C P, Rao M V C. A hybrid neural network model for rule generation and its application to process fault detection and diagnosis[J]. Engineering Applications of Artificial Intelligence, 2007, 20(2):203-213.
[25] Yang L, Jing L P, Yu J. Common latent space identification for heterogeneous co-transfer clustering[C]//Proceedings of the 5th International Conference on Intelligence Science and Big Data Engineering, Suzhou, Jun 14-16, 2015. New York:ACM, 2015:395-406.
[26] Lee D D, Seung H S. Learning the parts of objects with nonnegative matrix factorization[J]. Nature, 1999, 401(6755):788-791.
[27] Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering[J]. Advances in Neural Information Processing Systems, 2002, 14(6):585-591.
[28] Dua D, Karra Taniskidou E. UCI machine learning repository[DB/OL].(2017). http://archive.ics.uci.edu/ml.
[29] Leski J M. TSK-fuzzy modeling based onε-insensitive learning[J]. IEEE Transactions on Fuzzy Systems, 2005, 13(2):181-193.
[30] Chiu S L. Fuzzy model identification based on cluster estimation[J]. Journal of Intelligent&Fuzzy Systems, 1994, 2(3):267-278.
[31] Deng Z H, Zhang J B, Jiang Y Z, et al. Fuzzy subspace clustering based zero-order l2-norm TSK fuzzy system[J].Journal of Electronics&Information Technology, 2015, 37(9):2082-2088.
[32] Chao G Q, Sun S L. Alternative multiview maximum entropy discrimination[J]. IEEE Transactions on Neural Networks&Learning Systems, 2016, 27(7):1445-1456.
[31]邓赵红,张江滨,蒋亦樟,等.基于模糊子空间聚类的〇阶L2型TSK模糊系统[J].电子与信息学报, 2015, 37(9):2082-2088.
[2] Blum A, Mitchell T M. Combining labeled and unlabeled data with co-training[C]//Proceedings of the 11th Annual Conference on Computational Learning Theory, Madison,Jul 24-26, 1998. New York:ACM, 1998:92-100.
[3] Cleuziou G, Exbrayat M, Martin L, et al. CoFKM:a centralized method for multiple-view clustering[C]//Proceedings of the 9th IEEE International Conference on Data Mining,Miami, Dec 6-9, 2009. Washington:IEEE Computer Society,2004:752-757.
[4] Pan S J, Yang Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge&Data Engineering, 2010, 22(10):1345-1359.
[5] Li G X, Chang K Y, Hoi S C H. Multiview semi-supervised learning with consensus[J]. IEEE Transactions on Knowledge&Data Engineering, 2012, 24(11):2040-2051.
[6] Sun S L. Multi-view Laplacian support vector machines[C]//LNCS 7121:Proceedings of the 7th International Conference on Advanced Data Mining and Applications, Beijing, Dec17-19, 2011. Berlin, Heidelberg:Springer, 2011:209-222.
[7] Farquhar J D R, Hardoon D R, Meng H Y, et al. Two view learning:SVM-2K, theory and practice[C]//Proceedings of the International Conference on Neural Information Processing Systems, Vancouver, Dec 5-8, 2005. Cambridge:MIT Press,2005:355-362.
[8] Chen Q N, Sun S L. Hierarchical multi-view fisher discriminant analysis[C]//LNCS 5864:Proceedings of the International Conference on Neural Information Processing, Bangkok,Dec 1-5, 2009. Berlin, Heidelberg:Springer, 2009:289-298.
[9] Pedrycz W. Collaborative fuzzy clustering[J]. Pattern Recognition Letters, 2002, 23(14):1675-1686.
[10] Tzortzis G, Likas A. Kernel-based weighted multi-view clustering[C]//Proceedings of the 12th IEEE International Conference on Data Mining, Brussels, Dec 10-13, 2012.Washington:IEEE Computer Society, 2012:675-684.
[11] Yamanishi Y, Vert J P, Kanehisa M. Protein network inference from multiple genomic data:a supervised approach[J]. Bioinformatics, 2004, 20(1):363-370.
[12] Chaudhuri K, Kakade S M, Livescu K, et al. Multi-view clustering via canonical correlation analysis[C]//Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Jun 14-18, 2009. New York:ACM, 2009:129-136.
[13] Kakade S M, Foster D P. Multi-view regression via canonical correlation analysis[C]//LNCS 4539:Proceedings of the 20th Annual Conference on Learning Theory, San Diego, Jun 13-15, 2007. Berlin, Heidelberg:Springer, 2007:82-96.
[14] Jiang Y Z, Chung F L, Wang S T, et al. Collaborative fuzzy clustering from multiple weighted views[J]. IEEE Transactions on Cybernetics, 2015, 45(4):688-701.
[15] Wang Y T, Chen L H. Multi-view fuzzy clustering with minimax optimization for effective clustering of data from multiple sources[J]. Expert Systems with Applications,2017, 72:457-466.
[16] Jiang Y Z, Deng Z H, Chung F L, et al. Realizing two-view TSK fuzzy classification system by using collaborative learning[J]. IEEE Transactions on Systems, Man&Cybernetics:Systems, 2017, 47(1):145-160.
[17] Jiang Y Z, Deng Z H, Chung F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1):3-20.
[18] Zhang T, Deng Z H, Choi K S, et al. Robust extreme learning fuzzy systems using ridge regression for small and noisy datasets[C]//Proceedings of the IEEE International Conference on Fuzzy Systems, Naples, Jul 9-12, 2017. Piscataway:IEEE,2017:1-7.
[19] Takagi T, Sugeno M. Fuzzy identification of systems and its applications to modeling and control[J]. Readings in Fuzzy Sets for Intelligent Systems, 1993, 15(1):387-403.
[20] Mamdani E H. Application of fuzzy logic to approximate reasoning using linguistic synthesis[C]//Proceedings of the International Symposium on Multiple-Valued Logic, Logan,May 25-28, 1976. Washington:IEEE Computer Society, 1976:196-202.
[21] Azeem M F, Hanmandlu M, Ahmad N. Generalization of adaptive neuro-fuzzy inference systems[J]. IEEE Transactions on Neural Networks, 2000, 11(6):1332-1346.
[22] Wong S Y, Yap K S, Yap H J, et al. On equivalence of FIS and ELM for interpretable rule-based knowledge representation[J]. IEEE Transactions on Neural Networks&Learning Systems, 2015, 26(7):1417-1430.
[23] Yap K S, Lim C P, Au M T. Improved GART neural network model for pattern classification and rule extraction with application to power systems[J]. IEEE Transactions on Neural Networks, 2011, 22(12):2310-2323.
[24] Tan S C, Lim C P, Rao M V C. A hybrid neural network model for rule generation and its application to process fault detection and diagnosis[J]. Engineering Applications of Artificial Intelligence, 2007, 20(2):203-213.
[25] Yang L, Jing L P, Yu J. Common latent space identification for heterogeneous co-transfer clustering[C]//Proceedings of the 5th International Conference on Intelligence Science and Big Data Engineering, Suzhou, Jun 14-16, 2015. New York:ACM, 2015:395-406.
[26] Lee D D, Seung H S. Learning the parts of objects with nonnegative matrix factorization[J]. Nature, 1999, 401(6755):788-791.
[27] Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering[J]. Advances in Neural Information Processing Systems, 2002, 14(6):585-591.
[28] Dua D, Karra Taniskidou E. UCI machine learning repository[DB/OL].(2017). http://archive.ics.uci.edu/ml.
[29] Leski J M. TSK-fuzzy modeling based onε-insensitive learning[J]. IEEE Transactions on Fuzzy Systems, 2005, 13(2):181-193.
[30] Chiu S L. Fuzzy model identification based on cluster estimation[J]. Journal of Intelligent&Fuzzy Systems, 1994, 2(3):267-278.
[31] Deng Z H, Zhang J B, Jiang Y Z, et al. Fuzzy subspace clustering based zero-order l2-norm TSK fuzzy system[J].Journal of Electronics&Information Technology, 2015, 37(9):2082-2088.
[32] Chao G Q, Sun S L. Alternative multiview maximum entropy discrimination[J]. IEEE Transactions on Neural Networks&Learning Systems, 2016, 27(7):1445-1456.
[31]邓赵红,张江滨,蒋亦樟,等.基于模糊子空间聚类的〇阶L2型TSK模糊系统[J].电子与信息学报, 2015, 37(9):2082-2088.