一种图像检索框架的研究及其原型系统的实现
|
摘要
随着海量化的图像数据飞速膨胀，如何高效、快速地检索所需要的图像数据是当前图像应用领域的一个重要问题。因此，基于内容的图像检索(CBIR)已经成为当前国内外研究的热点。CBIR目前主要提取的是颜色、纹理、形状等底层视觉特征，而这些特征难以表示图像的高层语义特征。而传统的文本注释在描述高层抽象的语义概念时，有其简单清楚的优点。本文先是讨论了基于底层特征的图像检索，然后再讨论了将这两种方式结合起来进行检索的一种主动学习框架。
主动学习框架是国外近来提出的一种检索框架，本文对其进行了分析与改进。该框架假定图像的语义特征可用多层属性树来近似表示，并且图像拥有某个属性的概率值可由相邻的图像的概率值进行插值估计，框架中的相邻指图像在底层特征空间的相邻。该框架采用最大增益的原则对图像的注释过程进行指导，而不是盲目地随机注释，以便在注释相同数目图像的情况下，取得尽量好的检索结果。
本文引入颜色自相关图特征作为图像在底层特征空间相邻的度量，并修改了框架中带宽的计算函数，然后引入反馈机制，对于用户的正反馈和负反馈分别作不同的处理，以便在使用过程中，系统能够继续学习，根据反馈更新图像的概率链表，使之逐渐接近真实情况。
最后，使用C++Builder设计实现了该主动框架的原型系统。该原型系统提取了HSV直方图、共生矩阵纹理、颜色相关图等底层特征，同时根据图像数据库的特点，设计了平均位置比值等评价函数对语义特征与底层特征的各种结合的检索性能进行了比较分析，并对反馈效果进行了实验验证。
With the expansion of a large number of images, it become a very important issue in the field of image's application that how to retrieve the image wanted efficiently and quickly. So content-based image retrieval (CBIR) is now the study hot-point in the world. It extract low-level visual features such as color, texture, shape etc, but these low-level features contain little on high-level semantic contents. And traditional text annotation is simple and clear on expressing high-level semantics. In this paper, we first discuss the image retrieval methods based on low-level visual features, then discuss a active learning framework combining these two methods.
The active learning framework is proposed recently by American scholars. In this paper we analyze and improve the framework. The framework suppose that the image's semantic feature can be expressed by using a multi-level attribute tree, and the probability of a certain image having a certain attribute can be estimated by interpolation method using the neighboring image's value. Here, the neighboring relation refers to the neighboring in the low-level feature space. The framework direct the annotation process using the principle of max knowledge gain, not random annotation method, in order to gain better retrieval result in the same number of annotated images.
In this paper we use the color auto-correlogram as the similarity metrics of images in low-level feature space, and change the bandwidth function. Then we propose the semantic relevance feedback. The system react differently to the positive and negative user's feedback so that the system can go on learning after the annotation process by updating the probabilities of the list of attributes of the relevant images and reaching the real values.
Last, we design and implement the experimental system using C++Builder6. The experimental system extract the low-level features of images such as HSV histogram, the texture got from coexistence matrix, color correlogram, and according to the characteristic of our image database, design the evaluation function such as the average rank ratio to evaluate and compare the performance of different integration of different features including semantic, and validate the active effect of feedback using experiment results.
主动学习框架是国外近来提出的一种检索框架，本文对其进行了分析与改进。该框架假定图像的语义特征可用多层属性树来近似表示，并且图像拥有某个属性的概率值可由相邻的图像的概率值进行插值估计，框架中的相邻指图像在底层特征空间的相邻。该框架采用最大增益的原则对图像的注释过程进行指导，而不是盲目地随机注释，以便在注释相同数目图像的情况下，取得尽量好的检索结果。
本文引入颜色自相关图特征作为图像在底层特征空间相邻的度量，并修改了框架中带宽的计算函数，然后引入反馈机制，对于用户的正反馈和负反馈分别作不同的处理，以便在使用过程中，系统能够继续学习，根据反馈更新图像的概率链表，使之逐渐接近真实情况。
最后，使用C++Builder设计实现了该主动框架的原型系统。该原型系统提取了HSV直方图、共生矩阵纹理、颜色相关图等底层特征，同时根据图像数据库的特点，设计了平均位置比值等评价函数对语义特征与底层特征的各种结合的检索性能进行了比较分析，并对反馈效果进行了实验验证。
With the expansion of a large number of images, it become a very important issue in the field of image's application that how to retrieve the image wanted efficiently and quickly. So content-based image retrieval (CBIR) is now the study hot-point in the world. It extract low-level visual features such as color, texture, shape etc, but these low-level features contain little on high-level semantic contents. And traditional text annotation is simple and clear on expressing high-level semantics. In this paper, we first discuss the image retrieval methods based on low-level visual features, then discuss a active learning framework combining these two methods.
The active learning framework is proposed recently by American scholars. In this paper we analyze and improve the framework. The framework suppose that the image's semantic feature can be expressed by using a multi-level attribute tree, and the probability of a certain image having a certain attribute can be estimated by interpolation method using the neighboring image's value. Here, the neighboring relation refers to the neighboring in the low-level feature space. The framework direct the annotation process using the principle of max knowledge gain, not random annotation method, in order to gain better retrieval result in the same number of annotated images.
In this paper we use the color auto-correlogram as the similarity metrics of images in low-level feature space, and change the bandwidth function. Then we propose the semantic relevance feedback. The system react differently to the positive and negative user's feedback so that the system can go on learning after the annotation process by updating the probabilities of the list of attributes of the relevant images and reaching the real values.
Last, we design and implement the experimental system using C++Builder6. The experimental system extract the low-level features of images such as HSV histogram, the texture got from coexistence matrix, color correlogram, and according to the characteristic of our image database, design the evaluation function such as the average rank ratio to evaluate and compare the performance of different integration of different features including semantic, and validate the active effect of feedback using experiment results.
引文
[1] Thomas M. Lehmann, Berthold Wein, Jorg Dahmen, et al. Content-based Image Retrieval in Medical Application: A Novel Multi-Step Approach. In: http://libra.imib.rwth-aachen.de/irma/ps-pdf/SPIE_2000-3972(x)312-320.pdf.
[2] 向阳，庄越挺，潘云鹤. 基于内容的图像检索技术与系统. 计算机研究与发展，2001年3月，Vol 38, No3.
[3] C L Bird, P J Elliott, P M Hayward. Content-Based Retrieval for European Image Libraries. In: http://ewic.bcs.org/conferences/1999/ imageret/papers/paper2.pdf.
[4] Rickmann R, John S. Content based image retrieval using color tuple histograms. In: Proc of SPIE: Storageand Retrieval for Image and Video Database. SanJose, CA, 1996. 2670.
[5] Pass G, Zabih R, Miller J. Comparing images using color coherence vectors. In: Proc of ACM Intern Conf Multimedia Boston, MA, 1996.
[6] 王文惠. 基于内容的图像检索技术研究:[博士学位论文].长沙：国防科技大学研究生院，2001年10月.
[7] 顾燕. 基于内容的图像检索方法及其试验系统设计:[硕士学位论文]. 南京：河海大学，2002年2月.
[8] Rui Y, Alfred C, Huang T S. Modified Tourer descriptor for shape representation, a practical approach. In: Proc of First Int'l Workshop on Image Database and Multimedia Search, 1996.
[9] Jain A K, Vailaya. Shape-based retrieval: A case study with trademark image database. Pattern Recognition, 1998, 31(9). 1369一1390.
[10] Tamura H., Mori S., Yamawaky T. Textural Features Corresponding to Visual Perception. IEEE Transaction on Systems, Man and Cybernetics, Vol. SMC-8(6). pp. 460-473, 1972.
[11] Wu C M，Chen Y C. Statical feature fox texture analysis. CVGIP: Graphical Models and Image Processing, 1992. 54. (5): 407一419.
[12] Smith J R, Chang S F. Transform feature fox texture classification and discrim ination in large in age database. In: Proc of IEEE Int' 1 Conf on Image Processing. Austin, Texas, 1994.
[13] Allen J F. Maintaining knowledge about temporal interval. Communications ACM，1983,
26(11):832- 843.
[14] John Eakins, Margaret Graham. Content-based Image Retrieval. University of Norththumbria at Newcastle.
[15] Niblack W，Barber R, Equitz W et al. The QBIC project: Querying images by content using color, texture, and shape. In: Proc SPIE Electronic Imaging: Science and Technology. San Jose, CA, 1993.
[16] Bach J R, Fuller Charles, Cupta A et al. The Virage in age search engine: An open framework for image management. In: Proc of SPIE: Storage and Retrieval for Still Image and Vedio Database IV. San Jose, 1996, 2670:76一87.
[17] Smith J R，Chang S f. VisualSEEK: A fully automated content-based image query system.. In: The Fourth ACM Int'l Multimedia Conf 96 Proc (ACM Multimedia 96). Boston, 1996. 87一98.
[18] A. Pentland, R. W. Picard and S. Sclaroff. Photobook: Content-based Manipulation of Image Databases. Int. Journal of Compuler Vision, 1996.
[19] W. Y. Ma and B. S. Manjunath. Netra: A toolbox for Navigating Large Image Databases. Proc. IEEE Inl. Conf. On Image Processing, 1997.
[20] Cha Zhang, Tsuhan Chen. An Active Learning Framework for Content Based Information Retrieval. In: IEEE Trans on Multimedia, Special Issue on Multimedia Database, June 2002.
[21] Jing Huang. Color-Spatial Image Indexing And Application. A Dissertation Presented to the Faculty of the Graduate School of Cornell University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, August 1998.
[22] 阮秋琦. 数字图像处理学. 电子工业出版社，2001年1月第一版. 216-222, 419-425.
[23] 李志刚. 基于颜色特征的图像检索算法的研究及其系统的开发:[硕士学位论文]. 北京：中国农业大学，2001年3月.
[24] 鲁宏伟，肖茜. 一种基于内容的图像检索界面. 计算机工程与应用，2003年7月.
[25] Kenneth R. Castleman. 数字图像处理，朱志刚，林学闫，石定机，等译. 电子工业出版社，2002年2月第一版. P419-428.
[26] Kenneth R. Castleman. 数字图像处理，朱志刚，林学闫，石定机，等译. 电子工业出版社，2002年2月第一版. P409-418.
[27] Haralick R M, Shanmugam K, Dinstein 1. Texture features for image classification. In: IEEE Transactions on Systems,Man,and Cybernetics, 1973,3(6) .
[28] 闫薇. 基于颜色和纹理特征的图像检索:[硕士学位论文]. 哈尔滨：东北农业大学，2002年12月.
[29] Nastar C, et al. Efficient query refinement for image retrieval. In: Proceedings of IEEE Computer Society Conferenceon Computer Vision and Pattern Recognition, Santa, California, 1998.547-552.
[30] Rui Y, et al. Relevace feedback: a power tool for interactive content-based image retrieval. In: IEEE Trans. Circuits and systems form video technology, 1988,8(5).
[31] Hong-Jiang Zhang, Zheng Chen, Wen-Yin Liu, et al. Relevance Feedback in Content-Based Image Search. Microsoft Research, China.
[32] 杨娜，罗航哉，薛向阳，等. 基于内容的图像检索中相关反馈算法综述. 计算机科学，2001年，Vol.28，No.9.
[33] I.J. Cox, T.V. Papathomas, J. Ghosh, et al. Hidden annotation for content-based image retrieval. In: Porc. of IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997.
[34] S.-T. Chiu. A Comparative Review of Bandwidth Selection for Kernel Density Estimation. Statistica Sinica, pp. 129-145, No.6, 1996.
[35] Silverman B.W. Density Estimation from Statistics and Data Analysis. New York: Chapman and Hall, 1986.
[36] Lee C, Ma W Y, Zhang H J. Information Enbedding Based on User's Relevance Feedback for Image Retrieval:[Technical Report]. HP Labs, 1998.
[37] Vasconcelos N, Lippman A. Learning From User Feedback in Image Retrieval System. NIPS'99, Denver, Colorado, 1999.
[38] 林建成，陈昭炯. 一种图像检索带反馈的主动学习框架. 青岛大学学报（自然科学版），2003年增刊，2003年全国理论计算机科学学术年会论文集，Vol.16.
[39] Stanley B. Lippman, Josee Lajoie. C++ Primer(第三版)中文版，潘爱民，张丽译. 中国电力出版社，2002年5月第一版. 译序，前沿.
[40] 陈灿煌. C++ Builder6彻底研究. 中国铁道出版社，2003年2月第一版. 出版说明，Page 1-22.
[2] 向阳，庄越挺，潘云鹤. 基于内容的图像检索技术与系统. 计算机研究与发展，2001年3月，Vol 38, No3.
[3] C L Bird, P J Elliott, P M Hayward. Content-Based Retrieval for European Image Libraries. In: http://ewic.bcs.org/conferences/1999/ imageret/papers/paper2.pdf.
[4] Rickmann R, John S. Content based image retrieval using color tuple histograms. In: Proc of SPIE: Storageand Retrieval for Image and Video Database. SanJose, CA, 1996. 2670.
[5] Pass G, Zabih R, Miller J. Comparing images using color coherence vectors. In: Proc of ACM Intern Conf Multimedia Boston, MA, 1996.
[6] 王文惠. 基于内容的图像检索技术研究:[博士学位论文].长沙：国防科技大学研究生院，2001年10月.
[7] 顾燕. 基于内容的图像检索方法及其试验系统设计:[硕士学位论文]. 南京：河海大学，2002年2月.
[8] Rui Y, Alfred C, Huang T S. Modified Tourer descriptor for shape representation, a practical approach. In: Proc of First Int'l Workshop on Image Database and Multimedia Search, 1996.
[9] Jain A K, Vailaya. Shape-based retrieval: A case study with trademark image database. Pattern Recognition, 1998, 31(9). 1369一1390.
[10] Tamura H., Mori S., Yamawaky T. Textural Features Corresponding to Visual Perception. IEEE Transaction on Systems, Man and Cybernetics, Vol. SMC-8(6). pp. 460-473, 1972.
[11] Wu C M，Chen Y C. Statical feature fox texture analysis. CVGIP: Graphical Models and Image Processing, 1992. 54. (5): 407一419.
[12] Smith J R, Chang S F. Transform feature fox texture classification and discrim ination in large in age database. In: Proc of IEEE Int' 1 Conf on Image Processing. Austin, Texas, 1994.
[13] Allen J F. Maintaining knowledge about temporal interval. Communications ACM，1983,
26(11):832- 843.
[14] John Eakins, Margaret Graham. Content-based Image Retrieval. University of Norththumbria at Newcastle.
[15] Niblack W，Barber R, Equitz W et al. The QBIC project: Querying images by content using color, texture, and shape. In: Proc SPIE Electronic Imaging: Science and Technology. San Jose, CA, 1993.
[16] Bach J R, Fuller Charles, Cupta A et al. The Virage in age search engine: An open framework for image management. In: Proc of SPIE: Storage and Retrieval for Still Image and Vedio Database IV. San Jose, 1996, 2670:76一87.
[17] Smith J R，Chang S f. VisualSEEK: A fully automated content-based image query system.. In: The Fourth ACM Int'l Multimedia Conf 96 Proc (ACM Multimedia 96). Boston, 1996. 87一98.
[18] A. Pentland, R. W. Picard and S. Sclaroff. Photobook: Content-based Manipulation of Image Databases. Int. Journal of Compuler Vision, 1996.
[19] W. Y. Ma and B. S. Manjunath. Netra: A toolbox for Navigating Large Image Databases. Proc. IEEE Inl. Conf. On Image Processing, 1997.
[20] Cha Zhang, Tsuhan Chen. An Active Learning Framework for Content Based Information Retrieval. In: IEEE Trans on Multimedia, Special Issue on Multimedia Database, June 2002.
[21] Jing Huang. Color-Spatial Image Indexing And Application. A Dissertation Presented to the Faculty of the Graduate School of Cornell University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, August 1998.
[22] 阮秋琦. 数字图像处理学. 电子工业出版社，2001年1月第一版. 216-222, 419-425.
[23] 李志刚. 基于颜色特征的图像检索算法的研究及其系统的开发:[硕士学位论文]. 北京：中国农业大学，2001年3月.
[24] 鲁宏伟，肖茜. 一种基于内容的图像检索界面. 计算机工程与应用，2003年7月.
[25] Kenneth R. Castleman. 数字图像处理，朱志刚，林学闫，石定机，等译. 电子工业出版社，2002年2月第一版. P419-428.
[26] Kenneth R. Castleman. 数字图像处理，朱志刚，林学闫，石定机，等译. 电子工业出版社，2002年2月第一版. P409-418.
[27] Haralick R M, Shanmugam K, Dinstein 1. Texture features for image classification. In: IEEE Transactions on Systems,Man,and Cybernetics, 1973,3(6) .
[28] 闫薇. 基于颜色和纹理特征的图像检索:[硕士学位论文]. 哈尔滨：东北农业大学，2002年12月.
[29] Nastar C, et al. Efficient query refinement for image retrieval. In: Proceedings of IEEE Computer Society Conferenceon Computer Vision and Pattern Recognition, Santa, California, 1998.547-552.
[30] Rui Y, et al. Relevace feedback: a power tool for interactive content-based image retrieval. In: IEEE Trans. Circuits and systems form video technology, 1988,8(5).
[31] Hong-Jiang Zhang, Zheng Chen, Wen-Yin Liu, et al. Relevance Feedback in Content-Based Image Search. Microsoft Research, China.
[32] 杨娜，罗航哉，薛向阳，等. 基于内容的图像检索中相关反馈算法综述. 计算机科学，2001年，Vol.28，No.9.
[33] I.J. Cox, T.V. Papathomas, J. Ghosh, et al. Hidden annotation for content-based image retrieval. In: Porc. of IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997.
[34] S.-T. Chiu. A Comparative Review of Bandwidth Selection for Kernel Density Estimation. Statistica Sinica, pp. 129-145, No.6, 1996.
[35] Silverman B.W. Density Estimation from Statistics and Data Analysis. New York: Chapman and Hall, 1986.
[36] Lee C, Ma W Y, Zhang H J. Information Enbedding Based on User's Relevance Feedback for Image Retrieval:[Technical Report]. HP Labs, 1998.
[37] Vasconcelos N, Lippman A. Learning From User Feedback in Image Retrieval System. NIPS'99, Denver, Colorado, 1999.
[38] 林建成，陈昭炯. 一种图像检索带反馈的主动学习框架. 青岛大学学报（自然科学版），2003年增刊，2003年全国理论计算机科学学术年会论文集，Vol.16.
[39] Stanley B. Lippman, Josee Lajoie. C++ Primer(第三版)中文版，潘爱民，张丽译. 中国电力出版社，2002年5月第一版. 译序，前沿.
[40] 陈灿煌. C++ Builder6彻底研究. 中国铁道出版社，2003年2月第一版. 出版说明，Page 1-22.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |