立体电视中多视点视频增强和视线跟踪方法研究
|
摘要
数字多媒体是当前最活跃的研究领域之一。随着人们对服务质量和视觉体验要求的不断提高,数字多媒体处理技术和设备都在不停的更新换代。作为一种新兴的可视媒介形式,立体电视(Three Dimensional Television,3DTV)能够为用户提供很强的立体感和沉浸感,因而引起广泛关注并取得一系列成果。与传统二维(2D)电视相比,立体电视需要向用户提供多个视点的视频,所以传输数据量巨大,成为制约立体电视技术发展的瓶颈。
在立体电视系统中,多视点视频(Multi-view Video, MVV)的空间分辨率、帧率(frame rate)、视点的数量等对视觉效果有着很大的影响。多视点视频帧率越高,特别对于大尺寸液晶显示设备(Liquid Crystal Display, LCD),电视画面就会越流畅；视点数量越多,用户的观看范围越大,立体感也就越强。然而,随着传输视频帧率的提高、视点数量的增多,传输的数据量必然急剧增加。虽然当前提出了有效的多视点视频编码技术,但立体电视传输中多视点视频的帧率、视点数量仍然不能满足实际要求。另外,在交互式立体电视系统中,通常需要对用户视线或者头部进行跟踪,依此确定视点的播放与切换。
针对上述立体电视系统中存在的问题,本文着重对立体电视系统中多视点视频增强和视线跟踪方法进行了研究。其中,多视点视频增强包含立体视频帧率提升和视点合成两个方面。从发送端来看,利用帧率提升和视点合成可以减少数据传输量,节省带宽；从接收端来进行分析,帧率提升使单路视频看起来更流畅,减少了由于视频帧率过低造成的运动模糊和抖动,而通过视点合成增加了虚拟相机的数量,扩大了可视范围,使用户能够获得更好的立体视觉体验。另外,本论文对交互式立体电视中的人机交互技术进行了研究,提出了一种基于灰度分布视频处理的非接触式视线跟踪系统。
论文在以下几个方面的研究取得重要进展：
1、提出了一种彩色+深度格式(video plus depth)的立体视频帧率提升算法。首先,根据深度信息将运动向量(Motion Vector, MV)分成深度连续运动向量和深度不连续运动向量。在深度连续运动向量场(Motion Vector Field, MVF)中,利用深度层次约束的运动向量优化方法对错误运动向量进行检测和校正,提高运动向量的准确度；对于深度不连续运动向量场,则采用了一种基于前景匹配的运动向量优化方法,保持了运动补偿过程中前景运动物体边缘的完整性。
2、提出了一种基于深度的适应性运动补偿和图像块分割方案。根据视频场景中深度信息和运动向量的关系,适应性选择前向运动补偿与后向运动补偿,同时,利用基于深度和α-matting的图像分割方法对深度不连续图像块进行分割,相对于传统帧率提升算法,减少了运动补偿后遮挡区域和非遮挡区域出现的模糊和伪影的现象,可以有效提升立体视频的视觉质量。
3、提出了一种基于非对称图像修复的虚拟视点重建算法。在提出的算法中利用左右两路的彩色视频和深度视频对中间的任意虚拟视点的视频进行生成,利用虚拟视点与左右参考视点之间的空间位置关系确定主要参考视点和辅助参考视点。首先,利用三维图像变换技术将主要参考视点和辅助参考视点图像投影到虚拟视点。其次,通过图像处理技术去除虚拟视点图像中的裂纹和错误点,提高图像质量。然后,通过辅助虚拟视点图像对主要虚拟视点图像中的遮挡区域进行填充,.并且为了实现视点间视频色彩的统一,对左右参考视点图像进行亮度调整。最后,利用深度辅助的非对称图像修复方法对剩余的空洞区域进行填充。
4、提出了一种基于灰度分布视频处理的非接触式视线跟踪系统,该系统可以作为交互式立体电视系统中的人机交互设备。首先,在近红外光源照射条件下采集使用者头动视频,然后根据视频帧的灰度分布特征,依次进行面部区域、眼睛区域、瞳孔区域的检测与提取,最终计算出瞳孔角膜点反射坐标和瞳孔中心坐标；根据特征参数,采用基于交比不变性质的视线跟踪算法进行注视点位置的计算,实现视线跟踪；另外,针对视线跟踪过程中眼球视轴和光轴的不重合,提出了一种简单有效的五点定标算法。实验结果表明,针对佩戴眼镜和裸眼使用者,该系统均能够达到较高的精度,能够满足实际需要。
Digital multimedia technology is one of the most active areas of research. With the increasing requirements of quality of service and visual experience, there are continuous evolutions in digital multimedia processing technology and multimedia equipments. As a new form of visual media, three-dimensional television (3DTV) can provide users with a strong stereoscopic sensation and immersion, which caused widespread concern and made a series of achievements. Comparing with traditional two-dimensional television, multiple videos should be provided in stereoscopic dis-play. And hence, the amount of data is huge, which is the bottleneck in the devel-opment of3D TV technologies.
Resolution, frame rate and number of viewpoints have a great influence on the visual effect of multi-view video in3D TV system. Obviously, the multi-view video will be played more fluently with its frame rate higher, especially on large size liquid crystal display (LCD). If the system contains more viewpoints, the view angle will be wider and the3D effect will be more realistic. However, high frame rate and dense viewpoints result in a sharp increase of transmission data volume. Although efficient multi-view video coding technologies have been proposed, sometimes the frame rate and number of viewpoints cannot meet the requirement of practical applications. Additionally, in interactive3D TV system the head or gaze of the user usually should be tracked to determine which viewpoint in the multi-view video should be played.
Focusing on the issues existing in3DTV System, multi-view video enhance-ment and gaze tracking is studied in this paper. In our research, multi-view video enhancement contains frame rate up-conversion and view synthesis. The applica-tion of frame rate up-conversion and view synthesis can reduce the data volume in transmission. In display terminal, the motion blur and ghost artifacts will be suppressed by converting the video to a higher frame rate, furthermore, the video of every viewpoint will be played more fluently. On the other side, by using the tech-nology of view synthesis, the view angle will be wider and3D effect will be more realistic. Moreover, human-computer interaction technology in interactive3DTV system have been studied in this dissertation, and a remote gaze tracking system using gray-distribution-based video processing is proposed.
In general, the main contributions of our work can be summarized as follows:
1. A frame rate up-conversion algorithm for stereoscopic video is proposed in this paper. Firstly, by considering the depth cue in video plus depth representation, we categorize the blocks of the interpolated frame as depth-continuous and depth-discontinuous groups. The motion vector outliers in the depth continu-ous motion vector field are then detected and corrected by a layer-constrained motion vector refinement method. In order to preserve the boundary of the foreground objects, foreground matching is employed to refine motion vectors in depth discontinuous motion vector field.
2. A depth-based adaptive interpolation and block segmentation method is pro-posed in this paper. By considering the relationship of depth and motion in the video scene, motion compensation is chosen forward or backward adap-tively. At the same time, we propose a depth and alpha matting based block segmentation method to deal with the motion blur and ghost artifacts caused by disocclusion and occlusion at the boundary of the foreground area, and the visual quality of the stereoscopic video is improved.
3. In this dissertation, we propose a view synthesis algorithm based on asymmet-ric image inpainting. In our scheme, left and right reference color images and their associated depth maps are used to generate the arbitrary virtual view-point. Based on the spatial relationship between the virtual viewpoint and reference viewpoint, main and auxiliary reference viewpoint is distinguished. Firstly, the main and auxiliary viewpoint images are warped to the virtual viewpoint. After that, the cracks and error points are removed to enhance the image quality. Then, we complement the disocclusions of the virtual view-point image warped from the main viewpoint with the help of the auxiliary viewpoint. In order to reduce the color incontinuity of the virtual view, the brightness of the two reference viewpoint images are adjusted. Finally, the holes are filled by a depth-assistance asymmetric dilation inpainting method.
4. A remote gaze tracking system using gray-distribution-based video processing which can be used as human-computer interaction device in interactive3D TV system, is proposed in our research. First, the video of the user's head is captured under the illumination of near infrared light sources. The face region, eye region and pupil region are detected and extracted successively based on the gray-distribution of the video frame, and then the coordinates of the corneal glints and the pupil center are obtained. After that, the positions of the points on the screen that the user fixating are estimated by the gaze tracking algorithm based on cross-ratio-invariant. Additionally, a five-points calibration procedure is proposed to eliminate the error produced by the de-viation of the optical and visual axes. The proposed remote gaze tracking system has a low computational complexity and high robustness, and experi-ment results indicate that it is tolerant of head movement and still works well for users wearing glasses as well.
在立体电视系统中,多视点视频(Multi-view Video, MVV)的空间分辨率、帧率(frame rate)、视点的数量等对视觉效果有着很大的影响。多视点视频帧率越高,特别对于大尺寸液晶显示设备(Liquid Crystal Display, LCD),电视画面就会越流畅；视点数量越多,用户的观看范围越大,立体感也就越强。然而,随着传输视频帧率的提高、视点数量的增多,传输的数据量必然急剧增加。虽然当前提出了有效的多视点视频编码技术,但立体电视传输中多视点视频的帧率、视点数量仍然不能满足实际要求。另外,在交互式立体电视系统中,通常需要对用户视线或者头部进行跟踪,依此确定视点的播放与切换。
针对上述立体电视系统中存在的问题,本文着重对立体电视系统中多视点视频增强和视线跟踪方法进行了研究。其中,多视点视频增强包含立体视频帧率提升和视点合成两个方面。从发送端来看,利用帧率提升和视点合成可以减少数据传输量,节省带宽；从接收端来进行分析,帧率提升使单路视频看起来更流畅,减少了由于视频帧率过低造成的运动模糊和抖动,而通过视点合成增加了虚拟相机的数量,扩大了可视范围,使用户能够获得更好的立体视觉体验。另外,本论文对交互式立体电视中的人机交互技术进行了研究,提出了一种基于灰度分布视频处理的非接触式视线跟踪系统。
论文在以下几个方面的研究取得重要进展：
1、提出了一种彩色+深度格式(video plus depth)的立体视频帧率提升算法。首先,根据深度信息将运动向量(Motion Vector, MV)分成深度连续运动向量和深度不连续运动向量。在深度连续运动向量场(Motion Vector Field, MVF)中,利用深度层次约束的运动向量优化方法对错误运动向量进行检测和校正,提高运动向量的准确度；对于深度不连续运动向量场,则采用了一种基于前景匹配的运动向量优化方法,保持了运动补偿过程中前景运动物体边缘的完整性。
2、提出了一种基于深度的适应性运动补偿和图像块分割方案。根据视频场景中深度信息和运动向量的关系,适应性选择前向运动补偿与后向运动补偿,同时,利用基于深度和α-matting的图像分割方法对深度不连续图像块进行分割,相对于传统帧率提升算法,减少了运动补偿后遮挡区域和非遮挡区域出现的模糊和伪影的现象,可以有效提升立体视频的视觉质量。
3、提出了一种基于非对称图像修复的虚拟视点重建算法。在提出的算法中利用左右两路的彩色视频和深度视频对中间的任意虚拟视点的视频进行生成,利用虚拟视点与左右参考视点之间的空间位置关系确定主要参考视点和辅助参考视点。首先,利用三维图像变换技术将主要参考视点和辅助参考视点图像投影到虚拟视点。其次,通过图像处理技术去除虚拟视点图像中的裂纹和错误点,提高图像质量。然后,通过辅助虚拟视点图像对主要虚拟视点图像中的遮挡区域进行填充,.并且为了实现视点间视频色彩的统一,对左右参考视点图像进行亮度调整。最后,利用深度辅助的非对称图像修复方法对剩余的空洞区域进行填充。
4、提出了一种基于灰度分布视频处理的非接触式视线跟踪系统,该系统可以作为交互式立体电视系统中的人机交互设备。首先,在近红外光源照射条件下采集使用者头动视频,然后根据视频帧的灰度分布特征,依次进行面部区域、眼睛区域、瞳孔区域的检测与提取,最终计算出瞳孔角膜点反射坐标和瞳孔中心坐标；根据特征参数,采用基于交比不变性质的视线跟踪算法进行注视点位置的计算,实现视线跟踪；另外,针对视线跟踪过程中眼球视轴和光轴的不重合,提出了一种简单有效的五点定标算法。实验结果表明,针对佩戴眼镜和裸眼使用者,该系统均能够达到较高的精度,能够满足实际需要。
Digital multimedia technology is one of the most active areas of research. With the increasing requirements of quality of service and visual experience, there are continuous evolutions in digital multimedia processing technology and multimedia equipments. As a new form of visual media, three-dimensional television (3DTV) can provide users with a strong stereoscopic sensation and immersion, which caused widespread concern and made a series of achievements. Comparing with traditional two-dimensional television, multiple videos should be provided in stereoscopic dis-play. And hence, the amount of data is huge, which is the bottleneck in the devel-opment of3D TV technologies.
Resolution, frame rate and number of viewpoints have a great influence on the visual effect of multi-view video in3D TV system. Obviously, the multi-view video will be played more fluently with its frame rate higher, especially on large size liquid crystal display (LCD). If the system contains more viewpoints, the view angle will be wider and the3D effect will be more realistic. However, high frame rate and dense viewpoints result in a sharp increase of transmission data volume. Although efficient multi-view video coding technologies have been proposed, sometimes the frame rate and number of viewpoints cannot meet the requirement of practical applications. Additionally, in interactive3D TV system the head or gaze of the user usually should be tracked to determine which viewpoint in the multi-view video should be played.
Focusing on the issues existing in3DTV System, multi-view video enhance-ment and gaze tracking is studied in this paper. In our research, multi-view video enhancement contains frame rate up-conversion and view synthesis. The applica-tion of frame rate up-conversion and view synthesis can reduce the data volume in transmission. In display terminal, the motion blur and ghost artifacts will be suppressed by converting the video to a higher frame rate, furthermore, the video of every viewpoint will be played more fluently. On the other side, by using the tech-nology of view synthesis, the view angle will be wider and3D effect will be more realistic. Moreover, human-computer interaction technology in interactive3DTV system have been studied in this dissertation, and a remote gaze tracking system using gray-distribution-based video processing is proposed.
In general, the main contributions of our work can be summarized as follows:
1. A frame rate up-conversion algorithm for stereoscopic video is proposed in this paper. Firstly, by considering the depth cue in video plus depth representation, we categorize the blocks of the interpolated frame as depth-continuous and depth-discontinuous groups. The motion vector outliers in the depth continu-ous motion vector field are then detected and corrected by a layer-constrained motion vector refinement method. In order to preserve the boundary of the foreground objects, foreground matching is employed to refine motion vectors in depth discontinuous motion vector field.
2. A depth-based adaptive interpolation and block segmentation method is pro-posed in this paper. By considering the relationship of depth and motion in the video scene, motion compensation is chosen forward or backward adap-tively. At the same time, we propose a depth and alpha matting based block segmentation method to deal with the motion blur and ghost artifacts caused by disocclusion and occlusion at the boundary of the foreground area, and the visual quality of the stereoscopic video is improved.
3. In this dissertation, we propose a view synthesis algorithm based on asymmet-ric image inpainting. In our scheme, left and right reference color images and their associated depth maps are used to generate the arbitrary virtual view-point. Based on the spatial relationship between the virtual viewpoint and reference viewpoint, main and auxiliary reference viewpoint is distinguished. Firstly, the main and auxiliary viewpoint images are warped to the virtual viewpoint. After that, the cracks and error points are removed to enhance the image quality. Then, we complement the disocclusions of the virtual view-point image warped from the main viewpoint with the help of the auxiliary viewpoint. In order to reduce the color incontinuity of the virtual view, the brightness of the two reference viewpoint images are adjusted. Finally, the holes are filled by a depth-assistance asymmetric dilation inpainting method.
4. A remote gaze tracking system using gray-distribution-based video processing which can be used as human-computer interaction device in interactive3D TV system, is proposed in our research. First, the video of the user's head is captured under the illumination of near infrared light sources. The face region, eye region and pupil region are detected and extracted successively based on the gray-distribution of the video frame, and then the coordinates of the corneal glints and the pupil center are obtained. After that, the positions of the points on the screen that the user fixating are estimated by the gaze tracking algorithm based on cross-ratio-invariant. Additionally, a five-points calibration procedure is proposed to eliminate the error produced by the de-viation of the optical and visual axes. The proposed remote gaze tracking system has a low computational complexity and high robustness, and experi-ment results indicate that it is tolerant of head movement and still works well for users wearing glasses as well.
引文
[1]C. Fehn, "Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV," in Proc. SPIE, vol.5291, pp.93-104. Jan.2004.
[2]A. Gotchev, G. Bozdagi, T. Capin, D. Strohmeier, and A. Boev, "Three-dimensional media for mobile devices," Proc. IEEE, vol.99, no.4, pp.708-741, Apr.2011.
[3]C. Fehn, R. Barre, and R. S. Pastoor, "Interactive 3-DTV concepts and key technologies," Proc. IEEE, vol.94, no.3, pp.524-538, Mar.2006.
[4]C. Zitnick, S. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation,"ACM Trans. Graphics (Proc. SIGGRAPH), vol.23, no.3, pp.600-608, Aug.2004.
[5]M. Tanimoto, "Overview of free viewpoint television," Signal Process. Image Commun., vol.21, no.6, pp.454-461, Jul.2006.
[6]Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto, "View genera-tion with 3d warping using depth information for FTV," Signal Process. Image Commun., vol.24, no.1-2, pp.65-72, Jan.2009.
[7]M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, "Free-viewpoint TV,: IEEE Signal Process. Mag., vol.28, no.1, pp.61-76, Jan.2011.
[8]M. Tanimoto, "FTV:Free-viewpoint television," Signal Process. Image Com-mun., vol.27, no.6, pp.555-570, Jul.2012.
[9]D. Min, D. Kim, S. Yun, and K. Sohn. "2D/3D freeview video generation for 3DTV system," Signal Process. Image Commun., vol.24, no.1-2, pp.31-48, Jan.2009.
[10]L. Hill and A. Jacobs, "3-D liquid crystal display and their applications," Proc. IEEE, vol.94, no.3, pp.575-590, Mar.2006.
[11]J. Choi, D. Min, and K. Sohn, "2D-plus-depth based resolution and frame-rate up-conversion technique for depth video," IEEE Trans. Consum. Electron., vol. 56, no.4, pp.2489-2497, Nov.2010.
[12]D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comp. Vis., vol.47, no.1, pp.7-42, Apr.2002.
[13]J. W. Nadler, D. Barbash, H. R. Kim, S. Shimpi, D. E. Angelaki, and G. C. DeAngelis, "Joint representation of depth from motion parallax and binocular disparity cues in macaque area MT," J. Neurosci., vol.33, no.35, pp.14061-14074, Aug.2013.
[14]A. Saxena, M. Sun, and A. Y. Ng, "Make3D:learning 3D scene structure from a single still image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.5, pp.824-840, May 2009.
[15]Y. L. Chang, C. Y. Fang, L. F. Ding, and S. Y. Chen, "Depth map generation for 2D-to-3D conversion by short-term motion assisted color segmentation," in Proc. ICME, pp.1958-1961, Jul.2007.
[16]X. Cao, Z. Li, and Q. Dai, "Semi-automatic 2D-to-3D conversion using disparity propagation," IEEE Trans. Broadcast., vol.57, no.2, pp.491-499, Jun.2011.
[17]C. G. Gurler, B. Gorkemli, G. Saygili, and A. M. Tekalp, "Flexible transport of 3-d video over networks," Proc. IEEE, vol.99, no.4, pp.694-707, Apr.2011.
[18]K. Muller, P. Merkle, and T. Wiegand, "3-D video representation using depth maps," Proc. IEEE, vol.99, no.4, pp.643-656, Apr.2011.
[19]H. Urey, K. V. Chellappan, E. Erden, and P. Surman, "State of the art in stereoscopic and autostereoscopic displays," Proc. IEEE, vol.99, no.4, pp. 540-555, Apr.2011.
[20]J. D. Yun, Y. Kwak, and S. Yang, "Evaluation of perceptual resolution and crosstalk in sterescopic displays," J. Display Techno., vol.9, no.2, pp.106-111, Feb.2013.
[21]G. Lv, W. Zhao, D. Li, and Q. Wang, "Polarizer parallax barrier 3d display with high brightness, resolution and low crosstalk," J. Display Technol., vol.10, no.2, pp.120-124, Feb.2014.
[22]V. A. Ezhov, "Distant binocular filters for full-resolution autostereoscopic view-ing and for single-aperture stereo glasses," J. Display Technol., vol.10, no.2. pp.114-119, Feb.2014.
[23]C. T. E. R. Hewage, S. T. Worrall, S. Dogan, S. Villette, and A. M. Kondoz, "Qulity evaluating of color plus depth map-based stereoscopic video," IEEE J. Sel. Topics Signal Process., vol.3, no.2, pp.304-318, Apr.2009.
[24]P. Lebreton, A. Raake, M. Barkowsky, and P. L. Callet, "Evaluating depth perception of 3D stereoscopic videos," IEEE J. Sel. Topics Signal Process., vol. 6, no.6, pp.710-720, Oct.2012.
[25]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. "Imago quality assessment:From error visibility to structure similarity." IEEE Trans. Image Process., vol.13, no.4, pp.600-612, Apr.2004.
[26]D. V. S. X. De Silva. W. A. C. Fernando, G. Nur, E. Ekmekcioglu. and S. T. Worrall, "3D video assessment with just noticeable difference in depth evalua-tion," in Proc. ICIP, pp.4013-4016, Sep.2010.
[27]Y. Zhao, Z. Chen, C. Zhu, Y. P. Tan, and L. Yu, "Binocular just-noticeable-difference model for stereoscopic images," IEEE Signal Process. Lett., vol.18, no.1, pp.19-22, Jan.2011.
[28]T. Kim, J. Kang, S. Lee, and A. C. Bovik, "Multimodal interactive continuous scoring of subjective 3D video quality of experience," IEEE Trans. Multimedia, vol.16. no.2, pp.387-402, Feb.2014.
[29]Y. H. Lin and J. L. Wu, "Quality assessment of stereoscopic 3D image com-pression by binocular integration behaviors," IEEE Trans. Image Process., vol. 23, no.4, pp.1527-1542, Apr.2014.
[30]S. J. Kang, K. R. Cho, and Y. H. Kim, "Motion compensated frame rate up-conversion using extended bilateral motion estimation," IEEE Trans. Consum. Electron., vol.53, no.4, pp.1759-1767, Nov.2007.
[31]A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, "Multiview imaging and 3DTV," IEEE Signal Process. Mag., vol.24, no.7, pp. 10-21, Nov.2007.
[32]D. Wang, L. Zhang, and A. Vincent, "Motion-compensated frame rate up-conversion Part Ⅰ:Fast Multi-Frame Motion Estimation," IEEE Trans. Broad-cast, vol.56, no.2, pp.133-141, Jun.2010.
[33]D. Wang, A. Vincent, P. Blanchfield, and R. Klepko, "Motion-compensated frame rate up-conversion Part II:New algorithms for frame interpolation," IEEE Trans. Broadcast, vol.56, no.2, pp.142-149, Jun.2010.
[34]A. M. Huang and T. Q. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Pro-cess., vol.17, no.5, pp.694-708, May 2009.
[35]A. M. Huang and T. Q. Nguyen, "Correlation-Based Motion Vector Processing With Adaptive Interpolation Scheme for Motion-Compensated Frame Interpo-lation," IEEE Trans. Image Process., vol.18, no.4, pp.740-752, Apr.2009.
[36]M. J. Chen, L. G. Chen, and T. D Chiueh, "One-dimensional full search mo-tion estimation algorithm for video coding," IEEE Trans. Circuits Syst. Video Technol, vol.4, no.5, pp.504-509, Jan.1994.
[37]G. de Haan, P. W. A. C. Biezen, H. Huijgen, and O. A. Ojo, "True-motion estimation with 3-d recursive search block matching," IEEE Trans. Circuits Syst. Video Technol, vol.3, no.4, pp.368-379, Oct.1993.
[38]B. D. Choi, J. W. Han, C. S. Kim, and S. J. Ko, "Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation," IEEE Trans. Circuits Syst Video Technol, vol.17, no. 4, pp.407-416, Feb.2007.
[39]Z. Yu, H. Li, Z. Wang, Z. Hu, and C. Chen, "Multi-level video frame interpo-lation:Exploiting the interaction among different levels,"IEEE Trans. Circuits Syst. Video Technol., vol.23, no.7, pp.1235-1248, Jul.2013.
[40]贾茜,肖进胜,易本顺,甘良才,“基于三维递归搜索的多级运动估计视频帧率上转换算法,”电子与信息学报,vol.34,no.10.pp.2336-2341,2012.
[41]杨爱萍,董翠翠,侯正信,李娜,“基于多帧运动估计的帧率提升算法,”计算机应用研究,vol.29,no.10,pp.3952-3955,2012.
[42]Y. L. Lee and T. Nguyen, "Fast one-pass motion compensated frame inter-polation in high-definition video processing," in Proc. ICIP, pp.369-372, Nov. 2009.
[43]Y. Cho, H. Lee, and D. Park, "Temperal frame interpolation based on multiple feature trajectory," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.12. pp.2105-2115, Dec.2013.
[44]S. J. Kang, S. Yoo, and Y. H. Kim, "Dual Motion Estimation for Frame Rate Up-Conversion," IEEE Trans. Circuits Syst. Video Technol., vol.20, no.12, pp. 1909-1914, Dec.2010.
[45]X. Gao, C. Duanmu, and C. Zou, "A multilevel successive elimination algorithm for block matching motion estimation," IEEE Trans. Image Process., vol.9, no. 3, pp.501-504, Mar.2000.
[46]C. Wang, L. Zhang, Y. He, and Y. P. Tan, "Frame rate upconversion using trilateral filtering," IEEE Trans. Circuits Syst. Video Technol., vol.20, no.6, pp.886-893, Jun.2010.
[47]N. Jacobson, Y. Lee, V. Mahadevan, N. Vasconcelos, and T. Q. Nguyen, "A novel approach to fruc using discriminant saliency and frame segmentation,' IEEE Trans. Image Process., vol.19, no.11, pp.2924-2934, Nov.2010.
[48]Q. Lu and X. Fang, "Joint trilateral motion vector filter for bidirectional motion compensation," Electron. Lett., vol.49, no.13, pp.798-800,2013.
[49]D. Choi, W. Song, J. Park, H. Choi, and T. Kim, "A statistical approach to motion vector field smoothing for block-based motion-compensated frame interpolation," in Proc. ICASSP, pp.989-992,2012.
[50]S. G. Jeong, C. Lee, and C. S. Kim, "Example-based frame rate up-conversion with congruent segmentation," in Proc. ICIP, pp.845-848,2012.
[51]K. Chen and D. A. Lorenz, "Image sequence interpolation using optimal con-trol," J. Math. Imag. Vision, vol.41, no.3, pp.222-238,2011.
[52]M. Werlberger, T. Pock, M. Unger, and H. Bischof, "Optical flow guided TV-Ll video interpolation and restoration," in Proc. Energy Minimization Methods Comput. Vision Pattern Recognit., vol.6819, pp.273-286,2011.
[53]W. Lee, K. Choi, and J. Ra, "Frame rate up conversion based on vatiational image fusion," IEEE Trans. Image Process., vol.23, no.1, pp.399-412, Jan. 2014.
[54]Q. Lu, X. Fang, C. Xu, and Y. Wang, "Frame rate up-conversion for depth-based 3D video," in Proc. ICME, pp.598-603, Jul.2012.
[55]K. Choi, "Motion estimation for stereoscopic video and its application to frame rate up-conversion for 3-DTV," in Proc. ICCE, pp.608-609, Jun.2012.
[56]Y. Lee, Z. Lee, and T. Nguyen, "Frame rate up conversion of 3D video by motion and depth fusion," in Proc. IVMSP Workshop, pp.1-4, Jun.2013.
[57]G. Dane and T. Q. Nguyen, "Smooth motion vector resampling for standard compatible video post-processing," in Proc. Asilomar Conf. Signals Syst. Com-put, pp.1731-1735, Nov.2004.
[58]Y. Guo, Z. Gao, L. Chen, and X. Zhang, "Occlusion handling frame rate up-conversion," in Proc. ICASSP, pp.1424-1428,2013.
[59]D. Kim, H. Lim, and H. Park, "Iterative true motion estimation for motion-compensated frame interpolation," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.3, pp.445-454, Mar.2013.
[60]X. Yang, J. Liu, J. Sun, Y. Lee, and T. Q. Nguyen, "Depth-assisted frame rate up-conversion for stereoscopic video," IEEE Signal Process. Lett., vol.21, no. 4, pp.423-427, Apr.2014.
[61]A. Levin, D. Lischinski, and Y. Weiss, "A closed form solution to natural image matting," in Proc. CVPR, pp.61-68, Jun.2006.
[62]C. Liu, "Beyond pixels:exploring new representations and applications for motion analysis," Doctoral Thesis., Massachusetts Institute of Technology, May, 2009.
[63]N. Jacobson and T. Q. Nguyen, "Scale-aware saliency for application to frame rate upconversio," IEEE Trans. Image Process., vol.21, no.4, pp.2198-2206, Apr.2012.
[64]X. Yang, J. Liu, J. Sun, X. Li, and W. Liu, "DIBR based view synthesis for free-viewpoint television,"in Proc. IEEE 3DTV-CON, pp.1-4. May 2011.
[65]A. Smolic, K. Muller, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, "Interme-diate view interpolation based on multi-view video plus depth for advanced 3D video systems," in Proc. ICIP, pp.2448-2451,2008.
[66]W. Li, J. Zhou, B. Li, and M. I. Sezan, "Virtual view specification and synthesis for free viewpoint television," IEEE Trans. Circuits Syst. Video Technol.,vol. 19, no.4, pp.533-546, Apr.2009.
[67]L. Zhang and W. Tam, "Stereoscopic image generation based on depth images for 3D TV," IEEE Trans. Broadcast, vol.51, no.2, pp.191-199, Jun.2005.
[68]C. Cheng, S. Lin, S. Lai, and C. Yang, "Improved novel view synthesis from depth image with large baseline," in Proc. ICPR,2008.
[69]L. Tran, C. Pal, and T. Nguyen, "View synthesis based on conditional random fields and graph cuts," in Proc. ICIP, pp.433-436, Sep.2010.
[70]K. Oh, S. Yea, A. Vetro, and Y. Ho, "Virtual view synthesis method and self evaluation metrics for free viewpoint television and 3D video," Int. J. Imag. Syst. Technol,. vol.20, no.4, pp.378-390, Dec.2010.
[71]Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, "Boundary artifact reduction in view synthesis of 3D video:from perspective of texture-depth alignment," IEEE Trans. Broadcast, vol.57, no.2, pp.510-522, Jun.2011.
[72]A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol.13, no.9, pp.1200-1212, Sep.2004.
[73]T. F. M. Tanimoto and K. Suzuki, View Synthesis Algorithm in View Synthe-sis Reference Software 2.0 (VSRS 2.0), ISO/IEC JTC1/SC29/WG11M16090, Lausanne, Switzerland, Feb.2008.
[74]I. Ahn and C. Kim, "A novel depth-based virtual view synthesis method for free viewpoint video," IEEE Trans. Broadcast., vol.59, no.4, pp.614-626, Dec. 2013.
[75]J. Habigt and K. Diepold, "Image completion for view synthesis using Markov random fields and efficient belief propagation," in Proc. ICIP, pp.2131-2134, Sep.2013.
[76]C. M. Cheng, S. J Lin, and S. H. Lai, "Spatio-temporally consistent novel view synthesis algorithm from video-plus-depth sequences for autostereoscopic displays," IEEE Trans. Broadcast, vol.57, no.2, pp.523-532, Jun.2011.
[77]S. Choi, B. Ham, and K. Sohn, "Space-time hole filling with random walks in view extrapolation for 3D video," IEEE Trans. Image Process., vol.22, no.6, pp.2429-2441, Jun.2013.
[78]H. A. Hsu, C. K. Chiang, and S. H. Lai "Spatio-temporally consistent view synthesis from video-plus-depth data with global optimization," IEEE Trans. Circuits Syst. Video Technol, vol.24, no.1, pp.74-84, Jan.2014.
[79]M. Solh and G. AlRegib, "Hierarchical hole-filling for depth-based view syn-thesis in FTV and 3D video," IEEE J. Sel. Topics Signal Process., vol.6, no. 5, pp.495-504, Sep.2012.
[80]I. Daribo and H. Saito, "A novel inpainting-based layered depth video for 3DTV," IEEE Trans. Broadcast, vol.57, no.2, pp.533-541, Jun.2011.
[81]W. Sun, O. C. Au,L. Xu. Y. Li, W. Hu, and Z. Yu, "Texture optimization for seamless view synthesis through energy minimization," in Proc. ACM Int. Conf. Multimedia, pp.733-736, Oct.2012.
[82]S. H. Chan, R. Khoshabeh, K. B. Gibson. P. E. Gill, and T. Q. Nguyen, "An augmented Lagrangian method for total variation video restoration," IEEE Trans. Image Process., vol.20, no.11, pp.3097-3111, Nov.2011.
[83]S. Jung. "Enhancement of image and depth map using adaptive joint trilateral filter," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.2, pp.258-269, Feb.2013.
[84]J. G. Wang and E. Sung, "Gaze determination via images of irises," Image Vision Comput., vol.19, no.12, pp.891-911, Oct.2001.
[85]P. M. Corcoran, F. Nanu. S. Petrescu, and P. Bigioi, "Real-time eye gaze track-ing for gaming design and consumer electronics systems," IEEE Trans. Consum. Electron., vol.58, no.2, pp.347-355, May 2012.
[86]C. S. Lin, C. W. Ho, W. C. Chen, C. C. Chiu, and M. S. Yeh, "A cascaded scheme for eye tracking and head movement compensation," Opt. Appl., vol. XXXVI, no.2-3, pp.401-412,2006.
[87]X. Xie, R. Sudhakar, and H. Zhuang, "Enhancement of image and depth map using adaptive joint trilateral filter," IEEE Trans. Syst., Man, Cybern., vol.28, no.4, pp.487-490, Jul.1998.
[88]C. S. Lin, C. N. Chan. Y. L. Lay, J. F. Lee, and M. S. Yeh, "An eye-tracking human-machine interface using an auto correction method," J. Med. Biol. Eng., vol.27. no.2, pp.105-109, Mar.2007.
[89]T. Ohno and N. Mukawa, "A free-head, simple calibration, gaze tracking sys-tem that enables gaze-based interaction," in Proc. Eye Tracking Research and Applications Symp., pp.115-122, Mar.2004.
[90]A. Amir, L. Zimet, A. Sangiovanni-Vincentelli, and S. Kao, "An embedded system for an eye-detection sensor," Comput. Vis. Image Und., vol.98, no.1, pp.104-123, Apr.2005.
[91]D. Li, D. Winfield, and D. J. Parkhurst, "Starburst:A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches,' in Proc. CVPR, pp.79-86, Jun.2005.
[92]T. Ohno, N. Mukawa, and A. Yoshikawa, "Gaze tracking system for gaze-based human-computer interaction," NTT Technical Review, vol.1, no.2, pp.35-39, 2003.
[93]X. Yang, J. Sun, J. Liu, J. Chu, W. Liu, and Y. Gao, "A gaze tracking scheme for eye-based intelligent control," in Proc. IEEE WCICA, pp.50-55, Jul.2010.
[94]C. Ahlstrom, T. Victor, C. Wege, and E. Steinmetz, "Processing of eye/head-tracking data in large-scale naturalistic driving data sets," IEEE Trans. Intell. Transp. Syst., vol.13, no.2, pp.553-564, Jun.2012.
[95]X. Fu, X. Guan, E. Peli, H. Liu, and G. Luo, "Automatic calibration method for driver's head orientation in natural driving environment," IEEE Trans. Intell. Transp. Syst., vol.14, no.1, pp.303-312, Mar.2013.
[96]C. H. Morimoto, D. Koons, A. Amit, M. Flickner, and S. Zhai, "Keeping an eye for HCI," in Proc. XII Brazilian Symposium on Computer Graphics and Image Processing, pp.171-176, Mar.1999.
[97]Q. Ji and Z. Zhu, "Eye and tracking for interactive graphic display," Internet. Symp. on Smart Graphics, Jun.2002.
[98]D. Beymer and M. Flickner, "Eye gaze tracking using an active stereo head,' in Proc. CVPR, pp.451-458,2003.
[99]D. H. Yoo and M. J. Chung, "A novel nonintrusive eye gaze estimation using cross-ratio under large head motion," Comput. Vis. Image Und., vol.98, no.1, pp.25-51, Apr.2005.
[100]A. Villanueva and R. Cabeza. "A novel gaze estimation system with one cali-bration point," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.38, no.4, pp. 1123-1138, Aug.2008.
[101]C. Hennessey and P. Lawrence, "Improving the accuracy and reliability of re-mote system-calibration-free eye-gaze tracking," IEEE Trans. Biomedical Eng., vol.56, no.7, pp.1891-1900, Jul.2009.
[102]J. Sigut and S. A. Sidha. "Iris center corneal reflection method for gaze track-ing using visible light," IEEE Trans. Biomedical Eng., vol.58, no.2, pp.411-419, Feb.2011.
[103]刘瑞安,靳世久,宋维,张希坤：吴晓荣,“单摄像机视线跟踪,”计算机应用,vol.26.no.9,pp.2101-2104,Sep.2006.
[104]胡畔,董春鹏,吴晓荣,“基于视线跟踪技术的眼控鼠标,”天津师范大学学报,vol.27.no.3,pp.68-71,Sep.2007.
[105]程成,杜菁菁,蓝飞翔,“眼动交互的实时线性算法构造和实现,”电子学报,vol.37.no.4A,pp.12-15,Apr.2009.
[106]张闯,迟健男,张朝晖,高小亮,胡涛,王志良,“视线追踪系统中视线估计方法研究,”中国科学信息科学,vol.41,no.5,2011.
[107]D. W. Hansen and Q. Ji, "In the eye of the beholder:a survey of model for eyes and gaze," IEEE Trans. Pattern Anal. Mach. Intell., vol.32, no.3, pp. 478-500, Mar.2010.
[108]C. Topal, S. Gunal, O. Kocdeviren, A. Dogan, and O. N. Gerek, "A low-computational approach on gaze estimation with eye touch system," IEEE Trans. Cybern., vol.44, no.2, pp.228-239, Feb.2014.
[109]Z. Zhu and Q. Ji, "Novel eye gaze tracking techniques under natural head movement," IEEE Trans. Biomedical Eng., vol.54, no.12, pp.2246-2260, Dec. 2007.
[110]S. Zhao and R. R. Grigat, "Robust eye detection under active infrared illumi-nation," in Proc. ICPR, pp.481-484,2006.
[111]D. Cho and W. Kim, "Long-range gaze tracking system for large movements," IEEE Trans. Biomedical Eng., vol.60, no.12, pp.3432-3440, Dec.2013.
[112]温晴川,达飞鹏,方旭,“基于双目立体视觉的视线跟踪系统标定,”光学学报,vol.32,no.10,Oct.2012.
[113]D. Cho, W. Yap, H. Lee, I. Lee, and W. Kim, "Long range eye gaze tracking system for a large screen," IEEE Trans. Consum. Electron., vol.58, no.4, pp. 1119-1128, Nov.2012.
[114]J. Xiong, W. Xu, W. Liao, Q. Wang, J. Liu, and Q. Liang, "Eye control system base on ameliorated hough transform algorithm," IEEE Sens. J., vol.13, no.9, pp.3421-3429, Sep.2013.
[115]D. H. Yoo, J. H. Kim, B. R. Lee, and M. J. Chung, "Non-contact eye gaze tracking system by mapping of corneal reflections," in Proc. IEEE Conf. Auto-matic Face and Gesture Recognition, pp.94-99, May,2002
[116]Y. Feng, G. Cheung, W. Tan, P. Callet, and Y. Ji, "Low-cost eye gaze predic-tion system for interactive networked video streaming," IEEE Trans. Multime-dia, vol.15, no.8, pp.1865-1879, Dec.2013.
[117]H. Hadizadeh, M. J. Enriquez, and I. V. Bajic, "Eye-tracking database for a set of standard video sequences," IEEE Trans. Image Process., vol.21, no.2, pp.898-903, Feb.2012.
[118]X. Yang, J. Sun, J. Liu, X. Li, C. Yang, and W. Liu,, "A remote gaze tracking system using gray-distribution-based video processing," Biomed. Eng.:App. Bas. C., vol.24, no.3, pp.217-227, Jun.2012.
[119]W. Gander, G. H. Golub, and R. Strebel, "Least-square fitting of circles and ellipses," BIT Numerical Mathematics, vol.34, no.4, pp.558-578, Dec.1994.
[120]H. C. Lu, G. L. Fang, C. Wang, and Y. W. Chen, "A novel method for gaze tracking by local pattern model and support vector regressor," Signal Process., vol.90, no.4, pp.1290-1299, Apr.2010.
[121]A. Meyer, M. Bohme, T. Martinetz, and E. Barth, "A single-camera remote eye tracker," Perception and Interaction Technologies, vol.4021/2006, pp.208-211. Jun.2006.
[122]S. W. Shih and J. Liu, "A novel approach to 3-D gaze tracking using stereo cameras," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.34, no.1, pp. 234-245, Apr.2004.
[123]E. D. Guestrin and M. Eizenman,"General theory of remote gaze estimation using the pupil center and corneal reflections," IEEE Trans. Biomedical Eng., vol.53, no.6, pp.1124-1133, Jun.2006.
[2]A. Gotchev, G. Bozdagi, T. Capin, D. Strohmeier, and A. Boev, "Three-dimensional media for mobile devices," Proc. IEEE, vol.99, no.4, pp.708-741, Apr.2011.
[3]C. Fehn, R. Barre, and R. S. Pastoor, "Interactive 3-DTV concepts and key technologies," Proc. IEEE, vol.94, no.3, pp.524-538, Mar.2006.
[4]C. Zitnick, S. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation,"ACM Trans. Graphics (Proc. SIGGRAPH), vol.23, no.3, pp.600-608, Aug.2004.
[5]M. Tanimoto, "Overview of free viewpoint television," Signal Process. Image Commun., vol.21, no.6, pp.454-461, Jul.2006.
[6]Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto, "View genera-tion with 3d warping using depth information for FTV," Signal Process. Image Commun., vol.24, no.1-2, pp.65-72, Jan.2009.
[7]M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, "Free-viewpoint TV,: IEEE Signal Process. Mag., vol.28, no.1, pp.61-76, Jan.2011.
[8]M. Tanimoto, "FTV:Free-viewpoint television," Signal Process. Image Com-mun., vol.27, no.6, pp.555-570, Jul.2012.
[9]D. Min, D. Kim, S. Yun, and K. Sohn. "2D/3D freeview video generation for 3DTV system," Signal Process. Image Commun., vol.24, no.1-2, pp.31-48, Jan.2009.
[10]L. Hill and A. Jacobs, "3-D liquid crystal display and their applications," Proc. IEEE, vol.94, no.3, pp.575-590, Mar.2006.
[11]J. Choi, D. Min, and K. Sohn, "2D-plus-depth based resolution and frame-rate up-conversion technique for depth video," IEEE Trans. Consum. Electron., vol. 56, no.4, pp.2489-2497, Nov.2010.
[12]D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comp. Vis., vol.47, no.1, pp.7-42, Apr.2002.
[13]J. W. Nadler, D. Barbash, H. R. Kim, S. Shimpi, D. E. Angelaki, and G. C. DeAngelis, "Joint representation of depth from motion parallax and binocular disparity cues in macaque area MT," J. Neurosci., vol.33, no.35, pp.14061-14074, Aug.2013.
[14]A. Saxena, M. Sun, and A. Y. Ng, "Make3D:learning 3D scene structure from a single still image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.5, pp.824-840, May 2009.
[15]Y. L. Chang, C. Y. Fang, L. F. Ding, and S. Y. Chen, "Depth map generation for 2D-to-3D conversion by short-term motion assisted color segmentation," in Proc. ICME, pp.1958-1961, Jul.2007.
[16]X. Cao, Z. Li, and Q. Dai, "Semi-automatic 2D-to-3D conversion using disparity propagation," IEEE Trans. Broadcast., vol.57, no.2, pp.491-499, Jun.2011.
[17]C. G. Gurler, B. Gorkemli, G. Saygili, and A. M. Tekalp, "Flexible transport of 3-d video over networks," Proc. IEEE, vol.99, no.4, pp.694-707, Apr.2011.
[18]K. Muller, P. Merkle, and T. Wiegand, "3-D video representation using depth maps," Proc. IEEE, vol.99, no.4, pp.643-656, Apr.2011.
[19]H. Urey, K. V. Chellappan, E. Erden, and P. Surman, "State of the art in stereoscopic and autostereoscopic displays," Proc. IEEE, vol.99, no.4, pp. 540-555, Apr.2011.
[20]J. D. Yun, Y. Kwak, and S. Yang, "Evaluation of perceptual resolution and crosstalk in sterescopic displays," J. Display Techno., vol.9, no.2, pp.106-111, Feb.2013.
[21]G. Lv, W. Zhao, D. Li, and Q. Wang, "Polarizer parallax barrier 3d display with high brightness, resolution and low crosstalk," J. Display Technol., vol.10, no.2, pp.120-124, Feb.2014.
[22]V. A. Ezhov, "Distant binocular filters for full-resolution autostereoscopic view-ing and for single-aperture stereo glasses," J. Display Technol., vol.10, no.2. pp.114-119, Feb.2014.
[23]C. T. E. R. Hewage, S. T. Worrall, S. Dogan, S. Villette, and A. M. Kondoz, "Qulity evaluating of color plus depth map-based stereoscopic video," IEEE J. Sel. Topics Signal Process., vol.3, no.2, pp.304-318, Apr.2009.
[24]P. Lebreton, A. Raake, M. Barkowsky, and P. L. Callet, "Evaluating depth perception of 3D stereoscopic videos," IEEE J. Sel. Topics Signal Process., vol. 6, no.6, pp.710-720, Oct.2012.
[25]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. "Imago quality assessment:From error visibility to structure similarity." IEEE Trans. Image Process., vol.13, no.4, pp.600-612, Apr.2004.
[26]D. V. S. X. De Silva. W. A. C. Fernando, G. Nur, E. Ekmekcioglu. and S. T. Worrall, "3D video assessment with just noticeable difference in depth evalua-tion," in Proc. ICIP, pp.4013-4016, Sep.2010.
[27]Y. Zhao, Z. Chen, C. Zhu, Y. P. Tan, and L. Yu, "Binocular just-noticeable-difference model for stereoscopic images," IEEE Signal Process. Lett., vol.18, no.1, pp.19-22, Jan.2011.
[28]T. Kim, J. Kang, S. Lee, and A. C. Bovik, "Multimodal interactive continuous scoring of subjective 3D video quality of experience," IEEE Trans. Multimedia, vol.16. no.2, pp.387-402, Feb.2014.
[29]Y. H. Lin and J. L. Wu, "Quality assessment of stereoscopic 3D image com-pression by binocular integration behaviors," IEEE Trans. Image Process., vol. 23, no.4, pp.1527-1542, Apr.2014.
[30]S. J. Kang, K. R. Cho, and Y. H. Kim, "Motion compensated frame rate up-conversion using extended bilateral motion estimation," IEEE Trans. Consum. Electron., vol.53, no.4, pp.1759-1767, Nov.2007.
[31]A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, "Multiview imaging and 3DTV," IEEE Signal Process. Mag., vol.24, no.7, pp. 10-21, Nov.2007.
[32]D. Wang, L. Zhang, and A. Vincent, "Motion-compensated frame rate up-conversion Part Ⅰ:Fast Multi-Frame Motion Estimation," IEEE Trans. Broad-cast, vol.56, no.2, pp.133-141, Jun.2010.
[33]D. Wang, A. Vincent, P. Blanchfield, and R. Klepko, "Motion-compensated frame rate up-conversion Part II:New algorithms for frame interpolation," IEEE Trans. Broadcast, vol.56, no.2, pp.142-149, Jun.2010.
[34]A. M. Huang and T. Q. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Pro-cess., vol.17, no.5, pp.694-708, May 2009.
[35]A. M. Huang and T. Q. Nguyen, "Correlation-Based Motion Vector Processing With Adaptive Interpolation Scheme for Motion-Compensated Frame Interpo-lation," IEEE Trans. Image Process., vol.18, no.4, pp.740-752, Apr.2009.
[36]M. J. Chen, L. G. Chen, and T. D Chiueh, "One-dimensional full search mo-tion estimation algorithm for video coding," IEEE Trans. Circuits Syst. Video Technol, vol.4, no.5, pp.504-509, Jan.1994.
[37]G. de Haan, P. W. A. C. Biezen, H. Huijgen, and O. A. Ojo, "True-motion estimation with 3-d recursive search block matching," IEEE Trans. Circuits Syst. Video Technol, vol.3, no.4, pp.368-379, Oct.1993.
[38]B. D. Choi, J. W. Han, C. S. Kim, and S. J. Ko, "Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation," IEEE Trans. Circuits Syst Video Technol, vol.17, no. 4, pp.407-416, Feb.2007.
[39]Z. Yu, H. Li, Z. Wang, Z. Hu, and C. Chen, "Multi-level video frame interpo-lation:Exploiting the interaction among different levels,"IEEE Trans. Circuits Syst. Video Technol., vol.23, no.7, pp.1235-1248, Jul.2013.
[40]贾茜,肖进胜,易本顺,甘良才,“基于三维递归搜索的多级运动估计视频帧率上转换算法,”电子与信息学报,vol.34,no.10.pp.2336-2341,2012.
[41]杨爱萍,董翠翠,侯正信,李娜,“基于多帧运动估计的帧率提升算法,”计算机应用研究,vol.29,no.10,pp.3952-3955,2012.
[42]Y. L. Lee and T. Nguyen, "Fast one-pass motion compensated frame inter-polation in high-definition video processing," in Proc. ICIP, pp.369-372, Nov. 2009.
[43]Y. Cho, H. Lee, and D. Park, "Temperal frame interpolation based on multiple feature trajectory," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.12. pp.2105-2115, Dec.2013.
[44]S. J. Kang, S. Yoo, and Y. H. Kim, "Dual Motion Estimation for Frame Rate Up-Conversion," IEEE Trans. Circuits Syst. Video Technol., vol.20, no.12, pp. 1909-1914, Dec.2010.
[45]X. Gao, C. Duanmu, and C. Zou, "A multilevel successive elimination algorithm for block matching motion estimation," IEEE Trans. Image Process., vol.9, no. 3, pp.501-504, Mar.2000.
[46]C. Wang, L. Zhang, Y. He, and Y. P. Tan, "Frame rate upconversion using trilateral filtering," IEEE Trans. Circuits Syst. Video Technol., vol.20, no.6, pp.886-893, Jun.2010.
[47]N. Jacobson, Y. Lee, V. Mahadevan, N. Vasconcelos, and T. Q. Nguyen, "A novel approach to fruc using discriminant saliency and frame segmentation,' IEEE Trans. Image Process., vol.19, no.11, pp.2924-2934, Nov.2010.
[48]Q. Lu and X. Fang, "Joint trilateral motion vector filter for bidirectional motion compensation," Electron. Lett., vol.49, no.13, pp.798-800,2013.
[49]D. Choi, W. Song, J. Park, H. Choi, and T. Kim, "A statistical approach to motion vector field smoothing for block-based motion-compensated frame interpolation," in Proc. ICASSP, pp.989-992,2012.
[50]S. G. Jeong, C. Lee, and C. S. Kim, "Example-based frame rate up-conversion with congruent segmentation," in Proc. ICIP, pp.845-848,2012.
[51]K. Chen and D. A. Lorenz, "Image sequence interpolation using optimal con-trol," J. Math. Imag. Vision, vol.41, no.3, pp.222-238,2011.
[52]M. Werlberger, T. Pock, M. Unger, and H. Bischof, "Optical flow guided TV-Ll video interpolation and restoration," in Proc. Energy Minimization Methods Comput. Vision Pattern Recognit., vol.6819, pp.273-286,2011.
[53]W. Lee, K. Choi, and J. Ra, "Frame rate up conversion based on vatiational image fusion," IEEE Trans. Image Process., vol.23, no.1, pp.399-412, Jan. 2014.
[54]Q. Lu, X. Fang, C. Xu, and Y. Wang, "Frame rate up-conversion for depth-based 3D video," in Proc. ICME, pp.598-603, Jul.2012.
[55]K. Choi, "Motion estimation for stereoscopic video and its application to frame rate up-conversion for 3-DTV," in Proc. ICCE, pp.608-609, Jun.2012.
[56]Y. Lee, Z. Lee, and T. Nguyen, "Frame rate up conversion of 3D video by motion and depth fusion," in Proc. IVMSP Workshop, pp.1-4, Jun.2013.
[57]G. Dane and T. Q. Nguyen, "Smooth motion vector resampling for standard compatible video post-processing," in Proc. Asilomar Conf. Signals Syst. Com-put, pp.1731-1735, Nov.2004.
[58]Y. Guo, Z. Gao, L. Chen, and X. Zhang, "Occlusion handling frame rate up-conversion," in Proc. ICASSP, pp.1424-1428,2013.
[59]D. Kim, H. Lim, and H. Park, "Iterative true motion estimation for motion-compensated frame interpolation," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.3, pp.445-454, Mar.2013.
[60]X. Yang, J. Liu, J. Sun, Y. Lee, and T. Q. Nguyen, "Depth-assisted frame rate up-conversion for stereoscopic video," IEEE Signal Process. Lett., vol.21, no. 4, pp.423-427, Apr.2014.
[61]A. Levin, D. Lischinski, and Y. Weiss, "A closed form solution to natural image matting," in Proc. CVPR, pp.61-68, Jun.2006.
[62]C. Liu, "Beyond pixels:exploring new representations and applications for motion analysis," Doctoral Thesis., Massachusetts Institute of Technology, May, 2009.
[63]N. Jacobson and T. Q. Nguyen, "Scale-aware saliency for application to frame rate upconversio," IEEE Trans. Image Process., vol.21, no.4, pp.2198-2206, Apr.2012.
[64]X. Yang, J. Liu, J. Sun, X. Li, and W. Liu, "DIBR based view synthesis for free-viewpoint television,"in Proc. IEEE 3DTV-CON, pp.1-4. May 2011.
[65]A. Smolic, K. Muller, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, "Interme-diate view interpolation based on multi-view video plus depth for advanced 3D video systems," in Proc. ICIP, pp.2448-2451,2008.
[66]W. Li, J. Zhou, B. Li, and M. I. Sezan, "Virtual view specification and synthesis for free viewpoint television," IEEE Trans. Circuits Syst. Video Technol.,vol. 19, no.4, pp.533-546, Apr.2009.
[67]L. Zhang and W. Tam, "Stereoscopic image generation based on depth images for 3D TV," IEEE Trans. Broadcast, vol.51, no.2, pp.191-199, Jun.2005.
[68]C. Cheng, S. Lin, S. Lai, and C. Yang, "Improved novel view synthesis from depth image with large baseline," in Proc. ICPR,2008.
[69]L. Tran, C. Pal, and T. Nguyen, "View synthesis based on conditional random fields and graph cuts," in Proc. ICIP, pp.433-436, Sep.2010.
[70]K. Oh, S. Yea, A. Vetro, and Y. Ho, "Virtual view synthesis method and self evaluation metrics for free viewpoint television and 3D video," Int. J. Imag. Syst. Technol,. vol.20, no.4, pp.378-390, Dec.2010.
[71]Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, "Boundary artifact reduction in view synthesis of 3D video:from perspective of texture-depth alignment," IEEE Trans. Broadcast, vol.57, no.2, pp.510-522, Jun.2011.
[72]A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol.13, no.9, pp.1200-1212, Sep.2004.
[73]T. F. M. Tanimoto and K. Suzuki, View Synthesis Algorithm in View Synthe-sis Reference Software 2.0 (VSRS 2.0), ISO/IEC JTC1/SC29/WG11M16090, Lausanne, Switzerland, Feb.2008.
[74]I. Ahn and C. Kim, "A novel depth-based virtual view synthesis method for free viewpoint video," IEEE Trans. Broadcast., vol.59, no.4, pp.614-626, Dec. 2013.
[75]J. Habigt and K. Diepold, "Image completion for view synthesis using Markov random fields and efficient belief propagation," in Proc. ICIP, pp.2131-2134, Sep.2013.
[76]C. M. Cheng, S. J Lin, and S. H. Lai, "Spatio-temporally consistent novel view synthesis algorithm from video-plus-depth sequences for autostereoscopic displays," IEEE Trans. Broadcast, vol.57, no.2, pp.523-532, Jun.2011.
[77]S. Choi, B. Ham, and K. Sohn, "Space-time hole filling with random walks in view extrapolation for 3D video," IEEE Trans. Image Process., vol.22, no.6, pp.2429-2441, Jun.2013.
[78]H. A. Hsu, C. K. Chiang, and S. H. Lai "Spatio-temporally consistent view synthesis from video-plus-depth data with global optimization," IEEE Trans. Circuits Syst. Video Technol, vol.24, no.1, pp.74-84, Jan.2014.
[79]M. Solh and G. AlRegib, "Hierarchical hole-filling for depth-based view syn-thesis in FTV and 3D video," IEEE J. Sel. Topics Signal Process., vol.6, no. 5, pp.495-504, Sep.2012.
[80]I. Daribo and H. Saito, "A novel inpainting-based layered depth video for 3DTV," IEEE Trans. Broadcast, vol.57, no.2, pp.533-541, Jun.2011.
[81]W. Sun, O. C. Au,L. Xu. Y. Li, W. Hu, and Z. Yu, "Texture optimization for seamless view synthesis through energy minimization," in Proc. ACM Int. Conf. Multimedia, pp.733-736, Oct.2012.
[82]S. H. Chan, R. Khoshabeh, K. B. Gibson. P. E. Gill, and T. Q. Nguyen, "An augmented Lagrangian method for total variation video restoration," IEEE Trans. Image Process., vol.20, no.11, pp.3097-3111, Nov.2011.
[83]S. Jung. "Enhancement of image and depth map using adaptive joint trilateral filter," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.2, pp.258-269, Feb.2013.
[84]J. G. Wang and E. Sung, "Gaze determination via images of irises," Image Vision Comput., vol.19, no.12, pp.891-911, Oct.2001.
[85]P. M. Corcoran, F. Nanu. S. Petrescu, and P. Bigioi, "Real-time eye gaze track-ing for gaming design and consumer electronics systems," IEEE Trans. Consum. Electron., vol.58, no.2, pp.347-355, May 2012.
[86]C. S. Lin, C. W. Ho, W. C. Chen, C. C. Chiu, and M. S. Yeh, "A cascaded scheme for eye tracking and head movement compensation," Opt. Appl., vol. XXXVI, no.2-3, pp.401-412,2006.
[87]X. Xie, R. Sudhakar, and H. Zhuang, "Enhancement of image and depth map using adaptive joint trilateral filter," IEEE Trans. Syst., Man, Cybern., vol.28, no.4, pp.487-490, Jul.1998.
[88]C. S. Lin, C. N. Chan. Y. L. Lay, J. F. Lee, and M. S. Yeh, "An eye-tracking human-machine interface using an auto correction method," J. Med. Biol. Eng., vol.27. no.2, pp.105-109, Mar.2007.
[89]T. Ohno and N. Mukawa, "A free-head, simple calibration, gaze tracking sys-tem that enables gaze-based interaction," in Proc. Eye Tracking Research and Applications Symp., pp.115-122, Mar.2004.
[90]A. Amir, L. Zimet, A. Sangiovanni-Vincentelli, and S. Kao, "An embedded system for an eye-detection sensor," Comput. Vis. Image Und., vol.98, no.1, pp.104-123, Apr.2005.
[91]D. Li, D. Winfield, and D. J. Parkhurst, "Starburst:A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches,' in Proc. CVPR, pp.79-86, Jun.2005.
[92]T. Ohno, N. Mukawa, and A. Yoshikawa, "Gaze tracking system for gaze-based human-computer interaction," NTT Technical Review, vol.1, no.2, pp.35-39, 2003.
[93]X. Yang, J. Sun, J. Liu, J. Chu, W. Liu, and Y. Gao, "A gaze tracking scheme for eye-based intelligent control," in Proc. IEEE WCICA, pp.50-55, Jul.2010.
[94]C. Ahlstrom, T. Victor, C. Wege, and E. Steinmetz, "Processing of eye/head-tracking data in large-scale naturalistic driving data sets," IEEE Trans. Intell. Transp. Syst., vol.13, no.2, pp.553-564, Jun.2012.
[95]X. Fu, X. Guan, E. Peli, H. Liu, and G. Luo, "Automatic calibration method for driver's head orientation in natural driving environment," IEEE Trans. Intell. Transp. Syst., vol.14, no.1, pp.303-312, Mar.2013.
[96]C. H. Morimoto, D. Koons, A. Amit, M. Flickner, and S. Zhai, "Keeping an eye for HCI," in Proc. XII Brazilian Symposium on Computer Graphics and Image Processing, pp.171-176, Mar.1999.
[97]Q. Ji and Z. Zhu, "Eye and tracking for interactive graphic display," Internet. Symp. on Smart Graphics, Jun.2002.
[98]D. Beymer and M. Flickner, "Eye gaze tracking using an active stereo head,' in Proc. CVPR, pp.451-458,2003.
[99]D. H. Yoo and M. J. Chung, "A novel nonintrusive eye gaze estimation using cross-ratio under large head motion," Comput. Vis. Image Und., vol.98, no.1, pp.25-51, Apr.2005.
[100]A. Villanueva and R. Cabeza. "A novel gaze estimation system with one cali-bration point," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.38, no.4, pp. 1123-1138, Aug.2008.
[101]C. Hennessey and P. Lawrence, "Improving the accuracy and reliability of re-mote system-calibration-free eye-gaze tracking," IEEE Trans. Biomedical Eng., vol.56, no.7, pp.1891-1900, Jul.2009.
[102]J. Sigut and S. A. Sidha. "Iris center corneal reflection method for gaze track-ing using visible light," IEEE Trans. Biomedical Eng., vol.58, no.2, pp.411-419, Feb.2011.
[103]刘瑞安,靳世久,宋维,张希坤：吴晓荣,“单摄像机视线跟踪,”计算机应用,vol.26.no.9,pp.2101-2104,Sep.2006.
[104]胡畔,董春鹏,吴晓荣,“基于视线跟踪技术的眼控鼠标,”天津师范大学学报,vol.27.no.3,pp.68-71,Sep.2007.
[105]程成,杜菁菁,蓝飞翔,“眼动交互的实时线性算法构造和实现,”电子学报,vol.37.no.4A,pp.12-15,Apr.2009.
[106]张闯,迟健男,张朝晖,高小亮,胡涛,王志良,“视线追踪系统中视线估计方法研究,”中国科学信息科学,vol.41,no.5,2011.
[107]D. W. Hansen and Q. Ji, "In the eye of the beholder:a survey of model for eyes and gaze," IEEE Trans. Pattern Anal. Mach. Intell., vol.32, no.3, pp. 478-500, Mar.2010.
[108]C. Topal, S. Gunal, O. Kocdeviren, A. Dogan, and O. N. Gerek, "A low-computational approach on gaze estimation with eye touch system," IEEE Trans. Cybern., vol.44, no.2, pp.228-239, Feb.2014.
[109]Z. Zhu and Q. Ji, "Novel eye gaze tracking techniques under natural head movement," IEEE Trans. Biomedical Eng., vol.54, no.12, pp.2246-2260, Dec. 2007.
[110]S. Zhao and R. R. Grigat, "Robust eye detection under active infrared illumi-nation," in Proc. ICPR, pp.481-484,2006.
[111]D. Cho and W. Kim, "Long-range gaze tracking system for large movements," IEEE Trans. Biomedical Eng., vol.60, no.12, pp.3432-3440, Dec.2013.
[112]温晴川,达飞鹏,方旭,“基于双目立体视觉的视线跟踪系统标定,”光学学报,vol.32,no.10,Oct.2012.
[113]D. Cho, W. Yap, H. Lee, I. Lee, and W. Kim, "Long range eye gaze tracking system for a large screen," IEEE Trans. Consum. Electron., vol.58, no.4, pp. 1119-1128, Nov.2012.
[114]J. Xiong, W. Xu, W. Liao, Q. Wang, J. Liu, and Q. Liang, "Eye control system base on ameliorated hough transform algorithm," IEEE Sens. J., vol.13, no.9, pp.3421-3429, Sep.2013.
[115]D. H. Yoo, J. H. Kim, B. R. Lee, and M. J. Chung, "Non-contact eye gaze tracking system by mapping of corneal reflections," in Proc. IEEE Conf. Auto-matic Face and Gesture Recognition, pp.94-99, May,2002
[116]Y. Feng, G. Cheung, W. Tan, P. Callet, and Y. Ji, "Low-cost eye gaze predic-tion system for interactive networked video streaming," IEEE Trans. Multime-dia, vol.15, no.8, pp.1865-1879, Dec.2013.
[117]H. Hadizadeh, M. J. Enriquez, and I. V. Bajic, "Eye-tracking database for a set of standard video sequences," IEEE Trans. Image Process., vol.21, no.2, pp.898-903, Feb.2012.
[118]X. Yang, J. Sun, J. Liu, X. Li, C. Yang, and W. Liu,, "A remote gaze tracking system using gray-distribution-based video processing," Biomed. Eng.:App. Bas. C., vol.24, no.3, pp.217-227, Jun.2012.
[119]W. Gander, G. H. Golub, and R. Strebel, "Least-square fitting of circles and ellipses," BIT Numerical Mathematics, vol.34, no.4, pp.558-578, Dec.1994.
[120]H. C. Lu, G. L. Fang, C. Wang, and Y. W. Chen, "A novel method for gaze tracking by local pattern model and support vector regressor," Signal Process., vol.90, no.4, pp.1290-1299, Apr.2010.
[121]A. Meyer, M. Bohme, T. Martinetz, and E. Barth, "A single-camera remote eye tracker," Perception and Interaction Technologies, vol.4021/2006, pp.208-211. Jun.2006.
[122]S. W. Shih and J. Liu, "A novel approach to 3-D gaze tracking using stereo cameras," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.34, no.1, pp. 234-245, Apr.2004.
[123]E. D. Guestrin and M. Eizenman,"General theory of remote gaze estimation using the pupil center and corneal reflections," IEEE Trans. Biomedical Eng., vol.53, no.6, pp.1124-1133, Jun.2006.