三维人脸的口型合成研究
|
摘要
具有真实感的人脸模拟是计算机图形工作者长期以来所追求的目标，三维人脸
的口型合成是其重要组成部分。该技术可以应用于通信技术，辅助教学，虚拟现
实，医疗研究，电影制作，游戏娱乐等诸多方面。
本文主要在基于文本的口型合成，行为驱动的口型合成，交互式人脸编辑合成
系统的设计和实现等方面进行了研究和探讨。
对于文本驱动的汉语口型合成，本文通过归纳汉语拼音发音基本规律，用七种
基本口型及其时间分配比例，完成了基于汉语拼音文本的特定人口型合成的设计
实现，并且使显示速度达到了30帧/秒，基本满足实时显示的需要。
对于视频图像口型行为驱动的口型合成，本文将Kriging估值算法引入唇部模
型变形中，设计了一种可行的变形方案，能够较真实地反映特定人的运动特征。
根据视频图像中提取的唇部特征点的位置信息，计算出唇部模型特征点的运动信
息，应用Kriging方法，驱动唇部模型的运动。
最后本文将上述技术和算法组合到交互式人脸编辑合成系统中，实现了可以进
行交互式编辑，可以实时显示三维人脸唇部动画的人脸合成系统，为三维人脸图
象库的建立提供了一个方便的编辑工具。本文所阐述的技术和算法，在数字通信，
计算机辅助教学以及三维游戏等方面都将会有广阔的应用前景。
In this thesis, the lip-motion driven by text or actions respectively, and the design
and implementation of the interactive human face editor are all discussed and
lucubrated.
By investigating the pronunciation rules of the Chinese Phonetic Word, I use seven
lip-shapes of viseme and their time proportion to drive the 3D facial model to
implement lip-motion.
In the lip motion driven by action aspect, I use Kriging Estimator to control the lip
model morphing result. According to the position of lip feature points extracted from
video images, the lip model can be driven with Kriging method.
I build an interactive human face editing and synthesis system, which can not only
interactively edit individual human face but also real-time display 3D lip animation with
30 frame per second. No matter now or in the future, the technique and algorithm
accomplished in this paper will play an important role in many aspects such as
communication, education and entertainment.
的口型合成是其重要组成部分。该技术可以应用于通信技术，辅助教学，虚拟现
实，医疗研究，电影制作，游戏娱乐等诸多方面。
本文主要在基于文本的口型合成，行为驱动的口型合成，交互式人脸编辑合成
系统的设计和实现等方面进行了研究和探讨。
对于文本驱动的汉语口型合成，本文通过归纳汉语拼音发音基本规律，用七种
基本口型及其时间分配比例，完成了基于汉语拼音文本的特定人口型合成的设计
实现，并且使显示速度达到了30帧/秒，基本满足实时显示的需要。
对于视频图像口型行为驱动的口型合成，本文将Kriging估值算法引入唇部模
型变形中，设计了一种可行的变形方案，能够较真实地反映特定人的运动特征。
根据视频图像中提取的唇部特征点的位置信息，计算出唇部模型特征点的运动信
息，应用Kriging方法，驱动唇部模型的运动。
最后本文将上述技术和算法组合到交互式人脸编辑合成系统中，实现了可以进
行交互式编辑，可以实时显示三维人脸唇部动画的人脸合成系统，为三维人脸图
象库的建立提供了一个方便的编辑工具。本文所阐述的技术和算法，在数字通信，
计算机辅助教学以及三维游戏等方面都将会有广阔的应用前景。
In this thesis, the lip-motion driven by text or actions respectively, and the design
and implementation of the interactive human face editor are all discussed and
lucubrated.
By investigating the pronunciation rules of the Chinese Phonetic Word, I use seven
lip-shapes of viseme and their time proportion to drive the 3D facial model to
implement lip-motion.
In the lip motion driven by action aspect, I use Kriging Estimator to control the lip
model morphing result. According to the position of lip feature points extracted from
video images, the lip model can be driven with Kriging method.
I build an interactive human face editing and synthesis system, which can not only
interactively edit individual human face but also real-time display 3D lip animation with
30 frame per second. No matter now or in the future, the technique and algorithm
accomplished in this paper will play an important role in many aspects such as
communication, education and entertainment.
引文
[1]R. Madsen. Animated Film: Concepts, Methods, Uses. Interland, New York,1969.
[2]G. Faifin. The Artist's Complete Guide to Facial Expressions. Watson-Guptill,New York, 1990.
[3]J. Kleiser. A fast, efficient, accurate way to represent the human face. In State of the Art in Facial Animation, SIGGRAPH'89 Turorials, Volume 22, pages 37-40.ACM, New York, 1989.
[4]高文，多功能感知机的框架结构。智能人机接口与智能应用学术会议'95论文集，1995：435-440
[5]F. I. Parke. Computer generated animation of gaces. Master's thesis,University of Utah, Salt Lake City, UT, June 1972. UTEC-CSc-72-120.
[6]F. I. Parke. A Parameteric Model for Human Faces. PhD thesis, University of Utah, Salt Lake City, UT, December 1974. UTEC-CDc-75-047.
[7]H. Chernoff. The use of faces to represent points in n-dimensional space graphically.Technical Report Project NR-042-993, Office of Naval Research,Washington, DC, December 1971
[8]Nadia Magnenat-Thalmann. New Trends in the Direction of Synthetic Actors in the Film Rendezvous a Montreal, IEEE computer Graphics&Application,December, 1987:9-19
[9]Nadia Magnenat-Thalmann. "New Trends in the Direction of Synthetic Actors", T.S.Chua, T.L.kunii(Eds), CGInternational'90 Springer-verlag Tokyo 1990:17-34
[10]Philip Lee,Suganna Wei,Jianmin Zhao and Norman I.Badler. Strength Guided Motion. Computer Graphics.August 1990, 24(4): 253-262
[11]F. Parke.Parameterized Models for Facial Animation Revisited in SIGGRAPH Facial Animation Tutorial Notes. ACM SIGGRAPH, 1989:43-56
[12]F. Parke. Parameterized Models for Facial Animation. IEEE Computer Graphics and Applications, November 1982, 2(9): 61-68
[13]Keith Water. A Muscle Model for Animating Three-Dimensional Facial Expression. Computer Graphics. July 1987, 21(4): 17-24
[14]D.Terzopoulos and K.Waters. Physically-based Facial Modeling, analysis,and animation. Visualization and computer Animation, 1990, 1:73-80
[15]Y.C.Lee, D.Terzopoulos, and K.Waters, Constructing physics based facial models of individuals." In proceedings of Graphics Interface'93, Toronto, May, 1993:1-8
[16]P. Ekman and W. V. Friesen. Facial Action Coding System. Consulting Psychologists Press Inc., 577 CollegeAvenue, PaloAlto, California94306, 1978:83-87
[17]Keith Waters and Terzopoulos. Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models. IEEE Transaction On Pattern Analysis And Machine Intelligence. June 1993, 15(6): 569-579
[18]John E.Chadwick David R.Haumann Richard E.Parent. Layered Construction for Deformable Animated Characters. Computer Graphics. July 1989,23(3): 243-252
[19]D. Terzopoulos and K. Waters. Physically-based facial modeling, analysis,and animation. J. Of Visualizaton and Computer Animation, 1(4): 73-80, March 1990
[20]D. Terzopoulos and K. Waters. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6): 569-579
[21]K. Waters and T. M. Levergood. DECface: an automatic lip synchronization algorithm for synthetic faces. Technical Report CRL 93/4, DEC Cambridge Research Laboratory, Cambridge, MA, September 1993.
[22]晏洁，具有真实感的三维人脸合成方法的研究与实践，哈尔滨工业大学、博士论文，1998年，图书分类号：TP 391.4, UDC: 681.39
[23]Ashok Samal, Pkasana A. Iyengar, Automation Recognition and Analysis of Human Face and Facial Expressions: A survey, .Pattern Recognition, 1992, 25(1):65～77
[24]Kazuhito Murakaml. Facial Caricaturing Basedon Visual Illusion: A Mechanism to Evaluate Caricaturein CASSO System. IEICE TRANS. INF. & SYST,1993, E76-D(4): 470～477
[25]晏洁，从一般人脸模型到特定人脸模型的修改，计算机工程与科学，1997年5月，19卷第2期
[26]Tarcisio Coianiz, Lorenzo Torresani, and Bruno Caprile, 2D Deformable Models for Visual Speech Analysis, In NATO Advanced Study Institute:Speechreading by Man and Machine, 1995.
[27]Michael Kass, Andrew Witkin, and Demetri Terzopoulous, Snake: Active Contour Models, International Journal of Computer Vision, pages 321-331, 1988.
[28]A. Adjoudani and J.Benoit, On the Integration of Auditory and Visual Parameters in an HMM-based ASR, In NATO Advanced Study Institute:Speechreading by Man and Machine, 1995.
[29]K. Waters and J. Frisbie, A Coordinated Muscle Model for Speech Animation, In Graphics Interface, pages 163-170, 1995.
[30]Sumit Basu, A Three-Dimensional Model of Human Lip Motion, Master's Thesis, M.I,T, February 1997, M.I.T MLPCSTR No.417
[31]中国社会科学院语言研究所词典编辑室，附录，现代汉语词典，商务印书馆，1983年，第2版
[32]现代汉语语音知识，湖北人民出版社 1974年10月 第1版
[33]郭锦桴，综合语音学，福建人民出版社，1993年8月 第1版
[34]冯隆 北京话语流中声韵调的时长，北京语音实验录，北京大学出版社1985年
[35]F. Thomas and O. Johnson. Disney Animation: The Illusion of Life.Abbeville Press, New York, 1981.
[36]F. Pighin, J. Hecker, D. Lischinski, R. Szeliski and D. Salesin, Synthesizing realistic facial expressions from photographs, In Computer Graphics Proceedings SIGGRAPH'98, page 75-84, 1998.
[37]Baocai Yin and Wen Gao, Radial Basis Function Interpolation on space mesh,ACM SIGGRAPH97, vitual Proc. pp150, 1997.
[38]Cyberware Laboratory, Inc, Monterey, Callifornia. 4020/RGB 3D Scanner with Color Digitizer, 1990.
[39]Volker Blanz and Thomas Vetter, A Morphable Model For The Synthesis of 3D Faces, In SIGGRAPH 99 Conference Proceedings.
[40]Horace H.S.Ip and Lijun Yin. Constructing a 3D individualized head model from two orthogonal views. The Visual Computer 12: 254-266, 1996.
[41]M.J.T. Reinders, B. Sankur and J.C.A. van der Lubbe. Transformation of a general 3D facial model to an actual scene face. IEEE Pages 75-78, 1992.
[42]Chao-yi Lang, Kriging Interpolation,,June, 1998.
[43]Lsaaks and Srivastava, An Introduction to Applied Geostatistics, Oxford University Press, 1989.
[2]G. Faifin. The Artist's Complete Guide to Facial Expressions. Watson-Guptill,New York, 1990.
[3]J. Kleiser. A fast, efficient, accurate way to represent the human face. In State of the Art in Facial Animation, SIGGRAPH'89 Turorials, Volume 22, pages 37-40.ACM, New York, 1989.
[4]高文，多功能感知机的框架结构。智能人机接口与智能应用学术会议'95论文集，1995：435-440
[5]F. I. Parke. Computer generated animation of gaces. Master's thesis,University of Utah, Salt Lake City, UT, June 1972. UTEC-CSc-72-120.
[6]F. I. Parke. A Parameteric Model for Human Faces. PhD thesis, University of Utah, Salt Lake City, UT, December 1974. UTEC-CDc-75-047.
[7]H. Chernoff. The use of faces to represent points in n-dimensional space graphically.Technical Report Project NR-042-993, Office of Naval Research,Washington, DC, December 1971
[8]Nadia Magnenat-Thalmann. New Trends in the Direction of Synthetic Actors in the Film Rendezvous a Montreal, IEEE computer Graphics&Application,December, 1987:9-19
[9]Nadia Magnenat-Thalmann. "New Trends in the Direction of Synthetic Actors", T.S.Chua, T.L.kunii(Eds), CGInternational'90 Springer-verlag Tokyo 1990:17-34
[10]Philip Lee,Suganna Wei,Jianmin Zhao and Norman I.Badler. Strength Guided Motion. Computer Graphics.August 1990, 24(4): 253-262
[11]F. Parke.Parameterized Models for Facial Animation Revisited in SIGGRAPH Facial Animation Tutorial Notes. ACM SIGGRAPH, 1989:43-56
[12]F. Parke. Parameterized Models for Facial Animation. IEEE Computer Graphics and Applications, November 1982, 2(9): 61-68
[13]Keith Water. A Muscle Model for Animating Three-Dimensional Facial Expression. Computer Graphics. July 1987, 21(4): 17-24
[14]D.Terzopoulos and K.Waters. Physically-based Facial Modeling, analysis,and animation. Visualization and computer Animation, 1990, 1:73-80
[15]Y.C.Lee, D.Terzopoulos, and K.Waters, Constructing physics based facial models of individuals." In proceedings of Graphics Interface'93, Toronto, May, 1993:1-8
[16]P. Ekman and W. V. Friesen. Facial Action Coding System. Consulting Psychologists Press Inc., 577 CollegeAvenue, PaloAlto, California94306, 1978:83-87
[17]Keith Waters and Terzopoulos. Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models. IEEE Transaction On Pattern Analysis And Machine Intelligence. June 1993, 15(6): 569-579
[18]John E.Chadwick David R.Haumann Richard E.Parent. Layered Construction for Deformable Animated Characters. Computer Graphics. July 1989,23(3): 243-252
[19]D. Terzopoulos and K. Waters. Physically-based facial modeling, analysis,and animation. J. Of Visualizaton and Computer Animation, 1(4): 73-80, March 1990
[20]D. Terzopoulos and K. Waters. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6): 569-579
[21]K. Waters and T. M. Levergood. DECface: an automatic lip synchronization algorithm for synthetic faces. Technical Report CRL 93/4, DEC Cambridge Research Laboratory, Cambridge, MA, September 1993.
[22]晏洁，具有真实感的三维人脸合成方法的研究与实践，哈尔滨工业大学、博士论文，1998年，图书分类号：TP 391.4, UDC: 681.39
[23]Ashok Samal, Pkasana A. Iyengar, Automation Recognition and Analysis of Human Face and Facial Expressions: A survey, .Pattern Recognition, 1992, 25(1):65～77
[24]Kazuhito Murakaml. Facial Caricaturing Basedon Visual Illusion: A Mechanism to Evaluate Caricaturein CASSO System. IEICE TRANS. INF. & SYST,1993, E76-D(4): 470～477
[25]晏洁，从一般人脸模型到特定人脸模型的修改，计算机工程与科学，1997年5月，19卷第2期
[26]Tarcisio Coianiz, Lorenzo Torresani, and Bruno Caprile, 2D Deformable Models for Visual Speech Analysis, In NATO Advanced Study Institute:Speechreading by Man and Machine, 1995.
[27]Michael Kass, Andrew Witkin, and Demetri Terzopoulous, Snake: Active Contour Models, International Journal of Computer Vision, pages 321-331, 1988.
[28]A. Adjoudani and J.Benoit, On the Integration of Auditory and Visual Parameters in an HMM-based ASR, In NATO Advanced Study Institute:Speechreading by Man and Machine, 1995.
[29]K. Waters and J. Frisbie, A Coordinated Muscle Model for Speech Animation, In Graphics Interface, pages 163-170, 1995.
[30]Sumit Basu, A Three-Dimensional Model of Human Lip Motion, Master's Thesis, M.I,T, February 1997, M.I.T MLPCSTR No.417
[31]中国社会科学院语言研究所词典编辑室，附录，现代汉语词典，商务印书馆，1983年，第2版
[32]现代汉语语音知识，湖北人民出版社 1974年10月 第1版
[33]郭锦桴，综合语音学，福建人民出版社，1993年8月 第1版
[34]冯隆 北京话语流中声韵调的时长，北京语音实验录，北京大学出版社1985年
[35]F. Thomas and O. Johnson. Disney Animation: The Illusion of Life.Abbeville Press, New York, 1981.
[36]F. Pighin, J. Hecker, D. Lischinski, R. Szeliski and D. Salesin, Synthesizing realistic facial expressions from photographs, In Computer Graphics Proceedings SIGGRAPH'98, page 75-84, 1998.
[37]Baocai Yin and Wen Gao, Radial Basis Function Interpolation on space mesh,ACM SIGGRAPH97, vitual Proc. pp150, 1997.
[38]Cyberware Laboratory, Inc, Monterey, Callifornia. 4020/RGB 3D Scanner with Color Digitizer, 1990.
[39]Volker Blanz and Thomas Vetter, A Morphable Model For The Synthesis of 3D Faces, In SIGGRAPH 99 Conference Proceedings.
[40]Horace H.S.Ip and Lijun Yin. Constructing a 3D individualized head model from two orthogonal views. The Visual Computer 12: 254-266, 1996.
[41]M.J.T. Reinders, B. Sankur and J.C.A. van der Lubbe. Transformation of a general 3D facial model to an actual scene face. IEEE Pages 75-78, 1992.
[42]Chao-yi Lang, Kriging Interpolation,
[43]Lsaaks and Srivastava, An Introduction to Applied Geostatistics, Oxford University Press, 1989.