具有真实感的特定人脸模型合成方法的研究
|
摘要
人脸是人们最熟悉的器官，但人脸合成、表情模拟等对于图形学领域的研究
者来说，仍是一个具有挑战性并且值得深入研究的课题。本文在国内外在该领域
研究的基础上，结合课题的需要，提出了一个人脸编辑的系统，实现了从图象构
造人脸模型的方法，采取了纹理映射的方法来增强真实感，并在总结了汉语发音
规律的基础上对汉语的口型进行了模拟。
我们在课题中提出了Kriging的整体变形的方法，并提供了友好的界面以便
利用鼠标对模型进行适当的修改，通过运用我们提供的技术可以获得较为满意的
变形效果。
在具有真实感的人脸模型显示中，我们采用了子剖分的曲面生成方法对初始
的三角形模型进行细化，从而构造了光滑的人脸模型，改善了显示效果。为了作
到人脸动画的实时显示和增强真实感，我们采用了纹理映射的技术，我们将多副
人脸图象合成一副纹理图象，并利用OpenGL提供的纹理映射功能把纹理贴到三
维人脸模型上，获得了较为理想的效果。
汉语口型的模拟是本文的另一个重要部分，我们提炼出了汉语的七种基本口
型，利用唇部的抛物线模型获得了七种基本口型的数据，并归纳出汉语拼音与音
素、音素和口型的关系，当输入拼音流时，系统进行分析，将其分解成口型单位，
并给出口型单位间的时间比、连接等关系，利用这些控制模型作出适当的口型动
作。
本文的研究和实验表明，通过采用合理的技术，构造具有真实感的人脸模型
完全是可能的，而人脸模型的构造、合成、模拟等将在改善人机接口等许多领域
发挥重要的作用。
The human face is the most familiar organ to us, but to researchers in the
computer graph field, the facial synthesis and animation are still subjects with great
challenge and worth deeply studying. In this paper, we present a facial edition system
based on the previous work of this field, realize a new method to construct a specific
face model, employ the texture mapping method to enhance realism and simulate the
lip motion based on the rule of pinyin.
In this paper, we present a new morphing method riging method. We also
provide a convenient way to modify the face model by mouse. Experiment shows that
our method has good effect.
In the realistic face graph display, we use the subdivision method to refine the
face model. We construct smooth surface by using this method. We also employ the
texture mapping technique. First, we synthesize a texture image by merging four face
images, then we map the texture image onto face model using the texture mapping
function provided by OpenGL.
The simulation of Chinese mouth shape is another important part of this paper.
We summarize serve basic mouth shapes, gain the data of these serve basic lip
motions using parabola model. We also summarize the relation between pinyin and
phoneme. When we feed the stream of pinyin into our system, the system analyses
this stream and separates it into units of viseme, the system also gives parameters
such as the time of a unit and the joint of units, then the system uses these parameters
to operate the facial model.
Research and experiment in this paper shows that constructing a realistic face
model is fully possible by using reasonable technique. We believe that the
constructing and synthesizing of face model will play important role in
human-machine interface.
者来说，仍是一个具有挑战性并且值得深入研究的课题。本文在国内外在该领域
研究的基础上，结合课题的需要，提出了一个人脸编辑的系统，实现了从图象构
造人脸模型的方法，采取了纹理映射的方法来增强真实感，并在总结了汉语发音
规律的基础上对汉语的口型进行了模拟。
我们在课题中提出了Kriging的整体变形的方法，并提供了友好的界面以便
利用鼠标对模型进行适当的修改，通过运用我们提供的技术可以获得较为满意的
变形效果。
在具有真实感的人脸模型显示中，我们采用了子剖分的曲面生成方法对初始
的三角形模型进行细化，从而构造了光滑的人脸模型，改善了显示效果。为了作
到人脸动画的实时显示和增强真实感，我们采用了纹理映射的技术，我们将多副
人脸图象合成一副纹理图象，并利用OpenGL提供的纹理映射功能把纹理贴到三
维人脸模型上，获得了较为理想的效果。
汉语口型的模拟是本文的另一个重要部分，我们提炼出了汉语的七种基本口
型，利用唇部的抛物线模型获得了七种基本口型的数据，并归纳出汉语拼音与音
素、音素和口型的关系，当输入拼音流时，系统进行分析，将其分解成口型单位，
并给出口型单位间的时间比、连接等关系，利用这些控制模型作出适当的口型动
作。
本文的研究和实验表明，通过采用合理的技术，构造具有真实感的人脸模型
完全是可能的，而人脸模型的构造、合成、模拟等将在改善人机接口等许多领域
发挥重要的作用。
The human face is the most familiar organ to us, but to researchers in the
computer graph field, the facial synthesis and animation are still subjects with great
challenge and worth deeply studying. In this paper, we present a facial edition system
based on the previous work of this field, realize a new method to construct a specific
face model, employ the texture mapping method to enhance realism and simulate the
lip motion based on the rule of pinyin.
In this paper, we present a new morphing method riging method. We also
provide a convenient way to modify the face model by mouse. Experiment shows that
our method has good effect.
In the realistic face graph display, we use the subdivision method to refine the
face model. We construct smooth surface by using this method. We also employ the
texture mapping technique. First, we synthesize a texture image by merging four face
images, then we map the texture image onto face model using the texture mapping
function provided by OpenGL.
The simulation of Chinese mouth shape is another important part of this paper.
We summarize serve basic mouth shapes, gain the data of these serve basic lip
motions using parabola model. We also summarize the relation between pinyin and
phoneme. When we feed the stream of pinyin into our system, the system analyses
this stream and separates it into units of viseme, the system also gives parameters
such as the time of a unit and the joint of units, then the system uses these parameters
to operate the facial model.
Research and experiment in this paper shows that constructing a realistic face
model is fully possible by using reasonable technique. We believe that the
constructing and synthesizing of face model will play important role in
human-machine interface.
引文
[1]Frederic Pighin,Joel Auslander, Dani Lischinski, David H.Salesin,Richard Szeliski. Realistic Facial Animation Using Image-Based 3D Morphing.TR UW-CSE-97-01-03,1997.
[2]Frederic Pighin,Joel Auslander, Dani Lischinski, David H.Salesin,Richard Szeliski.Synthesizing Realistic Facial Expressions from Photographs. Computer Graphics Proceedings, Annual Conference Series, pp75-84,1998.
[3]Irfan Aziz Essa .Analysis Interpretation and Synthesis of Facial Expressions. MIT PH.D paper, 1995.
[4]Horace H.S.Ip and Lijun,Yin. Constructing a 3D Individualized Head Model from Two Orthogonal Views. The Visual Computer(1996)12:254-266.
[5]Tsuneya Kurihara and Kiyoshi Arai.A Transformation Method for Modeling and Animation of the Human Face from Photographs. In Nadia Magnenat Thalmann and Daniel Thalmann, editors. Computer Animation 91, pp45-58, Springer-Verlag,Tokyo, 1991.
[6]Keith Waters and Thomas M.Levergood.DECface: An Automatic Lip-Synthronization Algorithm for Synthetic Faces.DEC Cambridge Research Lab, CRL 93/4 ,September 23,1993.
[7]M.J.T. Reinders, B.Sankur and J.C.A van der Lubbe. Transformation of a general 3D facial model to an actual scene face.pp75-78,1995.
[8]David R.Hill ,Andrew Pearce, and Brian Wyvill. Animating speech: an automated approach using speech synthesised by rules. The Visual Computer, (1988)3:277-289, Springer-Verlag 1988.
[9]Irfan Essa, Sumit Basu, Trevor Darrell ,Alex Pentland .Modeling ,Tracking and Interactive Animation of Faces and Heads using Input from Video.M.I.T Media Lab Perceptual Computing Section Technical Report No.370.1996.
[10]Denis Zorin ,Peter Schroder, Wim Sweldens. Interpolating Subdivision for Meshes with Arbitrary Topology. SIGGRAPH 1996,pp189-192.
[11]Kobbelt,L. Interpolatory subdivision on open quadrilateral nets with arbitrary topology. Computer Graphics Forum(1996),vol 15Eurographics,Basil Blackwell Ltd. Eurographics'96 Conference issue.
[12]E.Catmull,J.Clark,Recursively generated B-spline surfaces on arbitrary topological meshes, CAD 10(1978),pp.350-355.
[13]David Baraff ,Andrew Witkin. Large Steps in Cloth Simulation. SIGGRAPH 98,Orlando,July 19-24,pp43-52,
[14]Heung-Yeung Shum and Richard Szeliski. Construction and refinement of panoramic mosaics with global and local alignment. Microsoft Research TR, 1997.
[15]R.F. Koch,M.h. Gross etc. Simulating Facial Surgery Using Finite Element Models.SIGGRAPH 96,pp421-428.
[16]Ken-ichi Anjyo, Yoshiaki Usami, Tsuneya Kurihara. A Simple method for Extracting the Natural Beauty of Hair.Computer Graphics 26,2,July 1992.pp111-120.
[17]晏洁，文本驱动的面部表情及唇动合成系统，哈尔滨工业大学硕士学位论文，1996
[18]晏洁，文本驱动的面部表情及唇动合成系统，哈尔滨工业大学博士学位论文，1999
[19]陈雪村，基于几何模型的面部图象合成方法的研究，北京工业大学硕士学位论文，1999
[20]Sumit Basu .A Three Dimensional Model of Human Lip Motion. M.I.T Media Lab Perceptual Computing Section Technical Report No.417,1997.
[21]Ali Azarbayejani and Alex Pentland. Camera self-calibration from one point correspondence. M.I.T Media Lab Perceptual Computing Section Technical Report No.341,1997.
[22]Richard Szeliski and Sing bing Kang. Recovering 3D Shape and Motion from Image Streams using Non-Linear Least Squares. DEC Cambridge Research Lab,CRL 93/3,1993.
[23]Roberto Brunelli. Estimation of Pose and Illuminant Direction for Face Processing. M.I.T AI Lab,1994.
[24]Volker Blanz,Thomas Vetter. A Morphable Model for The Synthesis of 3D Facial.SIGGRAPH99,1999.
[25]Denis Zorin ,Peter Schroder, Wim Sweldens.Interpolating Subdivision for Meshes with Arbitrary Topology. Caltech TR.
[26]Ulrich reif and Peter Schroder. Curvature smoothness of subdivision surfaces. SIGGRAPH99,1999.
[27]孙家广，杨长贵。计算机图形学，清华大学出版社，北京，1995。
[28]Frederic I.Parke and Keith Waters. Computer Facial Antimation, A.K Peters,Wellesley, Massachusetts, 1996.
[29]高文，多功能感知机的框架结构。智能人机接口与智能应用学术会议'95论文集，1995：435～440
[30]P. Ekman and W. V. Friesen. Facial Action Coding System. Consulting Psychologists Press Inc., 577 CollegeAvenue, PaloAlto, California94306, 1978: 83-87
[31]Keith Waters and Terzopoulos. Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models. IEEE Transaction On Pattern Analysis And Machine Intelligence. June 1993, 15(6): 569-579
[32]John E.Chadwick David R.Haumann Richard E.Parent. Layered Construction for Deformable Animated Characters. Computer Graphics. July 1989, 23(3): 243-252
[33]晏洁，高文，尹宝才，具有真实感的三维虚拟特定人脸生成方法，计算机学报，1999，22(2)：147-153
[34]Wen Gao, Baocai Yin, Jie Yan, Individual Facial Image synthesis, Proc. JCSSP 97, Nov. 14-15, Tokyo garden Palace, Tokyo, Japan, 1997:45-51
[35]郭锦桴。综合语音学，福建人民出版社，福州，1992
[36]Loop,C. Smooth Subdivision Surfaces Based on Triangles. Master's thesis,University of Utah ,Department of Mathematics(1987).
[37]Doo,D and Sabin, M. Analysis of the Behaviour of Recursive Division Surfaces near Extraordinary Point. CAD 10,6(1978).pp356-360.
[38]Heung-Yeung Shum,Li-Wei He. Rendering with Concentric Mosaics. ACM ,Siggraph,Aug, 1999.
[2]Frederic Pighin,Joel Auslander, Dani Lischinski, David H.Salesin,Richard Szeliski.Synthesizing Realistic Facial Expressions from Photographs. Computer Graphics Proceedings, Annual Conference Series, pp75-84,1998.
[3]Irfan Aziz Essa .Analysis Interpretation and Synthesis of Facial Expressions. MIT PH.D paper, 1995.
[4]Horace H.S.Ip and Lijun,Yin. Constructing a 3D Individualized Head Model from Two Orthogonal Views. The Visual Computer(1996)12:254-266.
[5]Tsuneya Kurihara and Kiyoshi Arai.A Transformation Method for Modeling and Animation of the Human Face from Photographs. In Nadia Magnenat Thalmann and Daniel Thalmann, editors. Computer Animation 91, pp45-58, Springer-Verlag,Tokyo, 1991.
[6]Keith Waters and Thomas M.Levergood.DECface: An Automatic Lip-Synthronization Algorithm for Synthetic Faces.DEC Cambridge Research Lab, CRL 93/4 ,September 23,1993.
[7]M.J.T. Reinders, B.Sankur and J.C.A van der Lubbe. Transformation of a general 3D facial model to an actual scene face.pp75-78,1995.
[8]David R.Hill ,Andrew Pearce, and Brian Wyvill. Animating speech: an automated approach using speech synthesised by rules. The Visual Computer, (1988)3:277-289, Springer-Verlag 1988.
[9]Irfan Essa, Sumit Basu, Trevor Darrell ,Alex Pentland .Modeling ,Tracking and Interactive Animation of Faces and Heads using Input from Video.M.I.T Media Lab Perceptual Computing Section Technical Report No.370.1996.
[10]Denis Zorin ,Peter Schroder, Wim Sweldens. Interpolating Subdivision for Meshes with Arbitrary Topology. SIGGRAPH 1996,pp189-192.
[11]Kobbelt,L. Interpolatory subdivision on open quadrilateral nets with arbitrary topology. Computer Graphics Forum(1996),vol 15Eurographics,Basil Blackwell Ltd. Eurographics'96 Conference issue.
[12]E.Catmull,J.Clark,Recursively generated B-spline surfaces on arbitrary topological meshes, CAD 10(1978),pp.350-355.
[13]David Baraff ,Andrew Witkin. Large Steps in Cloth Simulation. SIGGRAPH 98,Orlando,July 19-24,pp43-52,
[14]Heung-Yeung Shum and Richard Szeliski. Construction and refinement of panoramic mosaics with global and local alignment. Microsoft Research TR, 1997.
[15]R.F. Koch,M.h. Gross etc. Simulating Facial Surgery Using Finite Element Models.SIGGRAPH 96,pp421-428.
[16]Ken-ichi Anjyo, Yoshiaki Usami, Tsuneya Kurihara. A Simple method for Extracting the Natural Beauty of Hair.Computer Graphics 26,2,July 1992.pp111-120.
[17]晏洁，文本驱动的面部表情及唇动合成系统，哈尔滨工业大学硕士学位论文，1996
[18]晏洁，文本驱动的面部表情及唇动合成系统，哈尔滨工业大学博士学位论文，1999
[19]陈雪村，基于几何模型的面部图象合成方法的研究，北京工业大学硕士学位论文，1999
[20]Sumit Basu .A Three Dimensional Model of Human Lip Motion. M.I.T Media Lab Perceptual Computing Section Technical Report No.417,1997.
[21]Ali Azarbayejani and Alex Pentland. Camera self-calibration from one point correspondence. M.I.T Media Lab Perceptual Computing Section Technical Report No.341,1997.
[22]Richard Szeliski and Sing bing Kang. Recovering 3D Shape and Motion from Image Streams using Non-Linear Least Squares. DEC Cambridge Research Lab,CRL 93/3,1993.
[23]Roberto Brunelli. Estimation of Pose and Illuminant Direction for Face Processing. M.I.T AI Lab,1994.
[24]Volker Blanz,Thomas Vetter. A Morphable Model for The Synthesis of 3D Facial.SIGGRAPH99,1999.
[25]Denis Zorin ,Peter Schroder, Wim Sweldens.Interpolating Subdivision for Meshes with Arbitrary Topology. Caltech TR.
[26]Ulrich reif and Peter Schroder. Curvature smoothness of subdivision surfaces. SIGGRAPH99,1999.
[27]孙家广，杨长贵。计算机图形学，清华大学出版社，北京，1995。
[28]Frederic I.Parke and Keith Waters. Computer Facial Antimation, A.K Peters,Wellesley, Massachusetts, 1996.
[29]高文，多功能感知机的框架结构。智能人机接口与智能应用学术会议'95论文集，1995：435～440
[30]P. Ekman and W. V. Friesen. Facial Action Coding System. Consulting Psychologists Press Inc., 577 CollegeAvenue, PaloAlto, California94306, 1978: 83-87
[31]Keith Waters and Terzopoulos. Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models. IEEE Transaction On Pattern Analysis And Machine Intelligence. June 1993, 15(6): 569-579
[32]John E.Chadwick David R.Haumann Richard E.Parent. Layered Construction for Deformable Animated Characters. Computer Graphics. July 1989, 23(3): 243-252
[33]晏洁，高文，尹宝才，具有真实感的三维虚拟特定人脸生成方法，计算机学报，1999，22(2)：147-153
[34]Wen Gao, Baocai Yin, Jie Yan, Individual Facial Image synthesis, Proc. JCSSP 97, Nov. 14-15, Tokyo garden Palace, Tokyo, Japan, 1997:45-51
[35]郭锦桴。综合语音学，福建人民出版社，福州，1992
[36]Loop,C. Smooth Subdivision Surfaces Based on Triangles. Master's thesis,University of Utah ,Department of Mathematics(1987).
[37]Doo,D and Sabin, M. Analysis of the Behaviour of Recursive Division Surfaces near Extraordinary Point. CAD 10,6(1978).pp356-360.
[38]Heung-Yeung Shum,Li-Wei He. Rendering with Concentric Mosaics. ACM ,Siggraph,Aug, 1999.