Model-Based Face Capture from Orthogonal Images

Brian Eng || Douglas R. Lanman || Robin Mayes

Project Summary:
From a survey of techniques, we implemented a novel method for creating photo-realistic 3D facial models from 2D images of a human subject. Starting from two orthogonal views, we employ a user-assisted procedure to recover 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation algorithm is then used to deform a generic face mesh to fit the particular geometry of the subject's face. We then create a model texture map by combining both views. Using our technique, we have been able to generate highly realistic facial models.

The Process:
   - Image capture
   - Deforming the model
   - Laplacian smoothing
   - Cylindrical projection
   - Texture mapping
   - Post-capture image editing

The Results:
3D Models:
The Cast -
Doug Lanman - Mask- Full Head
Robin Mayes - Mask- Full Head
Brian Eng - Mask- Full Head

The Prof. -
Gabriel Taubin - Mask - Full Head

The Inspirations -
Bug (from A Bug's Life doll) - Mask - Full Head
Homer Simpson - Mask- Full Head

Enhancements (to be implemented by YOU):
Full body capture
Expression morphing
Animations
Caricatures

Timeline:
Preliminary: Project Proposal *old*

- Week 1: Image capture and face model deformation
- Week 2: Cylindrical projection and texture mapping (stuck)
- Week 3: Thinking and doing other work for finals
- Week 4: Fixed cylindrical projection and texture mapping
- Week 5: Full head capture, Laplacian smoothing, web page, post-image processing

Contributions:
Project Supervisor: Gabriel Taubin
Image Capture and User Interface: Doug Lanman
Model Deformation : Doug Lanman
Laplacian Smoothing: Robin Mayes
Cylindrical Projection: Brian Eng and Robin Mayes
Texture Mapping: Doug Lanman, Brian Eng, Robin Mayes
Post-capture Image Editing: Brian Eng
Project Web Pages and Images: Brian Eng
Technical Support: Silvio Savarese

References:
1. Frederic Pighin, Jamie Hecker, Dani Lischinski, Richard Szeliski, and David Salesin. Synthesizing Realistic Facial Expressions from Photographs. Proceedings of SIGGRAPH 98, in Computer Graphics Proceedings, AnnualConference Series, 1998.
(http://grail.cs.washington.edu/pub/abstracts.html)
2. Horace H.S. Ip and Lijun Yin. Constructing a 3D individualized head model from two orthogonal views in The Visual Computer, pp. 254-266,1996.
3. Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, and Michael Cohen. Rapid Modeling of Animated Faces from Video. Technical Report MSR-TR-2000-11,Microsoft Research, February 2000.
(http://ftp.research.microsoft.com/pub/tr/tr-2000-11.pdf)
4. Gabriel Taubin. Geometric Signal Processing on Polygonal Meshes, in Eurographics 2000 State of the Art Report (STAR), September 2000.
(http://mesh.caltech.edu/ee148/refs/Taubin-star-eg00.pdf)

Related Links:
1. Douglas Fidaleo, Jun-yong Noh, Taeyong Kim, Reyes Encisco, and Ulrich Neumann. Classification and Volume Morphing for Performance-Driven Facial Animation.
- http://merong.usc.edu/Papers/dcv99Anim.pdf
2. Outline of Pighin's Paper as described for a class
- http://www.palantir.swarthmore.edu/~maxwell/classes/e93/graphics/outline/wk07_facialExpression.pdf
3. Janet He and Hui Ji. Project report for face model capture similar to our project
- http://www.math.umd.edu/~xhe/C733/733.html
4. Parallel Graphics - home of the Cortona VRML plugin for viewing our models
- http://www.parallelgraphics.com

Source Code and Data:
Original Mask Capture- file list - download it now
Full Head Capture- file list - download it now