Project 3: Face Morphing

Background

A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The warp is controlled by defining a correspondence between the two pictures, for example by mapping eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc. In this project, we explored morphing by producing a "morph" animation from one face into another, computing the mean of a population of faces and extrapolating from a population mean to create a caricature of yourself.

1. Defining Correspondencies

Approach

First, I rotated and cropped the images to make them match as much as possible for better visual results in later parts. I used the labeling tool on the course website to select the correspondences between the two images. I used scipy.spatial.Delaunay to find the Delaunay triangulations of the two faces and plotted them using plt.triplot(pts[:, 0], pts[:, 1], tri.simplices), where tri is the triangulation.

Facial Keypoints and Triangulation

Alan Facial Keypoints

Alan Triangulation

Jacob Facial Keypoints

Jacob Triangulation

2. Computing the "Mid-Way" Face

Approach

First, I wrote an affine transformation function, affine_transform using the affine transformation formula from the class lecture slides, [x' y' 1] = [x y 1] * A^T (where A is a matrix). Next, I inverse warped the Delaunay triangulations of the midway image and alan.jpg the previous part. This was done by first using a for-loop that iterates through the correlating triangles in the triangulations and finding its affine transform. Next, I used scipy.draw.polygon to find all points within the midway triangle, and applied the inverse warping formula A^-1 * [x' y' 1] = [x y 1] to each point. I rounded each point to the nearest integer, used those points from the source image to set the points in the midway triangle to get the final warped image. I used this approach to warp alan.jpg and jacob.jpg with the midway triangulation. Finally, in order to find the midway image, I cross-dissolved by averaging the images.

Morphing Alan and Jacob

Alan

Jalan Morph (Midway)

Jacob

3. The Morph Sequence

Approach

Modifying my code from the previous part, I wrote a morph function that uses warp_frac to control the intermediate shape configuration and dissolve_frac to cross-dissolve. I morphed the faces over 45 times while incrementing the warp_frac and dissolve_frac parameters from 0 to 1. Finally, I took the resulting images and merged them into a GIF using an online GIF creator.

Morph Sequence GIF

4. The "Mean Face" of a Population

Approach

I used the non-smiling/mug faces from the FEI Face Database for this part. I parsed the .pts files with a helper function using np.loadtxt to get the corresponding points of every image. I then averaged all of the corresponding points and found its triangulation to get the average points and triangulation of the population. I used my morph function to morph every face with the average face shape of the population (some examples showed below). I then cross dissolved all of these morphs to create the average face of the population. Finally, I morphed my face into the average face shape of the population, and vice versa.

Morphing People to the Population Average

Me and the Population Average

Population Average Face

Warping Alan to Average Face

Warping Average Face to Alan

5. Caricatures: Extrapolating from the Mean

Approach

I used the formula alpha * mean_image + (1 - alpha) * self_image to extrapolate from the population mean to create a caricature of myself. I used alpha = 2 for this part. Note that this formula is analogous to the warp weighing formula involving warp_frac in my morph function. Thus, I called morph with warp_frac=2 and dissolve_frac=0 to generate my caricature.

My Caricature

6. Bells and Whistles: Ethnicity and Gender Morphing

Approach

In this part, I morphed my face and my roommate's face with the average face of a white female. I found new corresponding points using the same tool as part 1. I morphed the each of my and my roommate's faces with the average white female's face using the new corresponding points and Delaunay triangulation, then the appearances by averaging the colors. Finally, just like in part 1, I cross-dissolved the warps to morph each face with the average white female's face.

Alan and Jacob as White Girls

Average White Female
Alan Morphed to Shape of Average White Female	Alan Morphed to Appearance of Average White Female	Alan Morphed to Average White Female
Jacob Morphed to Shape of Average White Female	Jacob Morphed to Appearance of Average White Female	Jacob Morphed to Average White Female