This project refer to Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade.

This project present a new approach to localize extensive facial landmarks with a coarse-to-fine convolutional network cascade. This paper design a four-level convolutional network cascade, which tackles the problem in a coarse-to-fine manner. In this system, each network level is trained to locally refine a subset of facial landmarks generated by previous network levels.

In addition, each level predicts explicit geometric constraints (the position and rotation angles of a specific facial component) to rectify the inputs of the current network level. The combination of coarse-to-fine cascade and geometric refinement enables our system to locate extensive facial landmarks (194 points) accurately in the 300-W facial landmark localization challenge.

• Caffe
• python 2.7
• opencv 2.4.13

1.calibrate_pts.py: This is a tools to calibrate Helen_dataset or your new image. The user instruction is printed on the top of the calibrate window.

Step 1: Bounding Box Estimation.

According to the image bellow, it is quite convenient to calculate Innor Points Bounding Box and Contour Bounding Box if eyes', nose' and mouth corners' position are known. So I trained a shallow network to calculate the general position of them. You can check them on DCNN-caffe and you can find the model.

Innor Points Bounding Box :

x = LE.x - (RE.x - LE.x) / 2
y = N.y - [N.y - Max(RE.y, LE.y)] × 2
w = (RE.x - LE.x)×2
h = [N.y - Min(RE.y, LE.y)] × 3.5

Counter Points Bounding Box :

x = LE.x - (RE.x - LE.x)
y = Max(RE.y, LE.y) - [Max(LM.y,RM.y) - N.y]
w = (RE.x - LE.x) × 3
h = [Max(LM.y,RM.y) - Min(RE.y, LE.y)] * 2