Milestones

We shall first develop the robust face detectors. To do this, we have to first identify the basic face features, as well as composite features, that are less sensitive to changes in face orientations. Our aim is to develop a face detector that can work with most of the footage captured from typical surveillance cameras. Basic idea is to use skin-tone color and haar-feature as cues for face detection. The skin-tone color modeling is a fast and efficient tool to detect face candidate from video sequence in real time. Then facial features (e.g. eyes and mouth) will be finely extracted to verify the detected face and to estimate the head pose. Haar-feature detector and active appearance model are proposed to be implemented and improved for this. Facial features in frontal view and side view will be collected and used to train the classifier of Haar-feature detector by AdaBoost. After we have developed the face detectors, and estimated pose of detected faces, we can then proceed to the stage of 3D face reconstruction. During this stage, we need to establish a 3D face model firstly, so that we can map detected 2D face on it, and get texture of 3D face model, which is called texture mapping. So far in our preliminary work, this 3D face model has 6292 vertices and 6152 facet. By mapping with 2D face, facet texture of 3D face model could be synthesized by appropriate algorithms on affine transform and ITSP 6.0 interporlation. For those facet whose texture can not get from texture mapping, default texture was adopted and adjusted to make reconstructed 3D face become natural and harmonious. With this reconstructed 3D face, it is possible to transform the face in side view or angle view back to the frontal view.

Thereafter, we shall develop face image super resolution algorithm so that it can takes multiple face images of the same person to generate a better and higher resolution image, on which the face recognition technology can work more reliably and accurately. Every detected faces in different video frame can be utilized to generate a texture map of 3D faces by above processing, and classic super resolution algorithms can by applied on. Therefore, super resolution faces from any view point can be achieved.

Finally, we shall implement a people identification system prototype by integrating the technologies we have developed in this project with COTS face recognition system. The prototype system, however, takes typical video surveillance cameras as inputs to illustrate the effectiveness of the technologies developed in this project.

Phase 1	Robust Face Detection Method At this milestone, we should have already developed a face detection algorithm that can detect and extract human face snapshots from a single image.
Phase 2	3D Face Reconstruction Method During this stage, we can extract multiple snapshots of the same people from a flow of image sequences. So, by mapping a face model to each of these snapshots, we can estimate the face orientation, and derive the texture maps of the human face for 3D face reconstruction. At the end of this stage, we will end up with a 3D Face Reconstruction algorithm that can reconstruct a frontal face image of a person appeared in a video sequence.
Phase 3	Face SuperResolution Since the 3D Face Reconstruction algorithm developed in the previous milestone will produce low resolution frontal face image. At this milestone, we will develop a superresolution algorithm that can generate higher resolution 3D reconstructed frontal face image from a lower one.
Phase 4	Prototype People Identification System With all the algorithm and technologies developed in the previous milestones, the final goal is to put them all together to come up with a prototype people identification system. The prototype is expected to extract and identify moving peoples appeared in the most common types of surveillance videos.