- 4x 60s 20hz video
- Annotated face tracking video
- Head pose feature vector
- numpy
- sklearn
- skimage
- cv2
-
Core
- Frame preprocessing
- Facial detection
- Facial landmark identification
- Geometric orientation
- Rendering
- Main
-
Support
- Configuration
- SVM preprocessing
- SVM training
- File utilities
- Numpy utilities
- Image utilities
-
Data
- Trained SVM
- Input
- Video files
- Intermediate
- Preprocessing (optional)
- imagesToFaces
- videosToFrames
- Output
- Annotated video
- Head pose estimation feature vectors
- Haar cascade classifier
-
Spike
- Random excursions
-
Tests
- TBD
video -> video preprocess + dataset preprocess -> train svm -> face detection -> retrain svm -> face detection -> find landmarks -> calculate geometry -> render
read frame -> frame preprocess -> face detection -> find landmarks -> calculate geometry -> render
- HEVC video dataset -> frames
- Yale faces dataset -> cropped
- Cropped yale faces -> positive samples
- 256 object categories dataset -> negative samples
- Annotate samples
- Train linear SVM
- Save SVM model
- (After sliding window): Retain SVM with hard-negative mining
- Save new SVM model
- Sliding window over image pyramid
- Non-maximum suppression
- Facial landmark alignment
- 2D-3D point mapping
- Compute head orientation
- Pupil detection
- CDF
- Feature Extraction and Normalization
- Gaze Classification and Decision Pruning
Method:
Using comma ai dataset: Take in hevc video Extract frames from 60s of 20hz video (~1200)
Using yale faces dataset: Convert to jpg and grayscale Crop the images using builtin haar cascades uniform resize write to disk generate (~165) positive samples for SVM using skimage hog descriptor
Using 256_object_categories dataset: generate (~30600) negative samples for SVM using skimage hog in batches of 1000 saving to disk
arrange data correctly + add labels train svm with the positive and negative samples save trained svm
sliding window image pyramid non-maximum suppression hard negative mining retrain
find face find eyes geometric transformation for facial plane generate vector