matthewearl / photo-a-day-aligner Goto Github PK

View Code? Open in Web Editor NEW

84.0 9.0 22.0 34 KB

Tools to help compose images from daily self-portrait time lapses

License: MIT License

Python 94.33% Shell 5.67%

photo-a-day-aligner's Introduction

Photo-a-day Aligner

A tools to help with daily self-portrait projects:

pada.py which has a couple of sub-commands:
align: Take a set of photo-a-day images, and align them based on the detected face, and perform RGB scaling so that all the faces have the same average RGB value. Also outputs an image mask.png which is used by the next script. Duplicate images, images with no face, and images with more than one face are dropped at this stage.
framedrop: Produce a file list, based on the output files of the above script. The output will have approximately (100 / N) % of the input images (N is 10 by default). Output frames are selected to avoid temporal discontinuities in the face area.
make_vid.sh: A shell script which calls mencoder to encode the file list produced by the above into a .h264 MP4 file.

See below for usage details.

Recommended workflow

Create a directory for your project.
Copy examples/pada.conf into it. Change predictor_path to point to your dlib landmarks, [downloadable from here](http://sourceforge.net/projects/ dclib/files/dlib/v18.10/shape_predictor_68_face_landmarks.dat.bz2).
Create a sub-directory input, and place your input frames into it. When lexicographically sorted the file names should be in the correct order.
Run pada.py align to align and colour correct your input frames. At this point you can inspect the output in ./aligned. If the results are not satisfactory change settings and repeat this step.
Run pada.py framedrop to select a sequence of good frames and output them to filtered.txt.
Run make_vid.sh to convert the above file list into a video, output.mp4.

Usage

General pada.py options:

$ pada.py --help
usage: pada.py [-h] [--debug] [--config CONFIG] [--aligned-path ALIGNED_PATH]
               [--aligned-extension ALIGNED_EXTENSION]
               [--predictor-path PREDICTOR_PATH]
               [--filtered-files FILTERED_FILES]
               {print_config_paths,align,framedrop} ...

positional arguments:
  {print_config_paths,align,framedrop}
                        Sub-command help
    print_config_paths  print config paths and exit
    align               align a set of images
    framedrop           Drop frames from a set of images

optional arguments:
  -h, --help            show this help message and exit
  --debug               Print debug information
  --config CONFIG       Config file path
  --aligned-path ALIGNED_PATH
                        Path where aligned images will be stored
  --aligned-extension ALIGNED_EXTENSION
                        Extension (and filetype) to use for aligned images.
  --predictor-path PREDICTOR_PATH
                        DLib face predictor dat file
  --filtered-files FILTERED_FILES
                        File to write filtered files to

pada.py align options:

$ pada.py align --help
usage: pada.py align [-h] [--input-glob INPUT_GLOB] [--img-thresh IMG_THRESH]

optional arguments:
  -h, --help            show this help message and exit
  --input-glob INPUT_GLOB
                        Input files glob
  --img-thresh IMG_THRESH
                        Max duplicate frame delta

pada.py framedrop options:

$ pada.py framedrop --help
usage: pada.py framedrop [-h] [--erode-amount ERODE_AMOUNT]
                         [--frame-skip FRAME_SKIP]

optional arguments:
  -h, --help            show this help message and exit
  --erode-amount ERODE_AMOUNT
                        Amount to erode face mask by
  --frame-skip FRAME_SKIP
                        Ratio of input frames to output frames

Options can alternatively be specified in a pada.conf in the working directory, in the site config path, or global config path. To see the full list of config paths run pada.py print_config_paths

Requirements

pada.py requires numpy, dlib, scipy, cv2, and appdirs.

make_vid.sh requires mencoder and suitable codecs to be installed.

photo-a-day-aligner's People

Contributors

Stargazers

Watchers

Forkers

finkregh xuanhan863 fototo jccroft1 nesh108 hjguyhan melystar18 alfstorm victormn pselvana indeximal jcpsimmons dirceu-jr gillesc jawsthegame kayasky xfyecn matt-winchell jk-keller

photo-a-day-aligner's Issues

Walkthrough

I'm sort of new to this stuff, and it would be helpful if somebody walked me through this program and how to use it. Thanks!

General object alignment

First, great tool you built here. I am using it to combine the webcam images my laptop makes on every wakeup into a video that is then uploaded to my website - https://rybakov.com/about/

I asked myself if it would be possible to use it to train it to align any object, not just faces. I used imglab to create a predictor.dat trained on a black square with four corners as parts.
When I use my predictor.dat with pada however, no objects can be found on very similar testing images.
Is there something that can be done to use it for general object alignment?

Multiple Faces

Feature Request

I have some pics where there are multiple people i.e faces in it. Currently, those pics are skipepd.

It would be great if there's a way to face recognize and align image to that face.

TypeError: Expected Ptr<cv::UMat> for argument 'points'

Got error TypeError: Expected Ptr<cv::UMat> for argument 'points'. Changed points = cv2.convexHull(points) in draw_convex_hull to points = cv2.convexHull(points.astype('int32')) to resolve the issue.

On python 2.7.16, opencv-python==4.2.0.32, numpy==1.16.5

Error with pada.py framedrop

First, let me say, what a fantastic tool you have built. I will use it to watch my son grow - what fantastic times we live in.

I wanted to test the video export functionality however I get the following error on

python pada.py framedrop

WARNING:root:Could not open config file C:\Users\Notebook\AppData\Local\matthewearl\pada\pada.conf WARNING:root:Could not open config file C:\ProgramData\matthewearl\pada\pada.conf INFO:root:Read config file pada.conf INFO:pada:Filtering 8 files Traceback (most recent call last): File "pada.py", line 160, in <module> for fname in filtered_files: File "C:\BilderToni\pada\framedrop.py", line 114, in filter_files weights = find_weights(input_files, mask, frame_skip) File "C:\BilderToni\pada\framedrop.py", line 62, in find_weights link_layers(prev_layer, layer) File "C:\BilderToni\pada\framedrop.py", line 46, in link_layers for n1, m1 in layer1: TypeError: 'NoneType' object is not iterable

Thank you!

TypeError: points data type = 9 is not supported

The script fails for me with the above error, similar to the issue reported in the faceswap project: matthewearl/faceswap#15

The solution for photo-a-day-aligner is the same as for faceswap, copying it here:

Try adding the following line points=points.astype(numpy.int32) to the draw_convex_hull function.
def draw_convex_hull(im, points, color):
        points=points.astype(numpy.int32)
        points = cv2.convexHull(points)
        cv2.fillConvexPoly(im, points, color=color)
This is to make the data type compatible with the cv2 function.

"ValueError: operands could not be broadcast together with shapes"

Hi!
I try to use your script to align a stack of photos. It works great, but after about 70 pictures, the script stop and I get this message. I tried skipping a few pictures, but it did not help. Unfortunately, I'm not sure how to proceed to debug this myself.

Traceback (most recent call last): File "../pada.py", line 148, in <module> img_thresh=cfg['img_thresh']) File "/home/ubuntu/face-align/pada/align.py", line 177, in align_images for idx, (n, im, lms) in enumerate(ims_and_landmarks): File "/home/ubuntu/face-align/pada/align.py", line 111, in get_ims_and_landmarks for n, im in images: File "/home/ubuntu/face-align/pada/align.py", line 48, in read_ims if prev_im is None or numpy.linalg.norm(prev_im - im) > img_thresh: ValueError: operands could not be broadcast together with shapes (3264,2448,3) (4032,3024,3)