Use AWS infrastructure for timelapse videos and presentations.
- Images are submitted to an S3 bucket.
- A Lambda function is triggered on new image upload 2.2. The image is processed (cropped, resized, exif cleaned up, etc) 2.3. The image is added to the timelapse video
- A rules-based workflow is triggered for further processing
- A lambda funtion can be called via HTTP to retrive a list of file names for a slideshow given a date/time range
npm install
Set your AWS configuration in .env
file.
To test run locally:
npm start
To deploy on AWS:
npm run deploy
Cameras upload the original full size images to an S3 bucket. Every camera has its own path within a bucket. The path inside the bucket (object prefix) is configurable.
-bucket
config.txt
-cam1
config.txt
-full
-exif
-resized
-[size name 1]
-idx
last.txt
last100.txt
today.txt
24hr.txt
7days.txt
30days.txt
-[size name 2]
-idx
-[size name 3]
-cam2
...
- bucket name: can be any name. A λ-function assigned to the bucket will extract the bucket name and the cam prefix from the object name it was given.
- config.txt: a config file, which can be nested. The deeper level config file overwrites the higher level one.
- file names: uploaded file names must follow ISO 8601 + the file type in this format (YYYYMMDDThhmmss.ssss.jpg, e.g. 20160815T170001.050.jpg). The date/time is recorded by the camera at the moment of the image capture. It may be different from the exif data.
- object properties: mimetype=image/jpg, http caching=forever
- full: the folder for original files. This is where the cameras upload them in the first place.
- idx: a folder with indexes maintained by the λ-function as simple list of URLs, one per line. Set http caching to expire immediately.
- exif: a folder with with exif data files extracted from the originals. The file names must match those of the original file, except the extension (.txt) and the mime type is text/text, http caching=forever
- resized: a folder with resized images with the same file names as the original. The images are grouped in subfolders as per the config file. Set http caching=forever
The camera app knows the bucket name, AWS credentials and its name. It will construct the object name in the format: [bucketname]/[cameraname]/full/[filename].jpg
and send it to S3. The λ-function will be triggered by the upload and will process the file.
Theoretically, there is no need to pre-create the camera folder if the AWS credentials allow for bucket-wide uploads.
The λ-function creates the folders it needs on the fly. There is no need to pre-create them, unless it is required for access control purposes.
The goal is to grant public access to all objects in resized
folder.
{
"Id": "Hubsy-Public-Access-Policy",
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetObject",
"Effect": "Allow",
"Resource": "arn:aws:s3:::BUCKET-NAME/*/resized/*",
"Principal": "*"
}
]
}
The λ-function has its own set of policies. It should be able to read the entire contents of the bucket, but write only into resized
folder. This policy has to be set in IAM and attached to the role used for the function.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::BUCKET-NAME/*"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::BUCKET-NAME/*/resized/*"
}
]
}
Images are resized to multiple smaller sizes as per this section of the config file:
{
"resize": [
{"folder": "resized/fhd", "width": 1920, "height": 1080, "compression": 50}
{"folder": "resized/hd", "width": 1080, "height": 720, "compression": 50}
{"folder": "resized/small", "width": 500, "height": 500, "compression": 50}
],
"crop": {"top": 100, "left": 100, "width": 300, "height": 300}
}
- folder: folder name for the resized image to be put in, relative to the camera root. It's just an object prefix in the context of S3.
- width, height: the maximum size in pixels for the image. It may not be proportional to the image which has to fit into this bounding box without cropping.
- compression - JPEG compression / quality level, 1 - 100, where 1 is the lowest and 100 is uncompressed.
- crop - describes the box that has to be cropped from the original image before resizing.
When a new file is placed into the bucket the λ-function should check if it's a valid jpeg file, parse the name, extract paths, read the config files, crop, resize and save the results.
Every new image is added to the end of the timelapse video. Frame duration, video size and other parameters are specified in the config file.
The most commonly requested image sets are stored in [cam-name]/idx
folder as absolute image URLs, one per line.
An HTTP request can be made to a lambda function to get a list of files for a date/time range. The HTTP REST API is configured via Amazon API Gateway.
Parameters: dateBefore
, dateAfter
, count
.