Giter VIP home page Giter VIP logo

lamba-thumbnailer's Introduction

Build

IMPORTANT

This branch is currently under development and is provided as is

Roadmap for TypeScript

  • Convert All Code to TypeScript (done)
  • Use FFmpeg Layers for Lambda instead (done)
  • Make the thumbnailer more moduler (in-progress)
  • Implement Tests (to be done)

Table Of Contents

Introduction

This package was created to automatically generate thumbnails from s3 video uploads.

Getting Started

Install this package using the following the command then require the package in your code.

npm install lambda-video-thumbnailer

Using in your own function

You can use the provided s3example.js to get you started. Or you can use the following code snippet for a lambda function.

// Require the library
const lvt = require("lambda-video-thumbnailer");

exports.handler = function (event, context) {
  // Returns a S3Thumbnailer Class
  const t = new lvt.S3(event.Records[0].s3);
  // Create a thumbnail will return promise<string>
  return t.createThumbnail({
    width: 100,

    height: 100,

    time: "00:00:05",

    type: "webp",
  });
};

Thumbnail Configuration

createThumbnail() takes a thumbnailConfig object with that has the following default options

{
  "prefix": "thumbnails/",
  "width": 180,
  "height": 180,
  "time": "00:00:05",
  "type": "jpg",
  "quality": 2,
  "thumbnailOption": "default"
}

Size

To configure the output size of thumbnails simply supply a width/height number. If you want to automatically calculate width or height based on the aspect ratio of the input, just use -1. For example:

{
  "width": 700,
  "height": -1
}

This will produce a 700x300 image when the input aspect ratio is 21:9

If your output needs to be a multiple of n then supply -n and it will output an output that is scaled by the ratio and a multiple of -n. For example

{
  "width": -3,
  "height": 250
}

This will produce a 441x250 image when the aspect raito is 21:9

Type

To configure the output type of thumbnails just set the type to "jpg", "png" or "webp"

{
  "type": "webp"
}

NOTE: ffmpeg with png tends to output very large filesizes even with -pred mixed

Quality

The output quality of the thumbnail is controlled by the quality property. It is controlled on a scale of 1-10 with 1 being the best quality and 10 being the worst quality.

{
  "quality": 1
}

Thumbnail Option

This option determines which filter to use for capturing thumbnails via ffmpeg. The options currently are default,none. The default option will set the filter to thumbnail this will look for the best frame within the select timeslot, this can be use quite a bit of memory on large files but will generally give the best screenshots. none will not use any filter and pick the first frame at the given timeslot.

{
  "thumbnailOption": "none"
}

Output Location

The output bucket is defined by the outputBucket thumbnail config option, by default this will output the files to the source bucket if left blank.

{
  "outputBucket": "mybucket"
}

By default the output file name will use the source file with the new type suffix. For example movie_trailer.mp4 becomes move_trailer.jpg when used with the jpg type output. However you can customize it with the outputKey option in your thumbnail config. It will still attach the type or replace it my_new_filename.hello becomes my_new_filename.jpg

{
  "outputKey": "my_new_filename"
}

Setup the Lambda Function

This package requires a ffmpeg layer to function. You can use the following arns:

Or alternatively use the serverlesspub build (only available in ap-us-east-1) arn:aws:lambda:us-east-1:145266761615:layer:ffmpeg:4 or build your own from their github repo serverlesspub/ffmpeg-aws-lambda-layer

Then just go onto AWS Lambda create a new function and give it a trigger of put objects with a suffix filter of _.avi, _.mov etc..

Although you don't need to do this it just reduces the number of invocations. Apply a policy of LambdaExecute to the IAM role (this gives read and write access to s3) then set the memory limit to the maximum* and set timeout for 30seconds (again it normally takes around 6 seconds but its good to be safe).

*NOTE: I have never reached the maximum with this function, its normally around 700mb even for 4gb mov files

Lambda Layers

Asia

  • arn:aws:lambda:ap-east-1:260572601982:layer:ffmpeg:1 - Hong Kong (AP East 1)
  • arn:aws:lambda:ap-south-1:260572601982:layer:ffmpeg:1 - Mumbai (AP South 1)
  • arn:aws:lambda:ap-northeast-2:260572601982:layer:ffmpeg:1 - Seoul (AP Northeast 2)
  • arn:aws:lambda:ap-southeast-1:260572601982:layer:ffmpeg:1 - Singapore (AP Southeast 1)
  • arn:aws:lambda:ap-southeast-2:260572601982:layer:ffmpeg:1 - Sydney (AP Southeast 2)
  • arn:aws:lambda:ap-northeast-1:260572601982:layer:ffmpeg:1 - Tokyo (AP Northeast 1)

Europe

  • arn:aws:lambda:eu-central-1:260572601982:layer:ffmpeg:1 - Frankfurt (EU Central 1)
  • arn:aws:lambda:eu-west-1:260572601982:layer:ffmpeg:1 - Ireland (EU West 1)
  • arn:aws:lambda:eu-west-2:260572601982:layer:ffmpeg:1 - London (EU West 2)
  • arn:aws:lambda:eu-west-3:260572601982:layer:ffmpeg:1 - Paris (EU West 3)
  • arn:aws:lambda:eu-north-1:260572601982:layer:ffmpeg:1 - Stockholm (EU North 1)

Canada

  • arn:aws:lambda:ca-central-1:260572601982:layer:ffmpeg:1 - Central (CA Central 1)

South America

  • arn:aws:lambda:sa-east-1:260572601982:layer:ffmpeg:1 - São Paulo (SA East 1)

United States

  • arn:aws:lambda:us-east-1:260572601982:layer:ffmpeg:1 - North Viginia (US East 1)
  • arn:aws:lambda:us-east-2:260572601982:layer:ffmpeg:1 - Ohio (US East 2)
  • arn:aws:lambda:us-west-1:260572601982:layer:ffmpeg:1 - N. California (US West 1)
  • arn:aws:lambda:us-west-2:260572601982:layer:ffmpeg:1 - Oregon (US West 2)

Original Tutorial

Originally this was a tutorial I wrote back in July 2017, I updated it a bit since then. The original also contained mediainfo. Available Here

License

MIT

lamba-thumbnailer's People

Contributors

deek87 avatar megavasiliy007 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

lamba-thumbnailer's Issues

Error spawn EACCES

I know this has been reported in an older issue but I'm not able to resolve it. I've tried Node 6.10 and Node 8.10 and also experimented with /tmp/ffmpeg and ffmeg as the path - to no avail.

Error: spawn EACCES
at exports._errnoException (util.js:1018:11)
at ChildProcess.spawn (internal/child_process.js:319:11)
at Object.exports.spawn (child_process.js:378:9)

Directory structure of ZIP:

ffmpeg
index.js
node_modules\
package.json

Any ideas?

Error: spawn EACCES at _errnoException (util.js:1022:11)

I got the below error from Cloudwatch logs when i triggered the lambda function form S3 PUT event.
Error: spawn EACCES
at _errnoException (util.js:1022:11)
at ChildProcess.spawn (internal/child_process.js:323:11)
at Object.exports.spawn (child_process.js:502:9)
at createThumbnail (/var/task/index.js:66:36)
at nextTask (/var/task/node_modules/async/dist/async.js:5324:14)
at Object.waterfall (/var/task/node_modules/async/dist/async.js:5334:5)
at exports.handler (/var/task/index.js:60:9)

Any idea ? how to resolve it ?

cp: cannot stat ‘/var/task/ffmpeg’: No such file or directory

Whew.. since the ARN to the layer provided is not working, I have tried every which way to get the layer working myself. I am quite familiar with AWS Lambda and have added many layers in the past, but this one has evaded me.

I keep getting this error below when trying to run this library with the sample code provided. I have both this library and ffmpeg installed as layers.

Any guidance you can provide would be much appreciated. Thank you.

Here is the trace:

Response:
{
"errorType": "TypeError",
"errorMessage": "lvt.S3 is not a constructor",
"trace": [
"TypeError: lvt.S3 is not a constructor",
" at Runtime.exports.handler (/var/task/index.js:9:15)",
" at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
]
}

Request ID:
"3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a"

Function logs:
START RequestId: 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a Version: $LATEST
2020-10-05T21:12:47.378Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a INFO {"Records":[{"eventVersion":"2.1","eventSource":"aws:s3","awsRegion":"us-west-2","eventTime":"2020-10-05T19:37:04.616Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"AWS:AIDAYSRDSSHZ3OKC4T7VU"},"requestParameters":{"sourceIPAddress":"44.230.17.147"},"responseElements":{"x-amz-request-id":"F4334B0026CB45A7","x-amz-id-2":"srU/E35rWxvHs/iFUQvaetR4QwJVv/qT39HC2DcEXB/GQ8YEEF3QToZjveCPkDc2kmD5z2/HspFe3wH8husCU6Vjc4F1MWyt"},"s3":{"s3SchemaVersion":"1.0","configurationId":"ZjkyZDY2ODItNzU3Mi00NzM0LWIwMWMtYTJjNjgyNjVkMDAy","bucket":{"name":"contentjaguar-hosted.com","ownerIdentity":{"principalId":"AIOAF98JYAQFE"},"arn":"arn:aws:s3:::contentjaguar-hosted.com"},"object":{"key":"generated-videos/1-1/jagcomp-c1_a5_swim_418430c8-2782-410f-8404-07e461370eb3_0-d.mp4","size":6294286,"eTag":"db0a7bbf202ec7cccbe4d421ea3d9ac5","sequencer":"005F7B75E1BCB6F5FB"}}}]}
2020-10-05T21:12:47.385Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a ERROR Invoke Error {"errorType":"TypeError","errorMessage":"lvt.S3 is not a constructor","stack":["TypeError: lvt.S3 is not a constructor"," at Runtime.exports.handler (/var/task/index.js:9:15)"," at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"]}
2020-10-05T21:12:47.405Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a INFO ffmpeg permissions couldnt be set
2020-10-05T21:12:47.408Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a INFO Error: Command failed: cp /var/task/ffmpeg /tmp/ffmpeg && chmod 755 /tmp/ffmpeg
cp: cannot stat ‘/var/task/ffmpeg’: No such file or directory

at ChildProcess.exithandler (child_process.js:303:12)
at ChildProcess.emit (events.js:315:20)
at ChildProcess.EventEmitter.emit (domain.js:483:12)
at maybeClose (internal/child_process.js:1021:16)
at Socket.<anonymous> (internal/child_process.js:443:11)
at Socket.emit (events.js:315:20)
at Socket.EventEmitter.emit (domain.js:483:12)
at Pipe.<anonymous> (net.js:674:12) {

killed: false,
code: 1,
signal: null,
cmd: 'cp /var/task/ffmpeg /tmp/ffmpeg && chmod 755 /tmp/ffmpeg'
}
2020-10-05T21:12:47.408Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a INFO
2020-10-05T21:12:47.408Z 3d9418e8-a9c9-4f94-96ba-0c0898cc2f7a INFO cp: cannot stat ‘/var/task/ffmpeg’: No such file or directory

You are not authorized to perform: lambda:GetLayerVersion.

Hi there--

I am configuring a layer for my lambda function which exists in us-west-2.

When adding this ARN:
arn:aws:lambda:us-west-2:260572601982:layer:ffmpeg:1

And clicking Add, I get this error:
You are not authorized to perform: lambda:GetLayerVersion.

Please advise/enable global permissions on the layer. Thanks!

Unable to change the export time

I am using the Thumbnail property "time" as said in the documentation:

t.createThumbnail({
  ...
  time: "00:00:02",
  ...
})

but the export time isn't changing, the lib always uses the default value "00:00:10".
I think the problem is: the input var is called "time" but then in the lib is used "timestamp".

thumbnails are created with size 0

i created a function and uploaded with exactly the code in this repo. everything looks like it's working, and the thumbnail file is created, but it's size is 0 and it's obviously not good

  1. do you have any idea what can cause this behavior?

  2. is there any way to see the output of ffmpeg, maybe i'll see some hints there.

by the way, i tried running ffmpeg with exactly the same parameters on my local machine, and it works correctly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.