Giter VIP home page Giter VIP logo

whammy's Introduction

Whammy: A Real Time Javascript WebM Encoder*

* with caveats

screenshot of Weppy, two clocks with the one on the right embedded inside a video while the left is made in canvas, above the clocks is a progress bar at 100% and below the clocks is a status which reads "Compiled video in 17ms, file size 272KB" adjacent to a link titled "Download WebM"

This is sort of a conceptual reversal (or not, this might just be making the description needlessly confusing) of one of my older projects, Weppy. First, what Weppy did was it added support for WebP in browsers which didn't support it by converting it into a single-frame video. This is instead predicated on the assumption that the browser already has support for WebP (at this point, it means it only works on Chrome since it's the only browser which actually supports WebP), not only decoding WebP but encoding it as well.

The cool thing about WebP which was exploited in Weppy is that it's actually based on the same codec as WebM, On2's VP8. That means the actual image data, when the container formats are ignored, are virtually interchangable. With a catch: it's intraframe only.

So it's a video encoder in that it generates .webm files which should play in just about any program or device which supports the WebM format. But interframe compression is actually a fairly important thing which could reduce the file size by an order of magnitude or more.

But, there isn't too much you can do on the client side in the ways of encoding stuff. And whatever you do, you basically can't do interframe compression (aside from some really rudimentary delta encoding). More or less, when your only alternative is to maintain an array of DataURL encoded frames or encoding it (rather slowly) as a GIF, a fast but inefficient WebM encoder stops looking too bad.

Demo

http://antimatter15.github.com/whammy/clock.html

http://www.sysord.fr/Sysord/ressource_whammy.jsf

Basic Usage

First, let's include the JS file. It's self contained and basically namespaced, which is pretty good I guess. And it's not too big, minified it's only about 4KB and gzipped, it's under 2KB. That's like really really tiny.

<script src="whammy.js"></script>

The API isn't terrible either (at least, that's what I'd like to hope)

var encoder = new Whammy.Video(15); 

That 15 over there is the frame rate. There's a way to set the individual duration of each frame manually, but you can look in the code for that.

encoder.add(context or canvas or dataURL);

Here, you can add a frame, this happens fairly quickly because basically all it's doing is running .toDataURL() on the canvas (which isn't exactly a speed-demon either, but it's acceptable enough most of the time) and plopping the result onto an array (no computation or anything). The actual encoding only happens when you call .compile()

encoder.compile(false, function(output){});

Here, output is set to a Blob. In order to get a nice URL which you can use to stick in a <video> element, you need to send it over to createObjectURL

var url = (window.webkitURL || window.URL).createObjectURL(output);

And you're done. Awesome.

Documentation

Whammy.fromImageArray(image[], fps) this is a simple function that takes a list of DataURL encoded frames and returns a WebM video. Note that the images have to all be encoded with WebP.

new Whammy.Video(optional fps, optional quality) this is the constructor for the main API. quality only applies if you're sending in contexts or canvas objects and doesn't matter if you're sending in encoded stuff

.add(canvas or context or dataURL, optional duration) if fps isn't specified in the constructor, you can stick a duration (in milliseconds) here.

whammy's People

Contributors

alt- avatar antimatter15 avatar gengkev avatar gsuess avatar jeblackiii avatar josephrocca avatar livelazily avatar positlabs avatar sysord avatar szager avatar tschneid avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

whammy's Issues

Please add a license file

It would be great if there was a license file so that people know how they should and shouldn't use your code.

Compile Error

My code is:
`function saveVideo(){
recorder = new Whammy.Video(FrameRate.value);
for(var i = 0; i <= images_array.length - 1; i++){
recorder.add(images_array[i]);
}
console.log(recorder);
var output = recorder.compile(function(blob){
return blob;
});

}`

When I was trying to compile, an error :

RecordRTC.js:4343 Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': function(){return 0<arguments.length?(this.elt.value=arguments[0],this):"range"===this.elt.type?parseFloat(this......e} could not be cloned.

Can anybody help me tell what to do ??

question about whammy

if whammy is based on canvas, it means whammy can't capture cross domain images??

30 seconds limit?

Hi,

First of all thanks for open sourcing such useful library :)

If the recording is more than 30 seconds, video becomes problematic. The code to send video data to a preview player is :

videoBlob = new Whammy.fromImageArray(frames, localframerate);
videoBlobURL = window.URL.createObjectURL(videoBlob);
document.getElementById('previewVideo');
videoElement.src = videoBlobURL;

I also tried updating cluster max size as (10 times more) :

var CLUSTER_MAX_DURATION = 300000;

but this time video stuck on 32. second.

Test platform is Chrome v.36 on Mac OSX 10.9

Regards

clock.html demo does not work in Firefox and Chromium

Demo on http://antimatter15.com/whammy/clock.html does not work correctly.
Firefox shows an error in console (no line specified):

uncaught exception: Input must be formatted properly as a base64 encoded DataURI of type image/webp

Chromium show other errors (CORS-related):

Refused to load the script 'http://antimatter15.com/whammy/whammy.js' because it violates the following Content Security Policy directive: "script-src 'none'". (clock.html:13)
Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'none'". Either the 'unsafe-inline' keyword, a hash ('sha256-BfZLB9J/Cyv0abkx6zfwQQIH+DNd2s/8O6jyPoKNp1I='), or a nonce ('nonce-...') is required to enable inline execution.

Webm with transparency

Hello! Webm supports transparency. Is it possible to make webm with transparency with Whammy?

whammy cluster limitation

Hi,
there is a 32s limitation when recording to webm. That is, after 32s, the frame count shows a totally different timestamp (16h32m...and counting).
When viewed offline, the video does continue to run ( I used VLC to check that), but when run in HTML, using

Thanks
Yossi

Large browser memory usage...

Man, thanks for share your solution, i'm Having a problem with memory, in large videos, more than 20min, u know any solution ? You had this problem too?

iOS, Phonegap: Cannot find function atob. Works on Android

Hello,

I am building a project using this tool which records a canvas. I import the Whammy script along with the CanvasRecorder script as specified here. The problem I am having is that whenever I call the function recorder.stop((blob) => {}), it uses the whammy library to convert each webP to base 64 and combine it to webM.

It is in this function where the problem occurs:

var webm = new ArrayToWebM(frames.map(function(frame) {
    var webp = parseWebP(parseRIFF(atob(frame.image.slice(23))));
    webp.duration = frame.duration;
    return webp;
}));

WhammyVideo.compile -> processInWebWorker -> whammyInWebWorker -> ArrayToWebM -> parseWebP -> parseRIFF -> atob

As stated above, on android web view through phonegap this works but on iOS it does not. I tried logging window and even tried polyfilling the function with a shim but nothing has worked so far. It seems as though the window variable is lost in the function:

function whammyInWebWorker(frames) {}

Maybe I am just thinking about this incorrectly but I think this library should be working across all devices especially iOS through Phonegap since it is a growing development community.

Best,
Aaron

Bug in Firefox

Hi.
When using Whammy in Firefox i get this error in the console.
[00:05:27.913] TypeError: frames[0] is undefined @ whammy.js:184

The script is working with no problem in Chrome

jpg support?

Hello there

Would it be too much work to feed whammy with jpg images instead of webp for webm generation?

Cheers
Michael

should be able to add a blob

I have a bunch of images (blob) that i want to convert into a movie
So i'm using html5 multiple file upload or drag and drop with multiple images

Think it would be insufficient to add everything to canvas or convert everything to base64 string (base64 takes up ~3x more in size and 2x times more memory since javascript strings are utf16 and not utf8)

encoder.add(blob);

Got a broken webm file

I add almost 300 frames to make a 21 sec video.
There was no error message, however it return a webm with correct width, height and time length but the scene is whole black, and it can't be played.

The following is where I called whammy:
var whammy = new Whammy.Video();
whammy.add(webpDataURL, time);
var output = whammy.compile();

Is there any suggestion or limit I missed it?

InvalidCharacterError: String contains an invalid character

I ran encoder.add(canvas); in a loop. a total of 705 frames, but when I tried to compile it all together I got a error: InvalidCharacterError: String contains an invalid character How could I get this error when I rain encoder directly on canvas?

opus packets

Hello!

I want ot generate webm file like you, but I have packet opus audio data.

Can you help to modify your code for me?

Realtime usage?

Can it be used for realtime capture?
If so it can be used to send a videofeed to Youtube Livestream

Error 'Frame 2 has a different width' since Chrome version 108

Images that used to work fine now fail to convert with this error:
20221203_223104

The size of each frame image is wrong.

I found the reason, since this project seems to be unmaintained, I didn't submit a PR, but as an issue, so that more people who encounter this error can see it.

var p2 = str.substr(577)
// change into
var p2 = str.substr(ChromeVer >= 108 ? 497 : 577)

Since Chrome 108 the 80-byte whitespace is removed, so the offset of the second part is 577 - 80 = 497.

I added code to detect ChromeVer before this.encodeFrames:

    let ChromeVer = 1
    const test = navigator.appVersion.match(/.*Chrome\/([0-9]+)/)
    if(test && test[1]){
      ChromeVer = parseInt(test[1])
    }

Whammy gives Black Screen Webm video on latest Google Chrome

Backgrounds

We used whammy for about 1 year, and it works well.
Recently, some customers report that the generated video is black screen.

Google Chrome Versions

I have checked an older version of Google Chrome 90.0.4430.212 (64-bit),it works well.
But whammy gives black screen when on latest Google Chrome 92.0.4515.107 (64-bit)。

Guessing Cause

The lastest Google Chrome updated webp image format, and whammy parse latest webp incorrectly, then it get a black screen webm video file.

Can anyone help on webp and webm format?
webm-ok.zip
webm-blackscreen.zip

callback is not a function line 556

Trying to use whammy - and in the compile step I get thie console error: "callback is not a function whammy.js line 556.

Any clue on how to troubleshoot?

last frame duration not used in video

this is easiest to demonstrate with a one frame video:

var video = new Whammy.Video;
video.duration = 15000;
video.quality = 1;
//assuming here that we have an image from a canvas to use:
video.add(context);
video.compile();

the compiled video will have a duration of 15 seconds. But if you actually play it, the video ends instantly. You can also test this with a video with multiple frames, just make sure the frame durations are long enough that you can clearly see each frame individually, the last frame duration is never used.

Failed - No file

Is there a limit on the number of frames that can be compiled or the size of the blob created?
I added a print statement in WebMExporter and got:
"Got to compile 4170 movie frames"
and then I print the size and the type of the blob and this is what I get:
"Size of movie blob: 250739803 video/webm"

But when I try to save the blob using createObjectURL (blob) I get a "Failed - No file".
This happens in Chrome (latest).

Also in the Chrome downloads window I see:
"The server could not find the file.
blob:file%3A///8993d68b-503d-4221-8e34-d1d09a3e7d82"

Wrong size data on segments

I'm using Whammy.js on a library to capture canvas https://github.com/spite/ccapture.js/ and most of the times the videos are created correctly.

However, some big videos (long videos, or videos of 1920x1080, or videos at 60FPS), don't work correctly and pieces of 30 seconds of the video are missing/can't be played. As an example, a capture of 4:07 at 1920x1080@60FPS will have all segments correctly placed, except the 3:30 to 4:00, during which there's no image. And 4:00 to 4:07 will play correctly.

I've been using HexFiend and EBML Viewer and found that some of the lengths specified in the segments are quite wrong:

screen shot 2016-03-02 at 17 46 06

If I manually fix the values for that segment other blocks inside the file will fail, so it's not immediate to see what's really going on.

Fixing this would be perfect, since Whammy.js is an amazing library for capturing on the browser, but it's a bummer if after a few hours of rendering high quality video, the resulting file doesn't play completely.

Thanks in advance!

Size constraints

Are there any size constraints for WebM video?
When I create videos, some of the sizes play in the browser (ie 170x300), and some don't (84x150, 282x500). Interestingly, when I save them to disk, VLC or android players can play them without problems.

You can check it here: http://depthy.clients.stamina.pl/ . Click the left "share" icon, then "create video". The only size that works is "Small", though it's different for images with different aspect ratio.

Sluggish Recorded Video For higher resolution videos.

I was trying to record a video of my screen using html5's getUserMedia api. The recorded video plays smoothely if the resolution of the video is set to 320 x 240. When we try to set higher resolutions, Then the video tend to get sluggish and slow.
How can I fix this issue? Any pointers on this will be helpful.

Tks

Has anyone got Whammy to work recently?

Hi,

I see there's basically two issues it seems like with Whammy as a result of the passage of time -- (1) a modification to the webp code and (2) something with CreateObjectUrl (I think this has been superceded).

Has anyone successfully modified to work recently? If so, would you be open to sharing your updated .js?

Thank you,

Philip

Cannot use 'in' operator to search for 'canvas' in data:image/webp;base64...

when calling the method add(), if the argument is a string of dataURL, will encounter the error,
the codes in whammy

if('canvas' in frame){ //CanvasRenderingContext2D
	frame = frame.canvas;	
}
if('toDataURL' in frame){
	frame = frame.toDataURL('image/webp', this.quality)
}else if(typeof frame != "string"){
	throw "frame must be a a HTMLCanvasElement, a CanvasRenderingContext2D or a DataURI formatted string"
}
if (!(/^data:image\/webp;base64,/ig).test(frame)) {
	throw "Input must be formatted properly as a base64 encoded DataURI of type image/webp";
}

it seems that it is a little unreasonable

a way for passing chunks

hi ,

i would like convert a long video using whammy but my initial idea was to pass frames
but whammy call me for sending chunks sequentially

Add duration argument to `encoder.add`

Add duration argument to encoder.add so that you can make a frame last a certain amount of time without having to add frame-rate * desired-duration amount of frames.

Or instead of a duration in seconds, it could be number of frames to cover.

encoder.add(context or canvas or dataURL, /*optional*/duration);
// frame should last 3 seconds
encoder.add(context, 3000);

Freezes on compilation step

I am taking a simple image and trying to make a video that is 10 seconds long. However, my browser locks up at video.compile(). Has anyone seen this behavior or have any idea how I can solve my problem?

window.doIt = function() {
  var canvas = document.getElementById('canvas');
  var context = canvas.getContext('2d');

  var video = new Whammy.Video();
  var videoEl = document.getElementById('awesome')
  var img = new Image();

  var finalizeVideo = function() {
    var output = video.compile();
  };

  img.onload = function() {
    canvas.width = img.width;
    canvas.height = img.height;
    context.drawImage(img, 0, 0, canvas.width, canvas.height);
    video.add(context, 10000);
    requestAnimationFrame(finalizeVideo);
  };

  img.src = "./hqdefault.jpg";
};
 <!DOCTYPE html>
 <html>
   <head>
     <title>Test Whammy</title>
     <script src="./app.js"></script>
     <script src="./whammy.js"></script>
   </head>
   <body>
     <video id='awesome' controls></video>
     <canvas id='canvas'></canvas>
     <script type='text/javascript'>
       doIt();
     </script>
   </body>
 </html>

Every video returns "Uncaught frame [x] has a different width.

I am attempting to create a webm video from an array of images (webp). I am using the below code:
var webmBlob = Whammy.fromImageArray(frames, 24);
url = window.URL.createObjectURL(webmBlob);
Every time this code runs, it throws an error that 1 frame has a different width and the video fails to render.
I have set the width and height of the video source and the canvas both inline and in the css and don't know what else I can do.

Sound track

Can I add a sound track when compiling it? I didn't find such function in the source code though..

Web Workers

Recording at high res / fps is pretty choppy. Is there some way we can improve this by offloading the .toDataURL() call to web workers? I assume that is where the bottleneck is, but I don't think you can reference dom elements in worker scripts.

Thoughts?

Adding text

Is there a way to add some text to the frames?

Video and Canvas merging

I am trying to join a

Where can someone find the documentation on how to do it with whammy.js?

Thank you

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.