Convert HTML5 Canvas Sequence to a Video File

Question

I'd like to convert an animation in HTML5 canvas to a video file that could be uploaded to YouTube. Is there any sort of screen capture API or something that could allow me to do this programatically?

Answer 1

Back to 2020

Solved it by using MediaRecorder API. It builds exactly to do that, among other things.

Here is a solution that recorded X ms of canvas video you can extend it with Buttons UI to start, pause, resume, stop, generate URL.

function record(canvas, time) {
    var recordedChunks = [];
    return new Promise(function (res, rej) {
        var stream = canvas.captureStream(25 /*fps*/);
        mediaRecorder = new MediaRecorder(stream, {
            mimeType: "video/webm; codecs=vp9"
        });

        //ondataavailable will fire in interval of `time || 4000 ms`
        mediaRecorder.start(time || 4000);

        mediaRecorder.ondataavailable = function (e) {
            recordedChunks.push(event.data);
            if (mediaRecorder.state === 'recording') {
                // after stop data avilable event run one more time
                mediaRecorder.stop();
            }

        }

        mediaRecorder.onstop = function (event) {
            var blob = new Blob(recordedChunks, {
                type: "video/webm"
            });
            var url = URL.createObjectURL(blob);
            res(url);
        }
    })
}

Answer 2

Firefox has an experimental feature (disabled by default) that is called HTMLCanvasElement.captureStream()

Essentially it captures the canvas element as a video stream which can then be sent to another computer using RTCPeerConnection() or perhaps you can use the YouTube Live Streaming API to stream directly.

See: https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream

Also: https://developers.google.com/youtube/v3/live/getting-started

Answer 3

There exist the whammy library which claims to produce webm videos from stills using JavaScript:
http://antimatter15.com/wp/2012/08/whammy-a-real-time-javascript-webm-encoder/

Note that there are limitations (as to be expected). This encoder bases itself on the webp image format which is currently only supported in Chrome (perhaps the new Opera too but I haven't checked). This means you can't encode in other browsers unless you find a way to encode the image you want to use as a webp image first (see this link for possible solution for that).

Beyond that there is no way to create a video file from images using JavaScript and canvas using native browser APIs.

Answer 4

FileSaver.js + ffmpeg on the command line

With FilSaver.js we can download each canvas frame as PNG: Save to Local File from Blob

Then we just convert the PNGs to any video format with ffmpeg from the command line: How to create a video from images with FFmpeg?

Chromium 75 asks if you want to allow it to save multiple images. Then once you say yes, it downloads the images automatically one by one under your download folder, named as 0.png, 1.png, etc.

It also worked in Firefox 68, but less well, because the browser opens a bunch of "Do you want to save this file" windows. They do have a "do the same for similar downloads" popup, but you have to be quick to select it and hit enter, or else a new popup comes along!

To stop it, you have to close the tab, or add a stop button and some JavaScript logic.

var canvas = document.getElementById("my-canvas");
var ctx = canvas.getContext("2d");
var pixel_size = 1;
var t = 0;

/* We need this to fix t because toBlob calls are asynchronous. */
function createBlobFunc(t) {
  return function(blob) {
    saveAs(blob, t.toString() + '.png');
  };
}

function draw() {
    console.log("draw");
    for (x = 0; x < canvas.width; x += pixel_size) {
        for (y = 0; y < canvas.height; y += pixel_size) {
            var b = ((1.0 + Math.sin(t * Math.PI / 16)) / 2.0);
            ctx.fillStyle =
                "rgba(" +
                (x / canvas.width) * 255 + "," +
                (y / canvas.height) * 255 + "," +
                b * 255 +
                ",255)"
            ;
            ctx.fillRect(x, y, pixel_size, pixel_size);
        }
    }
    canvas.toBlob(createBlobFunc(t));
    t++;
    window.requestAnimationFrame(draw);
}
window.requestAnimationFrame(draw);

<canvas id="my-canvas" width="512" height="512" style="border:1px solid black;"></canvas>
<script src="https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/1.3.8/FileSaver.min.js"></script>

GitHub upstream.

Here's an image to GIF output using this instead: https://askubuntu.com/questions/648244/how-do-i-create-an-animated-gif-from-still-images-preferably-with-the-command-l

Frames get skipped if the FPS is too high

This can be observed by reducing the size of the canvas in the above demo to speed things up. At 32x32, my Chromium 77 download in chunks of about 10 files and skips about 50 files in between...

Unfortunately, there is no way to wait for the downloads to finish... close window after file save in FileSaver.js

So the only solution I can see if you have high framerate is framerate limiting... Controlling fps with requestAnimationFrame? Here is a live demo: https://cirosantilli.com/#html-canvas

Maybe one day someone will answer:

• H.264 video encoder in javascript
• Running ffmpeg in browser - options?

and then we will be able to download the video directly!

Here is an OpenGL version if you decide that the browser is not for you :-) How to use GLUT/OpenGL to render to a file?

Tested in Ubuntu 19.04.

Answer 5

This should help, it allows you to drop some images that get converted into HTML5 CANVAS and then converted into webm video: http://techslides.com/demos/image-video/create.html