18

I have created a webpage that receives base64 encoded bitmaps over a Websocket and then draws them to a canvas. It works perfectly. Except, the browser's (whether Firefox, Chrome, or Safari) memory usage increases with each image and never goes down. So, there must be a memory leak in my code or some other bug. If I comment out the call to context.drawImage, the memory leak does not occur (but then of course the image is never drawn). Below are snippets from my webpage. Any help is appreciated. Thanks! 

// global variables
var canvas;
var context;

...

ws.onmessage = function(evt)
{
    var received_msg = evt.data;
    var display_image = new Image();
    display_image.onload = function ()
    {
        context.drawImage(this, 0, 0);
    }
    display_image.src = 'data:image/bmp;base64,'+received_msg;
}

...

canvas=document.getElementById('ImageCanvas');
context=canvas.getContext('2d');

...

<canvas id="ImageCanvas" width="430" height="330"></canvas>

UPDATE 12/19/2011

I can work around this problem by dynamically creating/destroying the canvas every 100 images or so with createElement/appendChild and removeChild. After that, I have no more memory problems with Firefox and Chrome.

However, Safari still has a memory usage problem, but I think it is a different problem, unrelated to Canvas. There seems to be an issue with repeatedly changing the "src" of the image in Safari, as if it will never free this memory.

display_image.src = 'data:image/bmp;base64,'+received_msg;

This is the same problem described on the following site: http://waldheinz.de/2010/06/webkit-leaks-data-uris/

UPDATE 12/21/2011

I was hoping to get around this Safari problem by converting my received base64 string to a blob (with a "dataURItoBlob" function that I found on this site) and back to a URL with window.URL.createObjectURL, setting my image src to this URL, and then later freeing the memory by calling window.URL.revokeObjectURL. I got this all working, and Chrome and Firefox display the images correctly. Unfortunately, Safari does not appear to have support for BlobBuilder, so it is not a solution I can use. This is strange, since many places including the O'Reilly "Programming HTML5 Applications" book state that BlobBuilder is supported in Safari/WebKit Nightly Builds. I downloaded the latest Windows nightly build from http://nightly.webkit.org/ and ran WebKit.exe but BlobBuilder and WebKitBlobBuilder are still undefined.

UPDATE 01/03/2012

Ok, I finally fixed this by decoding the base64-encoded data URI string with atob() and then creating a pixel data array and writing it to the canvas with putImageData (see http://beej.us/blog/2010/02/html5s-canvas-part-ii-pixel-manipulation/). Doing it this way (as opposed to constantly modifying an image's "src" and calling drawImage in the onload function), I no longer see a memory leak in Safari or any browser.

bglaudel
  • 193
  • 1
  • 8
  • what happens if you add a clearRect call before drawing the image, or if you use the reset trick of setting the width to itself? (from http://stackoverflow.com/questions/2142535/how-to-clear-the-canvas-for-redrawing) – Mikeb Dec 16 '11 at 19:24
  • I have tried context.clearRect(0, 0, canvas.width, canvas.height); before drawImage but the memory leak still occurs. – bglaudel Dec 16 '11 at 19:33
  • Are you sure it isn't just storing the image just in case you go back to the page so it can just load it from memory? Might be a runtime optimization. – Andrew Rasmussen Dec 16 '11 at 21:04
  • 2
    2018 : Safari v11 --> data uri still leaks like CRAZY ("I would note put this in prod" kind of crazy). Firefox Quantum --> leaks a little bit. Chrome --> totally fine. – Guillaume Le Mière Apr 17 '18 at 05:10

4 Answers4

3

Without actual working code we can only speculate as to why.

If you're sending the same image over and over you're making a new image every time. This is bad. You'd want to do something like this:

var images = {}; // a map of all the images

ws.onmessage = function(evt)
{
    var received_msg = evt.data;
    var display_image;
    var src = 'data:image/bmp;base64,'+received_msg;
    // We've got two distinct scenarios here for images coming over the line:
    if (images[src] !== undefined) {
      // Image has come over before and therefore already been created,
      // so don't make a new one!
      display_image = images[src];
      display_image.onload = function () {
          context.drawImage(this, 0, 0);
      }
    } else {
      // Never before seen image, make a new Image()
      display_image = new Image();
      display_image.onload = function () {
          context.drawImage(this, 0, 0);
      }
      display_image.src = src;
      images[src] = display_image; // save it for reuse
    }
}

There are more efficient ways to write that (I'm duplicating onload code for instance, and I am not checking to see if an image is already complete). I'll leave those parts up to you though, you get the idea.

Simon Sarris
  • 58,131
  • 13
  • 128
  • 161
  • If every image is unique, and memory usage increases as you make each new HTMLImageElement (`new Image()`), that's just normal! – Simon Sarris Dec 16 '11 at 19:37
  • Then why does memory usage stay level if I comment out the call to drawImage? – bglaudel Dec 16 '11 at 19:45
  • Right now, in your code, every single time `drawImage` is called, `new Image()` is also called. Unless you are only calling `drawImage` X times where X is the number of unique images, you have a huge problem because you are making more `new Image()`s than you want. How many images are there, and how many times is `new Image()` getting called? – Simon Sarris Dec 16 '11 at 19:51
  • There are an infinite number of unique images, as many as my Websocket server wants to send. My point is, when I forgo calling drawImage (by commenting it out), it appears that the memory allocated by new Image() is freed after some period, probably by garbage collection. But when I leave in the call to drawImage that memory is never freed. – bglaudel Dec 16 '11 at 19:59
  • Ah, I see. Just for kicks, try commenting out the `drawImage` call and replacing it with a `context.fillRect(0,0,50,50)` call. Does it still leak? – Simon Sarris Dec 16 '11 at 20:08
  • No, that does not leak. There must be something about drawImage that permanently attaches my image to the canvas so it can never be freed. – bglaudel Dec 16 '11 at 20:26
  • I guess so. That's a shame, I had hoped it was a silly closure bug. It still seems unlikely somehow. I suppose right after calling `drawImage` you could say `this.onload=null;this=null;` and see if that helps. But thats the only other suggestion I've got. Sorry for misunderstanding your problem at first. – Simon Sarris Dec 16 '11 at 20:29
  • This memory leak was actually reported as a WebKit bug in 2009, and still open as of today. https://bugs.webkit.org/show_bug.cgi?id=31253 – Pierre F Jul 16 '18 at 15:22
  • If you're using https://github.com/fengyuanchen/cropperjs, you probably will encounter this problem. You might find the answer https://github.com/fengyuanchen/cropper/issues/189#issuecomment-74614136 – YuAn Shaolin Maculelê Lai Sep 18 '18 at 03:26
1

I don't believe this is a bug. The problem seems to be that the images are stacked on top of each other. So to clear up the memory, you need to use clearRect() to clear your canvas before drawing the new image in it.

ctx.clearRect(0, 0, canvas.width, canvas.height);

How to clear your canvas matters

Stitch10925
  • 21
  • 1
  • I noticed that for some reason the canvas height seems to be wrong in my case. Using the image height as the "canvas.height" in the code sample seems to do the trick. – Stitch10925 Apr 24 '12 at 15:47
0

you're probably drawing the image a lot more times than you are expecting to. try adding a counter and output the number to an alert or to a div in the page to see how many times the image is being drawn.

Ozzy
  • 7,817
  • 7
  • 50
  • 92
0

That's very interesting. This is worth reporting as a bug to the various browser vendors (my feeling is that it shouldn't happen). You might responses along the lines of "Don't do that, instead do such and such" but at least then you'll know the right answer and have an interesting thing to write up for a blog post (more people will definitely run into this issue).

One thing to try is unsetting the image src (and onload handler) right after the call to drawImage. It might not free up all the memory but it might get most of it back.

If that doesn't work, you could always create a pool of image objects and re-use them once they have drawn to the canvas. That's a hassle because you'll have to track the state of those objects and also set your pool to an appropriate size (or make it grow/shrink based on traffic).

Please report back your results. I'm very interested because I use a similar technique for one of the tightPNG encoding in noVNC (and I'm sure others will be interested too).

kanaka
  • 63,553
  • 21
  • 138
  • 135