HTML5 Canvas image segmentation

Question

I'm trying to remove the white/gray pixels of background (a wall) from an image so that the foreground (a hand in my case) should be left.

I tried manipulating canvas pixels but the foreground extraction is not very nice because of shadow or other inconsistencies in background. I even tried with green colored background but it doesn't enhance the performance.

for (i = 0; i < imgData.width * imgData.height * 4; i += 4) {
    var r = imgDataNormal.data[i + 0];
    var g = imgDataNormal.data[i + 1];
    var b = imgDataNormal.data[i + 2];
    var a = imgDataNormal.data[i + 3];
    // compare rgb levels for green and set alphachannel to 0;
    selectedR = *a value 0 - 255*
    selectedG = *a value 0 - 255*
    selectedB = *a value 0 - 255*
    if (r <= selectedR || g >= selectedG || b <= selectedB) {
        a = 0;
    }
}

Is there any sophisticated method or library to make background of an image transparent?

Answer 1

Converting the pixels from RGB to HSL and then running a threshold filter over the pixels to remove anything (by setting the pixel's alpha to be completely transparent) with a saturation less than 0.12 (measuring saturation on a scale of 0..1 - that value was found via trial-and-error).

JavaScript

function rgbToHsl( r, g, b ){
  r /= 255, g /= 255, b /= 255;
  var max = Math.max(r, g, b), min = Math.min(r, g, b);
  var h, s, l = (max + min) / 2;

  if(max == min){
      h = s = 0; // achromatic
  }else{
      var d = max - min;
      s = l > 0.5 ? d / (2 - max - min) : d / (max + min);
      switch(max){
      case r: h = (g - b) / d + (g < b ? 6 : 0); break;
      case g: h = (b - r) / d + 2; break;
      case b: h = (r - g) / d + 4; break;
      }
      h /= 6;
  }
  return [h, s, l];
}

function thresholdHsl(pixels,lower,upper){
  var d = pixels.data;
   var createTest = function( lower, upper ){
     return lower <= upper
            ? function(v){ return lower <= v && v <= upper; }
            : function(v){ return lower <= v || v <= upper; };
   }
   var h = createTest( lower[0], upper[0] );
   var s = createTest( lower[1], upper[1] );
   var l = createTest( lower[2], upper[2] );

  for (var i=0; i<d.length; i+=4) {
   var hsl = rgbToHsl( d[i], d[i+1], d[i+2] );
   if ( !h(hsl[0]) || !s(hsl[1]) || !l(hsl[2]) ){
     d[i+3] = 0;
   }
  }
}

var img = new Image();

img.onload = function() {
  var canvas = document.getElementById('myCanvas');
  var ctx    = canvas.getContext('2d');
  ctx.drawImage(img,0,0);
  var pixels = ctx.getImageData(0,0,canvas.width,canvas.height);
  thresholdHsl(pixels,[0,0.12,0],[1,1,1]);
  ctx.putImageData(pixels, 0, 0);
};
img.src = 'Hand.png';

HTML

<canvas id="myCanvas" width="638" height="475"></canvas>

Output

It could be further improved by removing noise from the picture to try to clean up the edges.

Answer 2

There are a few mistakes in your code :

• The last line a = 0 does nothing but changing a local variable. You have to modify it directly in the array
• After that, to see the result, you have to push back the pixel array to the context
• Since you need (in that example) to check for white/gray background (rgb(255, 255, 255)), you need to check if the value of the pixel is superior, not inferior. And using a and operator instead of or allows you here to filter a bigger part of the shadow (which is gray so every colour component has almost the same value) without removing the hand itself, which is more coloured.

Here is an example :

var imgData = context.getImageData(0, 0, canvas.width, canvas.height);

for (i = 0; i < imgData.width * imgData.height * 4; i += 4) {
    var r = imgData.data[i + 0];
    var g = imgData.data[i + 1];
    var b = imgData.data[i + 2];
    var a = imgData.data[i + 3];
    // compare rgb levels for green and set alphachannel to 0;
    selectedR = 105;
    selectedG = 105;
    selectedB = 105;
    if (r >= selectedR && g >= selectedG && b >= selectedB) {
        imgData.data[i + 3] = 0;
    }
}

context.putImageData(imgData, 0, 0);

That still isn't perfect because of the shadow, but it works.