Flash Eye Tracking

It’s been awhile since I did something in Flash, 懐かしい〜。

I’ve been experimenting with eye tracking system in Flash lately, there are a few demos out there but none of them provide the source code.

Through Google I came across to Fabian Timm and Erhardt Barth’s research, which basically explains the face detection process as follows;

  • Applying the face detector and extracting the eye region.
  • Isolating the eye area by detecting the color gradient between iris and sclera

It’s pretty simple and I thought the BitmapData class is robust enough to handle this type of algorithm

Whilst tinkering I’ve stumbled across Tomek’s blog, which describes a clever way of extracting color by juggling the threshold and blend method, that rang a few bells.

So after a few custom adjustments I’ve made my own eye tracking code, nothing too fancy since the requirement isn’t too complicated.

First add BlendMode

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public function addBlendMode(s:BitmapData):BitmapData {

    var r:BitmapData = new BitmapData(s.width,s.height);
    var r2:BitmapData = new BitmapData(s.width,s.height);
    var rect:Rectangle = new Rectangle(0,0,s.width,s.height);
    var pt:Point = new Point(0,0);
    r.draw(s);
    r2.draw(s);

    // to get more contrast
    r.draw(r2, new Matrix(), new ColorTransform(), BlendMode.MULTIPLY);
    return r;

}

Use the threshold method, convert the bitmap into 1 bit color.

1
2
3
4
5
6
7
8
9
10
11
12
// the threshold color is important
public function addThresholdColor(b:BitmapData, th:* = 0xff111111):BitmapData {
           
    var bmd2:BitmapData = new BitmapData(b.width, b.height);
    var pt:Point = new Point(0,0);
    var rect:Rectangle = new Rectangle(0, 0, b.width, b.height);
    var color:uint = 0xFF000000;

    bmd2.threshold(b, rect, pt, ">=", th, color, 0xFFFFFFFF, false);

    return bmd2;
}

Lastly mark the eye, I’ve done it only for one eye since the movement will be symmetrical anyway.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
public function getBound(b:BitmapData):Rectangle {

    var maxBlobs:int = 40;
    var i:int = 0;
   
    var minX:int = 640; // video width
    var maxX:int = 0;

    var minY:int = 480; // video height
    var maxY:int = 0;

    var hx:int;
    var hy:int;

    while(i < maxBlobs) {

        var bound:Rectangle = b.getColorBoundsRect(0xffffffff, 0xffffffff);  

        if(bound.isEmpty()) break;

        var bx:int = bound.x;
        var by:int = bound.y;
        var bwidth:int = bx + bound.width;
        var bheight:int = by + bound.height;

        if(bwidth < minX) minX = bx;
        if(bwidth > maxX) maxX = bwidth;

        if(bheight < minY) minY = by;
        if(bheight > maxY) maxY = bheight;

        for(var y:uint = by; y < bheight; y++) {
           
            if(b.getPixel32(bx,y) == 0xffffffff) {

                // fill color
                b.floodFill(bx,y, 0xffff0000);
            }
        }

        i++;
    }
   
    return new Rectangle(minX, minY, maxX - minX, maxY - minY);

}

At the moment the algorithm is not perfect, I need to add automatic light detector and stabilizer and due to the webcam resolution and environment lighting the gradient tracking seems impractical but the same principle applies, more or less, also I haven’t tested it against a person who has dark colored skin or light colored eyes, so I’ll post it later.

Resources

Beyond Reality Face Detection