Scribd Logo
Upload

Welcome to Scribd!

  • 98 views

    Xpolakm5 Computer Vision Eye Blink Detection

    Uploaded by

    api-288461888
    This document describes an approach to detecting eye blinking using optical flow analysis. It begins by detecting the face, then focuses on the eyes to calculate eye movement frame to frame. The initial approaches were unstable due to head movement. The final approach stabilizes the image by calculating global optical flow on the face, then analyzes each eye separately to detect blinking based on movement thresholds. It provides code samples and screenshots of the working application that detects individual eye blinks and displays results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Xpolakm5 Computer Vision Eye Blink Detection

    Uploaded by

    api-288461888
    98 views4 pages
    This document describes an approach to detecting eye blinking using optical flow analysis. It begins by detecting the face, then focuses on the eyes to calculate eye movement frame to frame. The initial approaches were unstable due to head movement. The final approach stabilizes the image by calculating global optical flow on the face, then analyzes each eye separately to detect blinking based on movement thresholds. It provides code samples and screenshots of the working application that detects individual eye blinks and displays results.

    Original Title

    xpolakm5 computer vision; eye blink detection

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document describes an approach to detecting eye blinking using optical flow analysis. It begins by detecting the face, then focuses on the eyes to calculate eye movement frame to frame. The initial approaches were unstable due to head movement. The final approach stabilizes the image by calculating global optical flow on the face, then analyzes each eye separately to detect blinking based on movement thresholds. It provides code samples and screenshots of the working application that detects individual eye blinks and displays results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    98 views4 pages

    Xpolakm5 Computer Vision Eye Blink Detection

    Uploaded by

    api-288461888
    This document describes an approach to detecting eye blinking using optical flow analysis. It begins by detecting the face, then focuses on the eyes to calculate eye movement frame to frame. The initial approaches were unstable due to head movement. The final approach stabilizes the image by calculating global optical flow on the face, then analyzes each eye separately to detect blinking based on movement thresholds. It provides code samples and screenshots of the working application that detects individual eye blinks and displays results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 4

    Eye blinking detection

    This application detect eye blinking by calculating optical flow in area of eyes inside stabilized picture
    of head. When the user have closed eyes for more than 5 seconds, the application starts to beep to
    wake him up. This can be used inside vehicles to prevent driver sleeping behind the steering wheel.
    For eye blinking detection is used generic webcam on computer.

    First approach
    The first approach to this problem was to detect face by using HAAR Cascade Classifier
    (haarcascade_frontalface_alt2 ). Then the program would take eyes position (static, defined by
    approximate position where the eyes should be or dynamic also by using HAAR Cascade).
    This approach was not working well (or at all) because Optical Flow (calculated by Farenback algorithm)
    was not stable. The Cascade Classifier was looking for face on every new frame and it was still changing
    its size and position the image was shaky.

    Image 1. Optical flow calculated by Farenback algorithm


    As seen on figure 1 this is how the blink was calculated (visible just pixels that moved more than defined
    length, calculated by Euclidean distance). This looks good, but problem was that the eyes was detected
    as moving all the time even when I was looking without movement on the camera.

    Image 2. Optical flow calculated by Farenback algorithm

    Second approach
    When I wanted to stabilize the image, I wanted to found darkest area of the eye image, and the other
    similar (dark) pixels to redraw as absolutely black (0h). Later I wanted to show just these black pixels
    and see just the one black circle. But this method is really unstable when there is not a good lightning
    and there are darker shadows than the actual eye.

    Final approach stabilization


    This method is based also on HAAR Cascade Classifier to found face. But when the face is found, after
    this first captured frame of face is activated stabilization. This first face is saved as rectangle and then
    on the second frame I will calculate global optical flow on this two images of faces. When the face is
    moved to the right by 5 pixels, first of all I will look at every pixel movement, add it to variable X and
    later, when there are for example 100 pixels and movement by 5 pixels, value X will be 500. Now I will
    divide this value by pixels and I get the movement 5 pixels to the right.
    void calcFlow(const Mat& flow, Mat& cflowmap, int step, int &globalMovementX, int &globalMovementY)
    {
    int localMovementX = 0;
    int localMovementY = 0;
    for (int y = 0; y < cflowmap.rows; y += step)
    {
    for (int x = 0; x < cflowmap.cols; x += step)
    {
    const Point2f& fxy = flow.at<Point2f>(y, x);
    localMovementX = localMovementX + fxy.x;
    localMovementY = localMovementY + fxy.y;
    }
    }
    globalMovementX = (localMovementX / (cflowmap.cols * cflowmap.rows))*2;
    globalMovementY = (localMovementY / (cflowmap.rows * cflowmap.cols))*2;
    }

    Code 1. Calculating flow of head movement


    Problem was that this calculated value was usually smaller than the real movement, so I multiplied this
    value by 2 so the movement would be more accurate. Also when this method is used, a lot of
    movement can move this rectangle of image where there is no face at all or is just moved that the eyes
    are not where they should be (and Im detecting nose and ear). Because of this reason there is
    implemented calibration, which is used every 100 frames.
    Now I had face that is not shaky when the real face in front of the camera was holding still. There I
    selected eyes by static values (estimated position of the eyes on the face). On each separate eye was
    used optical flow calculation, so application could detect blinking for each eye separately. This
    detection is not as with face, where is calculated real movement (for example real 5 pixels) but is used
    absolute movement of the eye (for example 500 pixels moved by 1 pixel to the right so the output is
    500).

    void detectBlink(Mat &matEyePrevious, Mat &matEyeCurrent, String eye, bool &eyeOpen, int
    &blinkNumber, clock_t &closeTime) {
    Mat leftFlow, leftCflow;
    calcOpticalFlowFarneback(matEyePrevious, matEyeCurrent, leftFlow, 0.5, 3, 15, 3, 5, 1.2, 0);
    cvtColor(matEyePrevious, leftCflow, CV_GRAY2BGR);
    int movementX, movementY;
    calcFlowEyes(leftFlow, leftCflow, 1, movementX, movementY);

    if (movementY == 0) {
    return;
    }
    if (movementY > 0 && eyeOpen) {

    //eye is now closed

    closeTime = clock();

    //time of eye closure

    eyeOpen = false;
    blinkNumber = blinkNumber + 1;

    //increment blink count

    }
    else if (movementY < 0 && !eyeOpen){
    eyeOpen = true;
    }

    //eye is now open

    Code 2. Detecting if eye is open or not

    Image 3. Top left and right eye on current frame, bottom previous frame

    Now, when the variables for eyes (if each eye is opened or closed) are set, we can show it on the Result
    window.

    Image 4. GUI of the application


    On final Result image is for each eye showing number of blinks, time of closed eye and color detection
    if the eye is opened or closed (green for opened, red for closed).
    Final results are publicly visible on YouTube:

    https://youtu.be/v0UNbB_vyrM

    You might also like