How to minimize face-detection error

Question

here is the code now I can both detect face and mouth together, and able to roughly measure the distance of its bounding box <--

the problem is the mouth detection seems to detects everything they defines as mouth even it is not

and I want to use the "face" bounding box as a mouth detection region to minimize its error, I don't know if Forloop stacked would work? by put mouth loop inside face loop?? I'm fairly new to coding any help would be appreciated

import gab.opencv.*;
import java.awt.Rectangle;
import processing.video.*;

Capture video;

OpenCV f;
OpenCV m;

void setup() {
size(800, 600);
video = new Capture(this, 800/2, 600/2);
f = new OpenCV(this, 800/2, 600/2);
m = new OpenCV(this, 800/2, 600/2);

video.start();
}

void draw() {
scale(2);
f.loadImage(video);
m.loadImage(video);
f.loadCascade(OpenCV.CASCADE_FRONTALFACE);
m.loadCascade(OpenCV.CASCADE_MOUTH);

image(video, 0, 0 );

noFill();
stroke(0, 255, 0);
strokeWeight(3);
Rectangle[] mouth = m.detect();
Rectangle[] face = f.detect();
println(mouth.length);

strokeWeight(3);
for (int i = 0; i < face.length; i++) {
println(face[i].x + "," + face[i].y);
rect(face[i].x, face[i].y, face[i].width, face[i].height);
}
for (int i = 0; i < mouth.length; i++) {
println(mouth[i].x + "," + mouth[i].y);
rect(mouth[i].x, mouth[i].y, mouth[i].width, mouth[i].height);
}

for (int i = 0; i < mouth.length; i++) {
fill(255, 0, 0);
noStroke();
ellipse((mouth[i].x)+(mouth[i].width/2), mouth[i].y, 5, 5);
ellipse((mouth[i].x)+(mouth[i].width/2), (mouth[i].y)+  (mouth[i].height), 5, 5);
}
for (int i = 0; i < mouth.length; i++) {
int px = (mouth[i].x)+(mouth[i].width/2);
int py = (mouth[i].y)+(mouth[i].height);
int mOpen = int (dist(px, mouth[i].y, px, py));
println(mOpen);
}
}
void captureEvent(Capture d) {
d.read();
}

Answer 1

There are a couple issues:

1. You shouldn't be loading OpenCV cascades multiple times a second in draw(). You should do it once in setup() and just call detect() in draw()
2. OpenCV for Processing seems to override the cascade loaded in the second instance with a cascade loaded in the first instance

If accuracy isn't a huge issue, you can get away with a single cascade: the mouth one. Note that there are options/hints you can use for the detect function which may help the detection. For example you can tell the detector to detect largest object only, give it a hint of the smallest and largest bounding boxes the mouth would have with your setup and how much should the results filtered out.

Here's a code sample for the above:

import gab.opencv.*;
import java.awt.Rectangle;
import org.opencv.objdetect.Objdetect;
import processing.video.*;

Capture video;
OpenCV opencv;

//cascade detections parameters - explanations from Mastering OpenCV with Practical Computer Vision Projects
int flags = Objdetect.CASCADE_FIND_BIGGEST_OBJECT;
// Smallest object size.
int minFeatureSize = 20;
int maxFeatureSize = 80;
// How detailed should the search be. Must be larger than 1.0.
float searchScaleFactor = 1.1f;
// How much the detections should be filtered out. This should depend on how bad false detections are to your system.
// minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means good detections are given but some are missed.
int minNeighbors = 6;

void setup() {
  size(320, 240);
  noFill();
  stroke(0, 192, 0);
  strokeWeight(3);

  video = new Capture(this,width,height);
  video.start();

  opencv  = new OpenCV(this,320,240);
  opencv.loadCascade(OpenCV.CASCADE_MOUTH);
}

void draw() {
  //feed cam image to OpenCV, it turns it to grayscale
  opencv.loadImage(video);
  opencv.equalizeHistogram();
  image(opencv.getOutput(), 0, 0 );

  Rectangle[] mouths = opencv.detect(searchScaleFactor,minNeighbors,flags,minFeatureSize, maxFeatureSize);
  for (int i = 0; i < mouths.length; i++) {
    text(mouths[i].x + "," + mouths[i].y + "," + mouths[i].width + "," + mouths[i].height,mouths[i].x, mouths[i].y);
    rect(mouths[i].x, mouths[i].y, mouths[i].width, mouths[i].height);
  }
}
void captureEvent(Capture c) {
  c.read();
}

Note that facial hair can cause false positives. I have provided more in depth notes in an answer to your previous related question. I recommend focusing on the FaceOSC part as it will be more accurate.