OpenCV Hierarchy is always null

Question

I have an image with overlapping contours and I have been trying to filter out the contours using hierarchy when I find them. What I am trying to do is filter out contours whose parents aren't equal to -1. However, when I try to get the information containing hierarchy the parent index is almost every time equal to null. Am I not looking at the right information for the getting the status of the current contours parent? Here is my code.

List<MatOfPoint> contours = new ArrayList<>();
    List<MatOfPoint> squareContours = new ArrayList<>();
    Mat hierarchy = new Mat();
    //find all contours
    Imgproc.findContours(dilated, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);

    //Remove contours that aren't close to a square shape.
    for(int i = 0; i < contours.size(); i++){
        if(hierarchy != null){
            double area = Imgproc.contourArea(contours.get(i)); 
            MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
            double perimeter = Imgproc.arcLength(contour2f, true);
            //Found squareness equation on wiki... 
            //https://en.wikipedia.org/wiki/Shape_factor_(image_analysis_and_microscopy)
            double squareness = 4 * Math.PI * area / Math.pow(perimeter, 2);

            if(squareness >= 0.7 && squareness <= 0.9 && area >= 2000){
                squareContours.add(contours.get(i));
            }
        }
    }
    //remove contour if it has a parent 
    List<MatOfPoint> finalContours = new ArrayList<>();
    for(int i = 0; i < squareContours.size();i++){
        if(hierarchy.get(i, 3)[3] == -1){ //this should be checking parent index I think.
            finalContours.add(squareContours.get(i));
        }
    }

This is the output of the program when I print the hierarchy matrix that contains the parent information Arrays.toString(hierarchy.get(i,3)))

[-1.0, -1.0, -1.0, 2.0]
null
null
null
null
null
null
null
null
null
null

Answer 1

When you are deleting item from list of hiearchy, it is needed to also check for relations of deleted item. I first transfer complete hierarchy Mat to List like this:

List<double[]> listHierarchy = new ArrayList<>();
                for (int i = 0; i< list.size(); i++){
                    listHierarchy.add(hierarchie.get(0, i));
                }

Then, when I need to delete item from this list I call this function:

 List<double[]> deleteHierarchyItem(int position, List<double[]> hierarchy){
    double[] itemHierarchy = hierarchy.get(position);
    double[] workItem;

    //doesnt have children?
    if (itemHierarchy[2]!=-1){
        int nextChild =(int) (itemHierarchy[2]);
        do {
            //nacte dite
            workItem = hierarchy.get(nextChild);
            //zmeni rodice v diteti na rodice puvodniho rodice
            workItem[3] = itemHierarchy[3];
            //nastavi nova data v listu
            hierarchy.set(nextChild, workItem);
            //zmeni ukazatel na nove dite
            nextChild = (int)(workItem[2]);

        } while (nextChild != -1);
    }

    //check for siblings
    boolean hasNextSibling = itemHierarchy[0] != -1;
    boolean hasPreviousSibling = itemHierarchy[1] != -1;

    double idPreviousSibling = itemHierarchy[1];
    double idNextSibling = itemHierarchy[0];

    //has both siblings
    if (hasPreviousSibling && hasNextSibling){
        //change previous sibling
        workItem = hierarchy.get((int)(idPreviousSibling));
        workItem[0] = idNextSibling;
        hierarchy.set((int)(idPreviousSibling), workItem);
        //change next sibling
        workItem = hierarchy.get((int)(idNextSibling));
        workItem[1] = idPreviousSibling;
        hierarchy.set((int)(idNextSibling), workItem);
    }

    //has only previous sibling
    if (hasPreviousSibling && !hasNextSibling){
        workItem = hierarchy.get((int)(idPreviousSibling));
        workItem[0] = -1;
        hierarchy.set((int)(idPreviousSibling), workItem);
    }

    //has only next sibling
    if (!hasPreviousSibling && hasNextSibling){
        workItem = hierarchy.get((int)(idNextSibling));
        workItem[1] = -1;
        hierarchy.set((int)(idNextSibling), workItem);

        //change of child parametres in parent
        if(itemHierarchy[3]>0)
        {
            workItem = hierarchy.get((int)(itemHierarchy[3]));
            workItem[2]=idNextSibling;
            hierarchy.set((int)(itemHierarchy[3]), workItem);
        }
    }

    //check for parent
    if (itemHierarchy[3]!=-1){
        workItem = hierarchy.get((int)(itemHierarchy[3]));
        if (workItem[2]==position){
            workItem[2] = -1;
            hierarchy.set((int)(itemHierarchy[3]), workItem);
        }
    }

    //iterate and decrement values
    for (int i = position; i< hierarchy.size();i++){
        workItem = hierarchy.get(i);
        if (workItem[0]>position){
            workItem[0] = workItem[0] - 1;
        }
        if (workItem[1]>position){
            workItem[1] = workItem[1] - 1;
        }
        if (workItem[2]>position){
            workItem[2] = workItem[2] - 1;
        }
        if (workItem[3]>position){
            workItem[3] = workItem[3] - 1;
        }
        hierarchy.set(i, workItem);
    }
    return hierarchy;
}

Answer 2

When you use a Mat to represent the hierarchy returned by findContours, you get an array which contains:

• a single row
• a column per detected contour
• 4 channels (id of next, previous, child, and parent contour)
• data type a 32bit signed integer

Now, your problem becomes immediately evident.

hierarchy.get(i, 3)[3]

The get method you use has the following signature:

public double[] get(int row, int col)

Notice that the first parameter is the row number. You pass the contour number as row, but there is only a single row.

Next, the second parameter is the column. You always get column 3 -- the hierarchy info for the 3rd contour.

What you really ought to be doing is something like

hierarchy.get(0, i)[3]

The final issue is that you're unnecessarily converting the indices to floating point numbers. This is wasteful, and counterproductive, since to be of much use you'd have to convert them back to ints. Just use an appropriate overload of get.

Now, my Java is rusty, but I think you could do something like this:

int[] current_hierarchy = new int[4];
for(int i = 0; i < squareContours.size();i++) {
    hierarchy.get(0, i, current_hierarchy);
    if (current_hierarchy[3] == -1) {
        // ... and so on

---

There's another problem I notice. After the call to findContours, the values in hierarchy correspond to the indices (positions) in the contours list. However, you first remove some contours (by inserting only a subset of them into another list), without any similar changes to the hierarchy data. Then you iterate over this subset, and end up using wrong hierarchy entries due to mismatching indices.

To solve this, I'd merge the two loops together, perhaps like this:

List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
//find all contours
Imgproc.findContours(dilated, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);

// Remove contours that aren't close to a square shape
// and remove contour if it has a parent 
List<MatOfPoint> finalContours = new ArrayList<>();
int[] current_hierarchy = new int[4];
for(int i = 0; i < contours.size(); i++){
    double area = Imgproc.contourArea(contours.get(i)); 
    MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
    double perimeter = Imgproc.arcLength(contour2f, true);
    //Found squareness equation on wiki... 
    //https://en.wikipedia.org/wiki/Shape_factor_(image_analysis_and_microscopy)
    double squareness = 4 * Math.PI * area / Math.pow(perimeter, 2);

    if(squareness >= 0.7 && squareness <= 0.9 && area >= 2000){
        hierarchy.get(0, i, current_hierarchy);
        if (current_hierarchy[3] == -1) {
            finalContours.add(contours.get(i));
        }
    }

}