Yolov9 C++ inference outputs for x,y, width and height not as expected

Question

I am trying to do yolov9 inference in c++ using the opencv library. I use opencv version 4.9.0 with cuda 11.8 and cudnn 8.9.7.29. I am also using the yolov9-s-converted model from this repo https://github.com/WongKinYiu/yolov9 in onnx format. I export it using the export.py like this:

python3 export.py --weights yolov9-s-converted.pt --imgsz 640 --include onnx

When doing an inference on a picture I get correct detections when I do my inference on my CPU but when I do it with cuda on GPU I get the the same number of detections with the same "class" predictions and confidence, but the x, y, width and height values are all 0. Meaning that if i process an image with a frisbee and multiple persons, I get multiple predections for person and frisbee with high confidence and correct bounding boxes on CPU and all 0s for x, y, width and height when running on cuda. Here you have a mininmal running example code:

#include <opencv2/opencv.hpp>

int main(int argc, char **argv)
{
    cv::dnn::Net net = cv::dnn::readNet("../path/to/model/yolov9-s-converted-simplified.onnx");
    bool is_cuda = true;
    if (is_cuda)
    {
        std::cout << "Attempty to use CUDA\n";
        net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA);
        net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA);
    }
    else
    {
        std::cout << "Running on CPU\n";
        net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
        net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
    }
    std::string filename = "path_to_image.png";
    cv::Mat image = cv::imread(filename);
    int _max = MAX(image.cols, image.rows);
    cv::Mat input_image = cv::Mat::zeros(_max, _max, CV_8UC3);
    image.copyTo(input_image(cv::Rect(0, 0, image.cols, image.rows)));
    cv::Mat blob; 
    cv::dnn::blobFromImage(input_image, blob, 1./255., cv::Size(640, 640), cv::Scalar(), true, false); 
    net.setInput(blob); 
    std::vector<cv::Mat> outputs;
    net.forward(outputs, net.getUnconnectedOutLayersNames()); 
    cv::Mat preds = outputs.at(0); 
    cv::Mat det_output(preds.size[1], preds.size[2], CV_32F, preds.ptr<float>()); 
    det_output = det_output.t();
    for (int i = 0; i < det_output.rows; ++i) {
        cv::Mat classes_scores = det_output.row(i).colRange(4, det_output.cols);
        double score;
        cv::Point classIdPoint;
        minMaxLoc(classes_scores, 0, &score, 0, &classIdPoint);

        if(score > 0.4) {
            float cx = det_output.at<float>(i, 0);
            float cy = det_output.at<float>(i, 1);
            float ow = det_output.at<float>(i, 2);
            float oh = det_output.at<float>(i, 3);
            std::cout << "found index: " << classIdPoint.x << " at cx: " << cx << ", cy: " << cy << ", ow: " << ow << ", oh: " << oh << " with confidence: " << score << std::endl;
        }
    }
}

This code is a altered version of [yolov5-opencv-cpp-python], which is licensed under the MIT License. The original repository can be found [here](https://github.com/doleron/yolov5-opencv-cpp-python).

Output when running on CPU:

Attempty to use CUDA
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.815521
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.809074
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.758471
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.726192
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.796474
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.828768
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.854483
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.812704
found index: 29 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.608229
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.920465
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.930458
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.924879
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.91286
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.904521
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.904832
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.905469
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.901314
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.895667
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.911088
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.91148
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.909622
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.92291
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.926244
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.889289
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.914144
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.907706
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.904138
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.92109
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.926035
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.916511
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.906686
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.902697
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.901318
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.919215
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.924047
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.917313
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.887939
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.914409
found index: 0 at cx: 0, cy: 0, ow: 0, oh: 0 with confidence: 0.914886

Output when running on CPU:

Running on CPU
found index: 29 at cx: 289.611, cy: 122.513, ow: 34.0658, oh: 20.5468 with confidence: 0.815522
found index: 29 at cx: 289.647, cy: 122.389, ow: 34.351, oh: 20.7311 with confidence: 0.809075
found index: 29 at cx: 289.749, cy: 122.344, ow: 34.2258, oh: 20.8556 with confidence: 0.758471
found index: 29 at cx: 289.82, cy: 122.272, ow: 34.193, oh: 20.9125 with confidence: 0.726192
found index: 29 at cx: 289.594, cy: 122.339, ow: 34.1768, oh: 20.8335 with confidence: 0.796475
found index: 29 at cx: 289.644, cy: 122.238, ow: 34.3859, oh: 20.7112 with confidence: 0.828769
found index: 29 at cx: 289.647, cy: 122.195, ow: 34.4007, oh: 20.8877 with confidence: 0.854484
found index: 29 at cx: 289.892, cy: 122.159, ow: 34.1567, oh: 20.8711 with confidence: 0.812705
found index: 29 at cx: 289.639, cy: 122.405, ow: 34.3852, oh: 21.2653 with confidence: 0.608234
found index: 0 at cx: 323.89, cy: 287.165, ow: 93.9451, oh: 269.682 with confidence: 0.920465
found index: 0 at cx: 323.976, cy: 287.407, ow: 93.6403, oh: 270.885 with confidence: 0.930458
found index: 0 at cx: 324.207, cy: 287.597, ow: 93.2349, oh: 271.135 with confidence: 0.924879
found index: 0 at cx: 323.889, cy: 287.495, ow: 93.7401, oh: 269.943 with confidence: 0.91286
found index: 0 at cx: 323.901, cy: 287.479, ow: 93.4483, oh: 270.616 with confidence: 0.904521
found index: 0 at cx: 324.074, cy: 287.59, ow: 92.9913, oh: 270.936 with confidence: 0.904831
found index: 0 at cx: 324.064, cy: 287.61, ow: 93.4742, oh: 270.561 with confidence: 0.905469
found index: 0 at cx: 323.926, cy: 287.633, ow: 93.6999, oh: 270.392 with confidence: 0.901314
found index: 0 at cx: 324.001, cy: 287.689, ow: 93.5008, oh: 270.508 with confidence: 0.895667
found index: 0 at cx: 465.54, cy: 312.337, ow: 116.727, oh: 210.617 with confidence: 0.911089
found index: 0 at cx: 465.757, cy: 312.078, ow: 116.757, oh: 210.52 with confidence: 0.91148
found index: 0 at cx: 466.246, cy: 311.914, ow: 118.289, oh: 210.543 with confidence: 0.909622
found index: 0 at cx: 231.701, cy: 326.317, ow: 90.925, oh: 186.438 with confidence: 0.92291
found index: 0 at cx: 231.629, cy: 326.256, ow: 90.8639, oh: 186.353 with confidence: 0.926244
found index: 0 at cx: 323.679, cy: 287.466, ow: 93.8954, oh: 270.34 with confidence: 0.889289
found index: 0 at cx: 465.627, cy: 312.421, ow: 116.809, oh: 210.823 with confidence: 0.914145
found index: 0 at cx: 465.722, cy: 312.388, ow: 116.932, oh: 210.904 with confidence: 0.907707
found index: 0 at cx: 466.115, cy: 312.279, ow: 117.909, oh: 210.981 with confidence: 0.904138
found index: 0 at cx: 231.786, cy: 326.117, ow: 91.1232, oh: 186.631 with confidence: 0.92109
found index: 0 at cx: 231.607, cy: 326.078, ow: 91.0004, oh: 186.639 with confidence: 0.926036
found index: 0 at cx: 231.571, cy: 326.173, ow: 91.1691, oh: 186.417 with confidence: 0.916511
found index: 0 at cx: 465.791, cy: 312.5, ow: 117.069, oh: 209.744 with confidence: 0.906686
found index: 0 at cx: 465.705, cy: 312.386, ow: 117.201, oh: 210.218 with confidence: 0.902697
found index: 0 at cx: 466.032, cy: 312.345, ow: 118.031, oh: 210.439 with confidence: 0.901318
found index: 0 at cx: 231.655, cy: 326.171, ow: 91.0509, oh: 186.889 with confidence: 0.919215
found index: 0 at cx: 231.596, cy: 326.172, ow: 91.0338, oh: 186.856 with confidence: 0.924047
found index: 0 at cx: 231.646, cy: 326.213, ow: 91.2963, oh: 186.845 with confidence: 0.917313
found index: 0 at cx: 465.667, cy: 312.452, ow: 117.44, oh: 210.378 with confidence: 0.887939
found index: 0 at cx: 231.719, cy: 326.015, ow: 91.4443, oh: 187.143 with confidence: 0.91441
found index: 0 at cx: 231.674, cy: 326.049, ow: 91.4489, oh: 187.108 with confidence: 0.914886

I was thinking if it maybe has something to do with how I export the model to onnx format but I tried different configurations for the export without success.

Answer 1

• The link you provided is about exporting Yolov5 pytorch model into ONNX format. So it also doesn't support exporting for the versions except Yolov5.

• To export ONNX for Yolov9 you may try to use the steps which are provided here. This is for YOlov8 but it will work for Yolov9 if you follow the steps.

• Once you got your ONNX model, you can continue to use the Yolov5
  github code to test it.