CODEWITHSUNDEEP
c++opencvcython
Rashid
Rashid

Reputation: 70

Using Opencv Cuda functions from python

For one of my course projects, I need to use the OpenCVs GPU libraries. I am working on an existing code where OpenCV python is used and my work is to find a way to access the OpenCV Cuda libraries as right now there are no accessible Python bindings to OpenCV's various CUDA modules.

Two of the functions that i extremely need right now are cuda::warpPerspective and cv::cuda::DescriptorMatcher::knnMatch().

i tried to implement the warpPerspective by following what @ostrumvulpes suggested in Accessing OpenCV CUDA Functions from Python (No PyCUDA) and it is working perfectly. Right now i am stuck in DescriptorMatcher::knnMatch(). To be more precise, i need to use the brute-force descriptor matchers knnmatch function (CUDA). i searched online for example written in C++, so that i get an initial idea how to convert it via cython to make it work.

Most of the examples that i found is like following:

Ptr<cuda::DescriptorMatcher> matcher = 
cuda::DescriptorMatcher::createBFMatcher();
vector< vector< DMatch> > matches;
matcher->knnMatch(descriptors_object_Gpu, descriptors_scene_Gpu, matches, 2);

To implement these three lines i first added what i thought was necessary in the .pxd file. My pxd file looks like the following:

GpuWrapper.pxd

from libcpp cimport bool
from cpython.ref cimport PyObject
from libcpp.vector cimport vector

# References PyObject to OpenCV object conversion code borrowed from OpenCV's own conversion file, cv2.cpp
cdef extern from 'pyopencv_converter.cpp':
    #mrc689 April 20,2017
    void import_array()
    cdef PyObject* pyopencv_from(const Mat& m)
    cdef bool pyopencv_to(PyObject* o, Mat& m)

cdef extern from 'opencv2/imgproc.hpp' namespace 'cv':
    cdef enum InterpolationFlags:
        INTER_NEAREST = 0
    cdef enum ColorConversionCodes:
        COLOR_BGR2GRAY

cdef extern from 'opencv2/core/core.hpp':
    cdef int CV_8UC1
    cdef int CV_32FC1

cdef extern from 'opencv2/core/core.hpp' namespace 'cv':
    cdef cppclass Size_[T]:
        Size_() except +
        Size_(T width, T height) except +
        T width
        T height
    ctypedef Size_[int] Size2i
    ctypedef Size2i Size
    cdef cppclass Scalar[T]:
        Scalar() except +
        Scalar(T v0) except +

cdef extern from 'opencv2/core/core.hpp' namespace 'cv':
    cdef cppclass Mat:
        Mat() except +
        void create(int, int, int) except +
        void* data
        int rows
        int cols

    #added to test the Algorithm class inside core.hpp on May5th 12.52 AM.
    cdef cppclass Algorithm:
        Algorithm() except +

cdef extern from 'opencv2/core/base.hpp' namespace 'cv':
    cdef enum NormTypes:
        NORM_INF= 1,
        NORM_L1= 2,
        NORM_L2= 4,
        NORM_HAMMING= 6,
        NORM_HAMMING2= 7,

cdef extern from 'opencv2/core/cuda.hpp' namespace 'cv::cuda':
    cdef cppclass GpuMat:
        GpuMat() except +
        void upload(Mat arr) except +
        void download(Mat dst) const
    cdef cppclass Stream:
        Stream() except +

cdef extern from 'opencv2/core/types.hpp' namespace 'cv':
    cdef cppclass DMatch:
        DMatch() except +
        float distance
        int imgIdx
        int queryIdx
        int trainIdx

cdef extern from 'opencv2/core/cvstd.hpp' namespace 'cv':
    cdef cppclass Ptr[T]:
        T element_type
        Ptr() except +   


cdef extern from 'opencv2/cudafeatures2d.hpp' namespace 'cv::cuda':
    cdef cppclass DescriptorMatcher:
        @staticmethod
        Ptr[DescriptorMatcher] createBFMatcher(int normType) except+
        #Expected to see error here
        void knnMatch(GpuMat queryDescriptors, GpuMat trainDescriptors, vector[vector[DMatch]] &matches,int k)

cdef extern from 'opencv2/cudawarping.hpp' namespace 'cv::cuda':
    cdef void warpPerspective(GpuMat src, GpuMat dst, Mat M, Size dsize, int flags, int borderMode, Scalar borderValue, Stream& stream)
    # Function using default values
    cdef void warpPerspective(GpuMat src, GpuMat dst, Mat M, Size dsize, int flags)

and my pyx looks like this:

GpuWrapper.pyx

import numpy as np  # Import Python functions, attributes, submodules of numpy
cimport numpy as np  # Import numpy C/C++ API

def match_feature(np.ndarray[np.float32_t, ndim=3] _src,
                               np.ndarray[np.float32_t, ndim=2] _M):

    np.import_array()
    # Create GPU/device InputArray for src
    cdef Mat src_mat
    cdef GpuMat src_gpu
    pyopencv_to(<PyObject*> _src, src_mat)
    src_gpu.upload(src_mat)

    cdef Mat src_mat_2
    cdef GpuMat src_gpu_2
    pyopencv_to(<PyObject*> _M, src_mat_2) 
    src_gpu_2.upload(src_mat_2) 

    cdef Ptr[DescriptorMatcher] matcher= Ptr() 
    matcher = DescriptorMatcher.createBFMatcher(4)
    cdef vector[vector[DMatch]] matches
    matcher.knnMatch(src_gpu,src_gpu_2,matches,2)
    print("no problem so far")

When i tried to compile it i got an error which says 'Ptr[DescriptorMatcher]' has no attribute 'knnMatch'.

Now as far i understood, the Ptr is a shared pointer of type DescriptorMatcher, So there must be something wrong in my way of defining Ptr from .pxd file.

i just dont know how to fix it. I will really appreciate if someone can help me solving it.

Upvotes: 4

Views: 3352

Answers (1)

DavidW
DavidW

Reputation: 30929

I don't think you're using Ptr correctly (it needs dereferencing in Cython before you can get to knnMatch).

A good place to look at how to make Ptr is the C++ standard library wrappers built into Cython which wrap the similar classes std::shared_ptr and std::unique_ptr.

You don't want to do the line T element_type since this isn't interpreted as a typedef (like in the OpenCV headers) - it's interpreted as having a member called element_type of type T (which doesn't exist).

You possibly want to set up some of the other constructors for Ptr. As it stands you've only wrapped the default empty one. (It doesn't look like it matters for your code since you get it from a factory function).

Most importantly, you also want to set up the dereference operator (operator*). That's probably all you need to implement for it to work:

cdef cppclass Ptr[T]:
    Ptr() except + 
    Ptr(Ptr*) except +
    T& operator* () # probably no exceptions

To use it you use the cython.operator.dereference:

# at the top
from cython.operator cimport dereference

# later
dereference(matcher).knnMatch(src_gpu,src_gpu_2,matches,2)

(I haven't looked at the rest of the code in detail so I have no comment on whether it is right)

Upvotes: 1

Related Questions