CODEWITHSUNDEEP
c++image-processingtensorflowcomputer-vision
Abhishek
Abhishek

Reputation: 680

Implementing Structured Tensor

I am trying to implement a paper called Structured Tensor Based Image Interpolation. In the paper what it does is the use structure tensor to classify each pixel in an image into three different classes (uniform, corners and edges) based on eigen values of a structured tensor.

To a achieve this I have written the following code:

void tensorComputation(Mat dx, Mat dy, Mat magnitude)
{

    Mat dx2, dy2, dxy; 
    GaussianBlur(magnitude, magnitude, Size(3, 3), 0, 0, BORDER_DEFAULT); 
    // Calculate image derivatives 
    multiply(dx, dx, dx2);
    multiply(dy, dy, dy2);
    multiply(dx, dy, dxy);

    Mat t(2, 2, CV_32F); // tensor matrix

    // Insert values to the tensor matrix.
    t.at<float>(0, 0) = sum(dx2)[0];
    t.at<float>(0, 1) = sum(dxy)[0];
    t.at<float>(1, 0) = sum(dxy)[0];
    t.at<float>(1, 1) = sum(dy2)[0];

    // eigen decomposition to get the main gradient direction. 
    Mat eigVal, eigVec;
    eigen(t, eigVal, eigVec);

    // This should compute the angle of the gradient direction based on the first eigenvector. 
    float* eVec1 = eigVec.ptr<float>(0);
    float* eVec2 = eigVec.ptr<float>(1);
    cout << fastAtan2(eVec1[0], eVec1[1]) << endl;
    cout << fastAtan2(eVec2[0], eVec2[1]) << endl;
}

Here dx, dy, magnitude are derivative in x-axis, derivative in y- axis and magnitude of an image respectively.

What I know is I have found structured tensor for the entire image. But my problem is that I need to compute structured tensor for each pixel in an image. How to achieve this?

Upvotes: 1

Views: 655

Answers (1)

Cris Luengo
Cris Luengo

Reputation: 60761

In your code you blur magnitude, but then don't use it. You don't need this magnitude at all.

You build the structure tensor correctly, but you average over the whole image. What you want to do is apply local averaging. For each pixel, the structure tensor is the average of your matrix over the pixels in the neighborhood. You compute this by applying a Gaussian blur to each of the components of the tensor: dx2, dy2, and dxy.

The larger the sigma of the Gaussian, the larger the neighborhood you average over. You get more regularization (less sensitive to noise) but also less resolution (less sensitive to small variations and short edges). Play around with the parameter until you get what you need. Sigma between 2 and 5 are quite common.

Next, you need to compute the eigendecomposition per pixel. I don't know if OpenCV makes this easy. I recommend you use DIPlib 3 instead. It has the right infrastructure to compute and use the structure tensor. See here how easy it can be.

Upvotes: 1

Related Questions