CODEWITHSUNDEEP
image-processingopencvcomputer-vision
Ann Orlova
Ann Orlova

Reputation: 1348

How to distinguish photo from picture?

I have the following problem: I am given a set of images and I need to devide them to photos and pictures(graphics) with means of OpenCV library.

I've already tried

  1. to analyze RGB histogram (in average picture has empty bins of histogram),
  2. to analyze HSV histogram (in average picture has not much colors),
  3. to search for contours (in average the number of contours on picture is less than on photo).

So I have 7% error (tested on 2000 images). I'm confused a little, because I have no a lot of experience in numerous computer vision means.

For example,this photo below. Its histograms (RGB and HSV) are very poor and number of contours is rather small. In addition there is a lot of background color, so I need to find an object to calculate only it histogram (I use findContours() for this). But in any case my algorithm detects this image as picture.

photo 1

And one more example:

photo 2

The problem with pictures is noise. I have images of small size (200*150) and in some cases noise is so perceptible, that my algorithm detects this image as photo. I've tried to blur images, but in this case the number of colors increases because of mixing pixels and also it decreases the number of contours (some dim boundaries become indistinguishable).

Example of pictures: picture 1 picture 2

I've also tried color segmentation and MSER, but my best result is still 7%.

Could you advice me what methods can I also try?

Upvotes: 3

Views: 738

Answers (3)

Tomas Greif
Tomas Greif

Reputation: 22661

I've used your dataset to create really simple models. To do this, I've used Rattle library in R.

Input data

 rgbh1 - number of bins in RGB histogram, which value > @param@, in my case @param@ = 30 (340 is maximum value)
 rgbh2 - number of bins in RGB histogram, which value > 0 (not empty)
 hsvh1 - number of bins in HSV histogram, which value > @param@, in my case @param@ = 30 (340 is maximum value)
 hsvh2 - number of bins in HSV histogram, which value > 0 (not empty)
 countours - number of contours on image
 PicFlag - flag indicating picture/photo (picture = 1, photo = 0)

Data exploration

To better understand your data, here is a plot of distribution of individual variables by picture/photo group (there is percentage on y axis):

enter image description here

It clearly shows that there are variables with preditive power. Most of them can be used in our model. Next I've created simple scatter plot matrix to see whether some combination of variables can be useful:

enter image description here

You can see the for example combination of number of countours and rgbh1 looks promising.

On the following chart you can notice that there is also quite strong correlation among variables. (Generally, we like to have a lot of variables with low correlation, while you have only a limited number of correlated variables). Pie chart shows how big is the correlation - full circle means 1, empty circle means 0, my opinion is that if correlation exceeds .4 it might not be good idea to have both variables in the model)

enter image description here

Model

Then I created simple models (keeping Rattle's default) using decision tree, random forest, logistic regression and neural network. As input I've used your data with 60/20/20 split (training, validiation, testing dataset). This is my result (please refer to google if you don't understand error matrix):

Error matrix for the Decision Tree model on pics.csv [validate] (counts):

      Predicted
Actual   0   1
     0 167  22
     1   6 204

Error matrix for the Decision Tree model on pics.csv [validate] (%):

      Predicted
Actual  0  1
     0 42  6
     1  2 51

Overall error: 0.07017544

Rattle timestamp: 2013-01-02 11:35:40 
======================================================================
Error matrix for the Random Forest model on pics.csv [validate] (counts):

      Predicted
Actual   0   1
     0 170  19
     1   8 202

Error matrix for the Random Forest model on pics.csv [validate] (%):

      Predicted
Actual  0  1
     0 43  5
     1  2 51

Overall error: 0.06766917

Rattle timestamp: 2013-01-02 11:35:40 
======================================================================
Error matrix for the Linear model on pics.csv [validate] (counts):

      Predicted
Actual   0   1
     0 171  18
     1  13 197

Error matrix for the Linear model on pics.csv [validate] (%):

      Predicted
Actual  0  1
     0 43  5
     1  3 49

Overall error: 0.07769424

Rattle timestamp: 2013-01-02 11:35:40 
======================================================================
Error matrix for the Neural Net model on pics.csv [validate] (counts):

      Predicted
Actual   0   1
     0 169  20
     1  15 195

Error matrix for the Neural Net model on pics.csv [validate] (%):

      Predicted
Actual  0  1
     0 42  5
     1  4 49

Overall error: 0.0877193

Rattle timestamp: 2013-01-02 11:35:40 
======================================================================

Results

As you can see the overall error rate oscilates between 6.5% and 8%. I do not think that this result can be significantly improved by tunning parameters of used methods. There are two ways how to decrease overall error rate:

  • add more uncorrelated variables (we do usually have 100+ input variables in the modeling dataset and +/- 5-10 are in the final model)
  • add more data (we can then tune the model without being scared by overfitting)

Used sofware:

Code used to create corrgram and scatterplot (other outputs were generated using Rattle GUI):

# install.packages("lattice",dependencies=TRUE)
# install.packages("car")

library(lattice)
library(car)

setwd("C:/")

indata <- read.csv2("pics.csv")

str(indata)


# Corrgram
corrgram(indata, order=TRUE, lower.panel=panel.shade,
         upper.panel=panel.pie, text.panel=panel.txt,
         main="Picture/Photo correlation matrix")

# Scatterplot Matrices
attach(indata)
scatterplotMatrix(~rgbh1+rgbh2+hsvh1+hsvh2+countours|PicFlag,main="Picture/Photo scatterplot matrix",
                  diagonal=c("histogram"),legend.plot=TRUE,pch=c(1,1))

Upvotes: 3

user334856
user334856

Reputation:

One feature that should be useful is the gradient histogram. Natural images have a particular distribution of gradient strengths.

Upvotes: 1

iNFINITEi
iNFINITEi

Reputation: 1534

Well a generic suggestion will be to increase the number of features ( or get better features) and to build a classifier using this features, trained with an appropriate machine learning algorithm. OpenCV already has couple of good machine learning algorithms, which you can make use of.

I have never worked on this problem, but a quick google search led me to this paper by Cutzu et. al. Distinguishing paintings from photographs

Upvotes: 1

Related Questions