CODEWITHSUNDEEP
imageopencvimage-processingface-detectionfeature-extraction
user2921008
user2921008

Reputation: 71

Match Sketch(Drawing) face photo to digital color photo

I'm going to match the sketch face (drawing photo) in to the color photo. so for the research i want to find out what are the challenges that matching sketch drawing in to color faces. for now i have find out that

  1. resolution pixel difference
  2. texture difference
  3. distance difference
  4. and color (not much effect)

I want to know (in technical terms) what are other challenges and what are available OPEN CV and JAVA CV method and algorithms to overcome that challenges?

Here is some example of the sketches and the photos that are known to match them:

data

Upvotes: 6

Views: 843

Answers (4)

carlosdc
carlosdc

Reputation: 12152

This problem is called multi-modal face recognition. There has been a lot of interest in comparing a high quality mugshot (modality 1) to low quality surveillance images (modality 2), another is frontal images to profiles, or pictures to sketches like the OP is interested in. Partial Least Squares (PLS) and Tied Factor Analysis (TFA) have been used for this purpose.

A key difficulty is computing two linear projections from the image in modality 1 (and modality 2) to a space where two points being close means that the individual is the same. This is the key technical step. Here are some papers on this approach:

  1. Abhishek Sharma, David W Jacobs : Bypassing Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch. CVPR 2011.
  2. S.J.D. Prince, J.H. Elder, J. Warrell, F.M. Felisberti, Tied Factor Analysis for Face Recognition across Large Pose Differences, IEEE Patt. Anal. Mach. Intell, 30(6), 970-984, 2008. Elder is a specialist in this area and has a variety of papers on the topic.
  3. B. Klare, Z. Li and A. K. Jain, Matching forensic sketches to mugshot photos, IEEE Pattern Analysis and Machine Intelligence, 29 Sept. 2010.

As you can understand this is an active research area/problem. In terms using OpenCV to overcome the difficulties, let me give you an analogy: you need to build build a house (match sketches to photos) and you're asking how will having a Stanley hammer (OpenCV) will help. Sure, it will probably help. But you'll also need a lot of other resources: wood, time/money, pipes, cable, etc.

Upvotes: 4

Totoro doesn't feed AI anymore
Totoro doesn't feed AI anymore

Reputation: 3408

OpenCV can be used for feature extraction and machine learning required for this task. I guess you can start with the papers in the answers above, start with some basic features and prototype a classifier with OpenCV.

I guess you might also want to detect and match feature points on the faces. If you use this approach, you will have to do the feature point detectors on your own (training the Viola-Jones detector in OpenCV with your own data is an option).

Upvotes: 1

Francesco Callari
Francesco Callari

Reputation: 11825

I think that James Elder's old work on the completeness of the edge map (using reconstruction by solving the Laplace equation) is quite relevant here. See the results at the end of this paper: http://elderlab.yorku.ca/~elder/publications/journals/ElderIJCV99.pdf

Upvotes: 2

Mailerdaimon
Mailerdaimon

Reputation: 6090

You could give Eigenfaces a try, though i never tested them with sketches i think they could a least be a good starting point for your research.

See Wiki: http://en.wikipedia.org/wiki/Eigenface and the Tutorial for OpenCV: http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html (including not only Eigenfaces!)

Upvotes: 1

Related Questions