CODEWITHSUNDEEP
javahtml
Idan
Idan

Reputation: 5745

How to find and extract "main" image in website

I need help tackling a problem. I need a program which, given a site, finds and extracts the "main" picture, i.e. the one which represents the site. (To say it is the biggest or the first picture is sometimes but not always true).

How should I approach this? Are there any libraries that could help me with this? Thanks!

Upvotes: 9

Views: 4065

Answers (5)

Ma'moon Al-Akash
Ma'moon Al-Akash

Reputation: 5393

ImageResolver can do that for you without the need of server side interaction, except for a small proxy script.

Upvotes: 0

mmx73
mmx73

Reputation: 972

Another solution would be to extract the meta tags for social media sharing first, if they are present, you are lucky otherwise you stil can try the other solutions.

<meta property="og:image" content="http://www.example.com/image.jpg"/>
<meta name="twitter:image" content="http://www.example.com/image.jpg">
<meta itemprop="image" content="http://www.example.com/image.jpg">

If you are yousing JSOUP the code would be like that:

    String imageUrlOpenGraph = document.select("meta[property=og:image]").stream()
            .findFirst()
            .map(doc -> doc.attr("content").trim())
            .orElse(null);

    String imageUrlTwitter = document.select("meta[name=twitter:image]").stream()
                .findFirst()
                .map(doc -> doc.attr("content").trim())
                .orElse(null);

    String imageUrlGooglePlus = document.select("meta[itemprop=image]").stream()
                .findFirst()
                .map(doc -> doc.attr("content").trim())
                .orElse(null);

Upvotes: 5

lex82
lex82

Reputation: 11307

You could use a service like embedly. Among a lot of other information they allow you to extract the main image of any page. Works particularly well for articles. You can try it here.

Upvotes: 1

mqchen
mqchen

Reputation: 4193

OPTION 1

You could checkout Goose. It does something similar to what Pocket and Readability does, i.e. try to extract the main article from a given webpage using a set of heuristics. It can apparently also extract the main image from that article, but it is a bit of a hit and miss, so 60% of the time it works everytime.

It used to be a Java project but rewritten to Scala.

From the readme

Goose will try to extract the following information:

  • Main text of an article
  • Main image of article
  • Any Youtube/Vimeo movies embedded in article
  • Meta Description
  • Meta tags
  • Publish Date

Try it here: http://jimplush.com/blog/goose

OPTION 2

You could use a Java wrapper (e.g. GhostDriver) for running a headless browser, like PhantomJS. Then, fetch the website and find the img element with the largest dimensions. This GhostDriver test case shows how to query the DOM for elements and get it's renderd size.

OPTION 3

Use a library like jsoup that helps you parse HTML. Then get the value from the src attribute from all img tags. Request each URL you find for an image and measure their sizes. The one with the biggest dimensions is likely to be the website's main image.

Upvotes: 11

Anshu Dwibhashi
Anshu Dwibhashi

Reputation: 4675

You need artificial intelligence to do so, Computer Vision namely. It too big to fit in an answer. This link might help

If you are a mathematician with experience of Probability and Bayes rule, then you can just take the unit called Image Processing and Computer Vision.

If you are looking for available software you want to use check this out...

This stackoverflow thread might help...

There's this software called moodstocks which might help.

Upvotes: 0

Related Questions