CODEWITHSUNDEEP
google-apigoogle-custom-search
Imanol Barba Sabariego
Imanol Barba Sabariego

Reputation: 115

Google Custom Search API returning HTML documents instead of images

I started using the Google Custom Search API for a project, the idea is to search for images, and I wanted to use the Custom Search because the Google Images API is deprecated.

I already enabled image search on the CSE console

My query is like this:

https://www.googleapis.com/customsearch/v1?key=APIKEY&cx=CSECX&q=flower&alt=json&searchType=image&num=1&start=NUMBER

Where NUMBER is a random value between 1 and 20

Sometimes, it returns results like this:

{u'kind': u'customsearch#result', u'title': u'Flower Wallpaper Tumblr #6790199', u'displayLink': u'7-themes.com', u'htmlTitle': u'<b>Flower</b> Wallpaper Tumblr #6790199', u'snippet': u'Flower Wallpaper Tumblr', u'htmlSnippet': u'<b>Flower</b> Wallpaper Tumblr', u'link': u'http://7-themes.com/data_images/out/7/6790199-flower-wallpaper-tumblr.jpg', u'mime': u'image/jpeg', u'image': {u'thumbnailWidth': 150, u'byteSize': 808360, u'height': 1200, u'width': 1920, u'contextLink': u'http://7-themes.com/6790199-flower-wallpaper-tumblr.html', u'thumbnailLink': u'https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSad0z_Wla0nRHAcQrjO5jLQkFjcoqnNHhejjuGmdA1AW2BqIVEpLARAk0s', u'thumbnailHeight': 94}}

Highlighting the interesting part:

u'link': u'http://7-themes.com/data_images/out/7/6790199-flower-wallpaper-tumblr.jpg', u'mime': u'image/jpeg'

So it seems that the URL is http://7-themes.com/data_images/out/7/6790199-flower-wallpaper-tumblr.jpg and mimetype is image/jpeg, but if you go to the URL, you'll see it's not an image, but an HTML document

Of course, I could capture this as an exception, but I don't want to waste daily API requests (out of a 100 limit per day) because the API didn't give me an image when I explicitly said so.

So, the question is: Is this normal behaviour, or misconfiguration/misuse on my part? If so, how could I fix it?

Thanks for your attention

Upvotes: 1

Views: 409

Answers (1)

Woodrow Barlow
Woodrow Barlow

Reputation: 9047

After a little bit of reading, my best guess is that some servers are doing a resource redirect to prevent external sources from hotlinking directly to a resource. The file in question is advertised as an image, but accessing it from an external server will provide an HTML document instead. This is not a URL redirect, so it isn't detected by clients (including the Google crawler) until the resource is downloaded.

This sort of resource redirect is done on Apache servers using the .htaccess file and the RewriteEngine, with a technique similar to the one described here, although that particular technique can't be used to bait-and-switch images for HTML documents.

In short, if a server is lying about what type of file it's hosting, Google can't do anything about that. You can confirm that this is not an issue with the custom search API by performing the same query on the normal web search interface -- notice that clicking the image loads an HTML document rather than the image itself.

Upvotes: 1

Related Questions