CODEWITHSUNDEEP
pythonpython-requestswikipedia
placeholder
placeholder

Reputation: 29

Python request.get error on Wikipedia image URL

Requests.get() does not seem to be returning the expected bytes for Wikipedia image URLs, such as https://upload.wikimedia.org/wikipedia/commons/0/05/20100726_Kalamitsi_Beach_Ionian_Sea_Lefkada_island_Greece.jpg:

import wikipedia
import requests

page = wikipedia.page("beach")
first_image_link = page.images[0]
req = requests.get(first_image_link)
req.content

b'<!DOCTYPE html>\n<html lang="en">\n<meta charset="utf-8">\n<title>Wikimedia Error</title>\n<style>\n*...

Upvotes: 1

Views: 729

Answers (2)

Nova
Nova

Reputation: 452

I typed your code and it seems to be an "Error: 403, Forbidden.". Wikipedia requires a user agent header in the request.

import wikipedia
import requests

headers = {
    'User-Agent': 'My User Agent 1.0'
}

page = wikipedia.page("beach")
first_image_link = page.images[0]
req = requests.get(first_image_link, headers=headers, stream=True)
req.content

For the user agent, you should probably supply something a bit more descriptive than the placeholder I use in my example. Maybe the name of your script, or just the word "script" or something like that. I tested it and it works fine. You will get back the image as you are expecting.

Upvotes: 2

Himanshu Poddar
Himanshu Poddar

Reputation: 7779

Most websites block requests that come in without a valid browser as a User-Agent. Wikimedia is one such.

import requests
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}
res = requests.get('https://upload.wikimedia.org/wikipedia/commons/0/05/20100726_Kalamitsi_Beach_Ionian_Sea_Lefkada_island_Greece.jpg', headers=headers)
res.content

which will give you expected output

enter image description here

Upvotes: 2

Related Questions