CODEWITHSUNDEEP
pythonweb-scrapingurllib
Kurt Peek
Kurt Peek

Reputation: 57761

In Python, urllib.urlretrieve downloads a file which says "Go away"

I'm trying to download the (APK) files from links such as https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041. When you enter the link in your browser, it brings up a dialog to open or save the file (see below).

enter image description here

I would like to save the file using a Python script. I've tried the following:

import urllib

download_link = 'https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041'
download_file = '/tmp/apkmirror_test/youtube.apk'

if __name__ == "__main__":
    urllib.urlretrieve(url=download_link, filename=download_file)

but the resulting youtube.apk contains only the words "Go away".

Since I am able to download the file by pasting the link in my browser's address bar, there must be some difference between that and urllib.urlretrieve that makes this not work. Can someone explain this difference and how to eliminate it?

Upvotes: 0

Views: 202

Answers (1)

Andrei
Andrei

Reputation: 2665

You should not programmatically access that download page as it is disallowed in the robots.txt: https://www.apkmirror.com/robots.txt

That being said, your request header is different. Python by default sets User-Agent to something like "Python...". That is the most likely cause of detection.

Upvotes: 2

Related Questions