CODEWITHSUNDEEP
pythonpython-requestsgzip
Velcorn
Velcorn

Reputation: 60

Can't download csv.gz from website using Python

I'm currently trying to download a csv.gz file from the following link: https://www.cryptoarchive.com.au/bars/pair. As you can see, opening the link with a browser simply opens the save file dialogue. However, passing the link to requests or urllib simply downloads HTML as opposed to the actual file.

This is the current approach I'm trying: EDIT: Updated to reflect changes I've made.

url = "https://www.cryptoarchive.com.au/bars/pair"
file_name = "test.csv.gz"
headers = {"PLAY_SESSION": play_session}
r = requests.get(url, stream=True, headers=headers)
with open(file_name, "wb") as f:
    for chunk in r.raw.stream(1024, decode_content=False):
        if chunk:
            f.write(chunk)
            f.flush()

The only saved cookie I can find is the PLAY_SESSION. Setting that as a header doesn't change the result I'm getting.

Further, I've tried posting a request to the login page like this:

login = "https://www.cryptoarchive.com.au/signup"
data = {"email": email,
        "password": password,
        "accept": "checked"}

with requests.Session() as s:
    p = s.post(login, data=data)
    print(p.text)

However, this also doesn't seem to work and I especially can't figure out what to pass to the login page or how to actually check the checkbox...

Upvotes: 0

Views: 256

Answers (2)

Velcorn
Velcorn

Reputation: 60

Both @Daniel Argüelles' and @Abhyudaya Sharma's answer have helped me. The solution was simply getting the PLAY_SESSION cookie after logging into the website and passing it to the request function.

cookies = {"PLAY_SESSION": play_session}
url = "https://www.cryptoarchive.com.au/bars/pair"
r = requests.get(url, stream=True, cookies=cookies)
with open(file_name, "wb") as f:
    for chunk in r.raw.stream(1024, decode_content=False):
        if chunk:
            f.write(chunk)
            f.flush()

Upvotes: 0

Daniel Argüelles
Daniel Argüelles

Reputation: 2349

Just browsing that url from a private navigation shows the error:

Please login/register first.

To get that file, you need to login first into the site. Probably with the login you will get a session token, some cookie or something similar that you need to put in the request command.

Upvotes: 1

Related Questions