CODEWITHSUNDEEP
pythonpython-3.x
Andre
Andre

Reputation: 370

Downloading File from URL and Writing to Location

I'm trying to download images from a list of URL's. Each URL contains a txt file with jpeg information. The URL's are uniform except for an incremental change in the folder number. Below are example URL's

Min: https://marco.ccr.buffalo.edu/data/train/train-00001-of-00407
Max: https://marco.ccr.buffalo.edu/data/train/train-00407-of-00407

I want to read each of these URL's and store the their output to another folder. I was looking into the requests python library to do this but Im wondering how to iterate over the URL's and essentially write my loop to increment over that number in the URL. Apologize in advance if I misuse the terminology. Thanks!

# This may be terrible starting code
# imported the requests library
import requests
url = "https://marco.ccr.buffalo.edu/data/train/train-00001-of-00407"

# URL of the image to be downloaded is defined as image_url
r = requests.get(url) # create HTTP response object

# send a HTTP request to the server and save
# the HTTP response in a response object called r
with open("data.txt",'wb') as f:

# Saving received content as a png file in
# binary format

# write the contents of the response (r.content)
# to a new file in binary mode.
f.write(r.content)

Upvotes: 0

Views: 89

Answers (1)

sP_
sP_

Reputation: 1866

You can generate urls like this and perform get for each

for i in range(1,408):
    url = "https://marco.ccr.buffalo.edu/data/train/train-" + str(i).zfill(5) + "-of-00407"
    print (url)

Also use a variable in the filename to keep a different copy of each. For eg, use this

with open("data" + str(i) + ".txt",'wb') as f:

Overall code may look something like this (not exactly this)

import requests

for i in range(1,408):
    url = "https://marco.ccr.buffalo.edu/data/train/train-" + str(i).zfill(5) + "-of-00407"
    r = requests.get(url) 
    # you might have to change the extension
    with open("data" + str(i).zfill(5) + ".txt",'wb') as f:
        f.write(r.content)

Upvotes: 1

Related Questions