CODEWITHSUNDEEP
python
david01
david01

Reputation: 13

how to update a variable inside loop in python?

I'm trying to get data from an api where the offset is not known.so I've to iterate over until the data is not null and in each offset there is 10 record so incrementing by 10 will next 10 record .

2.since the data is huge i want to write data in different files. For that i want that after each 500 offset it write's the data in the the next file. If I've 1300 offset in total i want that it writes the 0-500 to file1 510-1000 to file2 1010-1350 to file3

i=0
data = ''
while data != ',': 
     url = test.api?offset{i}
     response=requests.get(url)
     data=response.text
     data+=data
     if i%500==0:
         fo=open("sample.txt")
         fo.write(data)
         i+=10

Upvotes: 1

Views: 128

Answers (2)

jack
jack

Reputation: 76

Try this:

i = 0
data = ''
j = 1
while data !=',':
    if i%500==0:
        with open(f"file{j}.txt", "w") as fo:
            fo.write(data)
        j += 1
        data = ''
    url = f"test.api?offset{i}"
    response = requests.get(url)
    data += response.text
    i+=10

if data != '':
    with open(f"file{j}.txt", "w") as fo:
        fo.write(data)

Basically I created another variable j to keep track of the file name, and the data variable is reset to an empty string every time after writing to the file. Also like D.L mentioned, using the with keyword would allow the file to close automatically after writing, resulting in cleaner syntax.Also, I am assuming you are developing with Python3 so I used f-string syntax.

From what I can gather, I think the data variable shouldn't be assigned and then added to itself every iteration, as that will cause a duplication of data obtained from the current iteration's get request, without preserving the data from the get requests of all previous iterations. I assume you would want to keeping adding more data until the offset reaches a multiple of 500, so that's what I tried to do.

Upvotes: 1

Omer Glazer
Omer Glazer

Reputation: 36

import requests

i: int = 0
data: str = ""

while payload != ",":
    url: str = f"test.api?offset{i}"
    response: requests.Response = requests.get(url)
    payload: str = response.text
    data += payload
    if i != 0 and i % 500 == 0:
        with open(f"offset_{i-500}-{i}.txt", "w+") as f:
            f.write(data)
        data = ""
    i += 10

a solution that complies with python conventions,

plus you need to verify i != 0, since 0 % 500 == 0 will result True.

Upvotes: 0

Related Questions