CODEWITHSUNDEEP
pythoncsv
Maxinef23
Maxinef23

Reputation: 169

Why does my Python program close after running the first loop?

I'm new to Python and scraping. I'm trying to run two loops. One goes and scrapes ids from one page. Then, using those ids, I call another API to get more info/properties.

But when I run this program, it just runs the first bit fine (gets the IDs), but then it closes and doesn't run the 2nd part. I feel I'm missing something really basic about control flow in Python here. Why does Python close after the first loop when I run it in Terminal?

import requests
import csv
import time
import json
from bs4 import BeautifulSoup, Tag

file = open('parcelids.csv','w')
writer = csv.writer(file)
writer.writerow(['parcelId'])

for x in range(1,10):
    time.sleep(1) # slowing it down
    url = 'http://apixyz/Parcel.aspx?Pid=' + str(x)
    source = requests.get(url)
    response = source.content
    soup = BeautifulSoup(response, 'html.parser')
    parcelId = soup.find("span", id="MainContent_lblMblu").text.strip()
    writer.writerow([parcelId])

out = open('mapdata.csv','w')   
with open('parcelIds.csv', 'r') as in1:
    reader = csv.reader(in1)
    writer = csv.writer(out)
    next(reader, None) # skip header
    for row in reader:
        row = ''.join(row[0].split())[:-2].upper().replace('/','-') #formatting
        url="https://api.io/api/properties/"
        url1=url+row
        time.sleep(1) # slowing it down
        response = requests.get(url1)       
        resp_json_payload = response.json()
        address = resp_json_payload['property']['address']
        writer.writerow([address])

Upvotes: 0

Views: 52

Answers (1)

lostbard
lostbard

Reputation: 5220

If you are running in windows (where filenames are not case sensitive), then the file you have open for writing (parcelids.csv) is still open when you reopen it to read from it.

Try closing the file before opening it to read from it.

Upvotes: 2

Related Questions