CODEWITHSUNDEEP
pythonloopscsv
juasmilla
juasmilla

Reputation: 155

How to iterate over URLs for writing CSV

I would like to have the information captured in the web by different URLs (I have them in a list called "cod") written to a CSV file, row by row (for export to Excel).

I have tried with just one link, but if I want to do it with all the elements of the List, I'd need to iterate, and am having difficulty.

My code:

import urllib
from bs4 import BeautifulSoup
import csv
urlfixed = "http://www.fatm.com.es/Datos_Equipo.asp?"


cod = ["01GR0001","01GR0004","03GR0006","02GR0003","01GR0030","01GR0018","04GR0007","03GR0032","01AL0001","02AL0003"]
loong = len(cod)
i = 0

sock = urllib.urlopen(urlfixed + "Cod=" + cod[i])
htmlSource = sock.read()
sock.close()
soup = BeautifulSoup(htmlSource)
form = soup.find("form", {'id': "FORM1"})

valores = [item.get('value') for item in form.find_all('input')]
valores.remove('Imprimir')
valores.remove('Cerrar')
values = valores

out = open('tomate.csv', 'w')
w = csv.writer(out)
w.writerow([s.encode("utf-8") for s in values])
out.close()

So, one row with the info from one "cod", and that should makes 10 lines in the "tomate.csv".

Upvotes: 0

Views: 297

Answers (1)

Stormvirux
Stormvirux

Reputation: 909

Just use a for loop with the iterator iterating through the list cod and you are opening the file for writing when it should have been append :

urlfixed = "http://www.fatm.com.es/Datos_Equipo.asp?"
cod = ["01GR0001","01GR0004","03GR0006","02GR0003","01GR0030","01GR0018","04GR0007","03GR0032","01AL0001","02AL0003"]
for i in cod:
    sock = urllib.urlopen(urlfixed + "Cod=" + i)
    htmlSource = sock.read()
    sock.close()
    soup = BeautifulSoup(htmlSource)
    form = soup.find("form", {'id': "FORM1"})

    valores = [item.get('value') for item in form.find_all('input')]
    valores.remove('Imprimir')
    valores.remove('Cerrar')
    values = valores

    out = open('tomate.csv', 'ab')
    w = csv.writer(out)
    w.writerow([s.encode("utf-8") for s in values])
    out.close()
#the loop ends here

Upvotes: 3

Related Questions