CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
H D
H D

Reputation: 81

I am trying to web scrape a website and when I am trying to turn this into a csv file, data isn't going to the correct column

I am new to web scraping and for practice I am trying to web scrape a website and turn the results into a csv file. When I come to the part to turn the results into a csv file, it doesn't put the address in the address column. I want the data to go into the address column. The code is as follows.

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://www.allagents.co.uk/find-agent/london/'

uClient = uReq(my_url)

page_html = uClient.read()

uClient.close()

page_soup = soup(page_html, 'html.parser')

containers = page_soup.findAll('div', {'class':'itemlabel3'})

filename = "webscrape.csv" 

f = open(filename, "w")

headers = "Company Name, Address, Telephone Number\n"

f.write(headers)

for container in containers:
    comp_name   = container.find('div', {'class':'labelleft2 col-md- 
10'}).div.h4.a.text

    address     = container.find('div', {'class':'labelleft2 col-md- 
   10'}).div.p.text

    tel         = container.find('div', {'class':'labelleft2 col-md- 
   10'}).div.find('p', {'style':'clear: both; margin-bottom: 
15px;'}).strong.text

    print("Company Name:", comp_name)
    print("Address:", address)
    print("Telephone", tel)

    f.write(comp_name.replace(",", ("|")) + "," + address.replace(",", ("|")) + 
"," + tel + "\n")

f.close()

Any help is appreciated. Thanks you in advance.

Upvotes: 1

Views: 66

Answers (1)

PythonUser
PythonUser

Reputation: 776

it seems like in your address data new line character is present

try to replace below line for address in your code and try running again 

address=(container.find('div', {'class':'labelleft2 col-md-10'}).div.p.text).replace('\n','')

Upvotes: 2

Related Questions