CODEWITHSUNDEEP
python-3.xweb-scrapingbeautifulsoup
Tab Key
Tab Key

Reputation: 171

Scraping store locations from a complex website

I am new to web scraping and I need to scrape store locations from the given website. The information I need includes location title, address, city, state, country, phone. So far I have extracted the webpage but I don't know how to go forward 

url = 'https://www.rebounderz.com/all-locations/'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 
Safari/537.36'}

response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')

Please guide me how can I get the required information. I have searched other answers and looked into tutorials too but the structure of this website has made me confused.

Upvotes: 0

Views: 1695

Answers (1)

Ariful Islam
Ariful Islam

Reputation: 7675

import urllib
from urllib.request import urlopen
from bs4 import BeautifulSoup
import ssl


url = "https://www.rebounderz.com/all-locations/"
context = ssl._create_unverified_context()
headers = {}
headers['User-Agent'] = 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.27 Safari/537.17'

request = urllib.request.Request(url, headers=headers)
html = urlopen(request, context=context)

soup = BeautifulSoup(html, 'lxml')

divs = soup.find_all('div', {"class":"size1of3"})

for div in divs:
    print(div.find("h5").get_text())
    print(div.find("p").get_text())

Upvotes: 1

Related Questions