CODEWITHSUNDEEP
pythonpython-3.xweb-scraping
w13rfed
w13rfed

Reputation: 375

Python Web Scraper print issue

I have created a web scraper in python but when printing at the end I want to print ("Bakerloo: " + info_from_website) that I have downloaded as you can see in the code, but it always comes out just as info_from_website and ignores the "Bakerloo: " string. Can't find anyway of solving it.

import urllib
import urllib.request
from bs4 import BeautifulSoup
import sys

url = 'https://tfl.gov.uk/tube-dlr-overground/status/'
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page,"html.parser")

try:
   bakerlooInfo = (soup.find('li',{"class":"rainbow-list-item bakerloo "}).find_all('span')[2].text)
except:
   bakerlooInfo = (soup.find('li',{"class":"rainbow-list-item bakerloo disrupted expandable "}).find_all('span')[2].text)

bakerloo = bakerlooInfo.replace('\n','')
print("Bakerloo     : " + bakerloo)

Upvotes: 1

Views: 118

Answers (1)

alecxe
alecxe

Reputation: 473753

I would use a CSS selector instead, getting the element with disruption-summary class:

import requests
from bs4 import BeautifulSoup

url = 'https://tfl.gov.uk/tube-dlr-overground/status/'
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")

service = soup.select_one('li.bakerloo .disruption-summary').get_text(strip=True)
print("Bakerloo: " + service)

Prints:

Bakerloo: Good service

(using requests here).

Note that if you want to just list all stations with the disruption summaries, do:

import requests
from bs4 import BeautifulSoup

url = 'https://tfl.gov.uk/tube-dlr-overground/status/'
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")

for station in soup.select("#rainbow-list-tube-dlr-overground-tflrail-tram ul li"):
    station_name = station.select_one(".service-name").get_text(strip=True)
    service_info = station.select_one(".disruption-summary").get_text(strip=True)

    print(station_name + ": " + service_info)

Prints:

Bakerloo: Good service
Central: Good service
Circle: Good service
District: Good service
Hammersmith & City: Good service
Jubilee: Good service
Metropolitan: Good service
Northern: Good service
Piccadilly: Good service
Victoria: Good service
Waterloo & City: Good service
London Overground: Good service
TfL Rail: Good service
DLR: Good service
Tram: Good service

Upvotes: 2

Related Questions