CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
PieSquare
PieSquare

Reputation: 327

Python: BeautifulSoup extract all the heading text from div class

import requests
from bs4 import BeautifulSoup

res = requests.get('http://aicd.companydirectors.com.au/events/events-calendar')
soup = BeautifulSoup(res.text,"lxml")


event_containers = soup.find_all('div', class_ = "col-xs-12 col-sm-6 col-md-8")

first_event = event_containers[0]  
print(first_event.h3.text)

By using this code i'm able to extract the event name,I'm trying for a way to loop and extract all the event names and dates ? and also i'm trying to extract the location information which is visable after clicking on readmore link

Upvotes: 1

Views: 2799

Answers (2)

t.m.adam
t.m.adam

Reputation: 15376

event_containers is a bs4.element.ResultSet object, which is basically a list of Tag objects.
Just loop over the tags in event_containers and select h3 for the title, div.date for the date and a for the URL, example: 

for tag in event_containers:
    print(tag.h3.text)
    print(tag.select_one('div.date').text)
    print(tag.a['href'])

Now, for the location information you'll have to visit each URL and collect the text in div.date.
Full code: 

import requests
from bs4 import BeautifulSoup

res = requests.get('http://aicd.companydirectors.com.au/events/events-calendar')
soup = BeautifulSoup(res.text,"lxml")
event_containers = soup.find_all('div', class_ = "col-xs-12 col-sm-6 col-md-8")
base_url = 'http://aicd.companydirectors.com.au'

for tag in event_containers:
    link = base_url + tag.a['href']
    soup = BeautifulSoup(requests.get(link).text,"lxml")
    location = ', '.join(list(soup.select_one('div.event-add').stripped_strings)[1:-1])
    print('Title:', tag.h3.text)
    print('Date:', tag.select_one('div.date').text)
    print('Link:', link)
    print('Location:', location)

Upvotes: 1

SIM
SIM

Reputation: 22440

Try this to get all the events and dates you are after:

import requests
from bs4 import BeautifulSoup

res = requests.get('http://aicd.companydirectors.com.au/events/events-calendar')
soup = BeautifulSoup(res.text,"lxml")
for item in soup.find_all(class_='lead'):
    date = item.find_previous_sibling().text.split(" |")[0]
    print(item.text,date)

Upvotes: 1

Related Questions