CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
HansDampf
HansDampf

Reputation: 121

How to Scrape href from Webpage (done it before but does not work for one website?

i am trying to scrape the links from certain div class (class="card-img-block") of the following website : https://milled.com/OneKingsLane

I have managed to do this before on many different websites with the following method:

import requests
from bs4 import BeautifulSoup

session = requests.Session()
html = 'https://milled.com/OneKingsLane'
req = session.get(html)
bs = BeautifulSoup(req.text, 'html.parser')

link_box = bs.find_all('div', attrs={'class': 'card-img-block'})

for links in link_box:
    print(links['href'])

But for some reason when I use the same method I get the following error:

    return self.attrs[key]
KeyError: 'href'

Has anyone got an idea how I can scrape the URL from that div class?

Thanks :)

Upvotes: 0

Views: 582

Answers (1)

drec4s
drec4s

Reputation: 8077

You need to access the a tag inside each links element:

for links in link_box:
    print(links.a['href'])

Upvotes: 1

Related Questions