CODEWITHSUNDEEP
pythonbeautifulsoup
Pawel Huszcza
Pawel Huszcza

Reputation: 49

Extracting links with href attribute in Python BeautifulSoup

I have a simple task to extract links from html (url). I do this:

> #!/usr/bin/python
> 
> import urllib import webbrowser from bs4 import BeautifulSoup
> 
> URL = "http://54.75.225.110/quiz" URL_end = "/question"
> 
> LINK = URL + URL_end file =
> urllib.urlopen("http://54.75.225.110/quiz/question") soup =
> BeautifulSoup(file)
> 
> for item in soup.find_all(href=True):
>     print item
> 
> 
> print 'Hey there!'

and this is the html:

> <html><head><meta http-equiv="Content-Type" content="text/html;
> charset=ISO-8859-1"> <script
> src="./question_files/jquery.min.js"></script> <script
> type="text/javascript">
>        function n(s) {
>               var m = 0;
>               if (s.length == 0) return m;
>               for (i = 0; i < s.length; ++i) {
>                         o = s.charCodeAt(i);          m = ((m<<5)-m)+o;           m = m & m;
>               }
>         return m;
>        };
>        $(document).ready(function() {
>                document.cookie = "client_time=" + (+new Date());
>                $(".x").attr("href", "./answer/"+n($("p[id|='magic_number']").text()));
>        }); </script> </head> <body> <p> <a class="x" style="pointer-events: none;cursor: default;"
> href="http://54.75.225.110/quiz/answer/56595">this page</a> (be
> quick). </p>

Any idea why everything my script returns is: "Hey there!"? If I modify my code to:

for item in soup.find_all('a'): print item

All I get is: 

> <a class="x" style="pointer-events: none;cursor: default;">this
> page</a>

Why, where is "href" attribute?

Upvotes: 0

Views: 2290

Answers (2)

user3742339
user3742339

Reputation:

You have a spelling mistake:

for item in soup.find_all(herf=True):

It should be href:

for item in soup.find_all(href=True):

Upvotes: 0

PepperoniPizza
PepperoniPizza

Reputation: 9112

I tested you HTML code using BeautifulSoup 4:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html)

for a in soup.find_all('a'):
    if 'href' in a.attrs:
        print a['href']


http://54.75.225.110/quiz/answer/56595

Upvotes: 1

Related Questions