CODEWITHSUNDEEP
pythonbeautifulsoup
CSharpdocsz
CSharpdocsz

Reputation: 71

BeautifulSoup Get a link's content/text

URL = "https://bitcointalk.org/index.php?board=1.0"
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')
links_with_text = []


for random in soup.find_all("span", attrs={"id": re.compile("^msg")}):
    for b in random.find_all('a', href=True):
          print(b['href'])

The above code works just fine. I can get all the topics from the first page, but the problem lays in getting the topics names. Any thoughts?

Upvotes: 0

Views: 911

Answers (1)

Diogenis Siganos
Diogenis Siganos

Reputation: 797

The following code should work:

URL = "https://bitcointalk.org/index.php?board=1.0"
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')
links_with_text = []

for random in soup.find_all("span", attrs={"id": re.compile("^msg")}):
    for b in random.find_all('a', href=True):
          print(b['href'])
          print(b.string)

Sample output:

...
https://bitcointalk.org/index.php?topic=5320354.0
Craig Wright's Latest Escapade -- Give me the bitcoins I stole from Mt. Gox!
https://bitcointalk.org/index.php?topic=5233719.0
Opera now lets US users buy crypto with Apple Pay or debit card
...

Upvotes: 2

Related Questions