CODEWITHSUNDEEP
pythonbeautifulsoupweb-crawler
s_khan92
s_khan92

Reputation: 979

Fetching a list from a website using beautifulsoup in a dataframe column

I am trying to fetch the keywords from the article website. The website keywords look like this:

`This is the link:` `https://www.horizont.net/marketing/nachrichten/bgh-haendler-haftet-nicht-fuer-kundenbewertungen-auf-amazon-180980`

enter image description here

I am using this to fetch the keywords:

   Article_Keyword = bs.find('div', {'class':'ListTags'}).get_text()

and this is how what i am getting:

Themen Bundesgerichtshof Amazon Verband Sozialer Wettbewerb Kundenbewertung Tape dpa

I need to get it by separating each keyword by comma. I can do it by RE but some keywords are with more than one word so i need that as one keyword.

is there any way to get each keyword by separating with comma?

Upvotes: 0

Views: 35

Answers (3)

Manali Kagathara
Manali Kagathara

Reputation: 761

try this

import requests
from bs4 import BeautifulSoup

url = 'https://www.horizont.net/marketing/nachrichten/bgh-haendler-haftet-nicht-fuer-kundenbewertungen-auf-amazon-180980'
page = requests.get(url)
soup1 = BeautifulSoup(page.content, "lxml")

Article_Keyword = soup1.find('div',{'class':'ListTags'}).find_all("a")
Article_Keyword = ", ".join([keyword.text.strip() for keyword in Article_Keyword])

print(Article_Keyword)

Upvotes: 1

Spark
Spark

Reputation: 2487

I used a child class element to Identify each element separately. I hope the below code helps. 

from bs4 import BeautifulSoup as soup
from requests import get
url = "https://www.horizont.net/marketing/nachrichten/bgh-haendler-haftet-nicht-fuer-kundenbewertungen-auf-amazon-180980"
clnt = get(url)
page=soup(clnt.text,"html.parser")
data = page.find('div', attrs={'class':'ListTags'})
data1 = [ele.text for ele in data.find_all('a',attrs={'class':'PageArticle_keyword'})]
print(data1)
print(",".join(data1))

Output:

>> ['Bundesgerichtshof', 'Amazon', 'Verband Sozialer Wettbewerb', 'Kundenbewertung', 'Tape', 'dpa']
>> Bundesgerichtshof,Amazon,Verband Sozialer Wettbewerb,Kundenbewertung,Tape,dpa

Make sure you approve the answer if usefull.

Upvotes: 1

Wonka
Wonka

Reputation: 1886

Try this:

Article_Keyword = bs.find('div', {'class':'ListTags'})
aes_Article_Keyword  = Article_Keyword.find_all("a")

s_Article_Keyword = ", ".join([x.text for x in aes_Article_Keyword])

Upvotes: 1

Related Questions