Reputation: 21
from bs4 import BeautifulSoup
from pprint import pprint
import requests
url = "http://chk.tbe.taleo.net/chk01/ats/careers/searchResults.jsp?org=CDI&cws=1"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
table_main = soup.select("table#cws-search-results")
table = table_main.find_all("tr")
for tr in table:
job_title = tr.find_all("a")
job_location = tr.find_all("b")
job = {
"job_title": job_title,
"job_location": job_location
}
data.append(job)
pprint(jobs)
Upvotes: 0
Views: 5849
Reputation: 16993
You are getting an error because soup.select("table#cws-search-results")
is returns a list of Tag
objects (a 1-item list in this case) rather than a single Tag
object, and find_all
is a method of Tag
objects not of the Python list
object.
Change:
table_main = soup.select("table#cws-search-results")
to:
table_main = soup.select_one("table#cws-search-results")
to get the Tag
object representing the main table, and then calling find_all
on that object will work as expected.
Upvotes: 4