CODEWITHSUNDEEP
pythonseleniumweb-scrapingbeautifulsoupcss-selectors
Void S
Void S

Reputation: 802

WebScrape -Getting the href

At the end of each row of this page, there is a "View Posters" link that contains a URL.

The first one I have pulled in my code, pulls fine as "ur"

I am not sure how to pull the view poster url.

rom selenium import webdriver
import time
import pandas as pd
driver = webdriver.Chrome()
import requests
from bs4 import BeautifulSoup

val=[]

absinfo=[]
sesinfo=[]

url = 'https://meetings.asco.org/meetings/2022-gastrointestinal-cancers-symposium/286/program-guide/search?q=&filters=%7B%22sessionType%22:%5B%7B%22key%22:%22Poster%20Session%22%7D%5D%7D'
res=requests.get(url)
soup=BeautifulSoup(res.content,'html.parser')


driver.get(url)
time.sleep(4)


productlist =driver.find_elements_by_xpath(".//div[@class='session-card']")
#times = soup.select('.time')

for b in productlist:
    ur=b.find_element_by_css_selector('a').get_attribute('href')

Upvotes: 0

Views: 62

Answers (1)

KunduK
KunduK

Reputation: 33384

If you want to use selenium then try with following xpath to identify both the href links under the productlist.

driver.get("https://meetings.asco.org/meetings/2022-gastrointestinal-cancers-symposium/286/program-guide/search?q=&filters=%7B%22sessionType%22:%5B%7B%22key%22:%22Poster%20Session%22%7D%5D%7D")

productlist =driver.find_elements_by_xpath(".//div[@class='session-card']")

for item in productlist:
     print("Url 1 :" + item.find_element_by_xpath(".//span[@data-cy='sessionTitle']//a").get_attribute('href'))
     print("View Poster :" + item.find_element_by_xpath(".//a[.//span[text()='View Posters']]").get_attribute('href'))

Output:

Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14170
View Poster :https://meetings.asco.org/session/14170
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14145
View Poster :https://meetings.asco.org/session/14145
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14169?presentation=205955
View Poster :https://meetings.asco.org/session/14169
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14168
View Poster :https://meetings.asco.org/session/14168
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14450
View Poster :https://meetings.asco.org/session/14450
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14163
View Poster :https://meetings.asco.org/session/14163
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14449
View Poster :https://meetings.asco.org/session/14449
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14451
View Poster :https://meetings.asco.org/session/14451
Url 1 :https://meetings.asco.org/2022-asco-gastrointestinal-cancers-symposium/14166
View Poster :https://meetings.asco.org/session/14166

Upvotes: 2

Related Questions