CODEWITHSUNDEEP
pythonhtmlseleniumvideoweb-scraping
user5964275
user5964275

Reputation: 83

Video url cannot be found when scraping page

I want to download the video from the following page:

https://www.indiegogo.com/projects/protest-the-hero-new-album--3#/

Using Firebug, I can see the url of the video,

<video src="https://09-lvl3-pdl.vimeocdn.com/01/1449/2/57246728/138634997.mp4?expires=1457996449&amp;token=089e435c20e7781d36fce" preload="metadata">
</video>

However, I tried to scape the page using Python, this sentence is lost and I could not get the url. I also tried Selenium, but the same problem remained. How could I access the video url with my scraper?

Also, it seems that the video url does not work. How could I get the url from which I can download the video?

Upvotes: 1

Views: 1367

Answers (1)

alecxe
alecxe

Reputation: 474231

You can solve it with selenium.

The trick is that the desired video tag is inside the iframe - you would need to switch into it's context and then search for the video element. Then, use get_attribute() to get the src attribute value. Complete working code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Chrome()  # or webdriver.Firefox(), or webdriver.PhantomJS() or etc.
wait = WebDriverWait(browser, 10)

browser.get('https://www.indiegogo.com/projects/protest-the-hero-new-album--3#/')

# waiting for the frame to become present
frame = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "#vimeoPlayer")))
browser.switch_to.frame(frame)

# get video url
url = browser.find_element_by_tag_name("video").get_attribute("src")
print(url)

browser.close()

Prints:

https://09-lvl3-pdl.vimeocdn.com/01/1449/2/57246728/138634997.mp4?expires=1457998452&token=0c54810bc365a94ea8486

Upvotes: 1

Related Questions