CODEWITHSUNDEEP
pythonseleniumwebscreen-scraping
Natko Kraševac
Natko Kraševac

Reputation: 111

How to get links from "Inspect element" on page in python?

I need to get the video link from a web page. I click on inspect element and go to Network tab, and I see a link I need to get... But how can I access this link trough python?

this is the situation: https://i.sstatic.net/qH26K.jpg

the link is positioned in the header:

https://i.sstatic.net/2XtUM.jpg

I need only link, I don't need to download the video.

What would be the best path to go? Maybe Selenium?

Upvotes: 0

Views: 8384

Answers (2)

Hugo Sousa
Hugo Sousa

Reputation: 916

You just need to do a HTTP request to get the page and then go through the response to get the url. You need to define the XPath and use lxml to get the URL. Something like (it is just an example, probably will not work straight forward): 

import lxml.html as parser
import requests

path = <define the XPATH>
url = <your url>

data = do_request(url)
if data:
    doc = parser.fromstring(data) 
    url_res = doc.xpath(path) #the url from the webpage

#do_requests() example
def do_request(url):
    r = requests.get(url)
    return r.text if r.status_code == 200 else None

Upvotes: 0

wholevinski
wholevinski

Reputation: 3828

Selenium will work, yes. What you'll want to do is find the element in the DOM that's pulling it in. Before you go that route though, you should try to figure out what element you're after manually. You're probably after a video tag and its child source tag.

HTML 5 video tag docs: http://www.w3schools.com/tags/tag_video.asp

Selenium selector docs: https://selenium-python.readthedocs.org/locating-elements.html

Upvotes: 1

Related Questions