CODEWITHSUNDEEP
pythonseleniumweb-scrapingproxyheadless
John DeBritto
John DeBritto

Reputation: 141

How do you run headless chrome and a proxy using selenium in python?

My python script has the headless chrome in selenium up and functional but how, if possible, can I use a proxy as well? How can I pass the proxy host port to my headless chrome browser?

options = webdriver.ChromeOptions()  
options.add_argument('headless')  
browser = webdriver.Chrome(chrome_options=options)

How do I use the proxy host port with selenium and mainly headless chrome? Thanks!

Upvotes: 1

Views: 9292

Answers (2)

SIM
SIM

Reputation: 22440

The simplest way to do is something like below. Put any ip address in the proxy variable to find out how it works. ip and port are separated by :.

from selenium import webdriver

proxy = "94.122.251.105:3128" #test with any ip address which supports `http` as well because the link within the script are of `http`

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--proxy-server={}'.format(proxy))
driver = webdriver.Chrome(chrome_options=chrome_options)

driver.get('http://www.lagado.com/proxy-test')
items = driver.find_element_by_css_selector(".main-panel p:nth-of-type(2)").text
print(items) #it should print out the ip address you are using in the `proxy` variable above
driver.quit()

Upvotes: 0

Adelina
Adelina

Reputation: 11931

Something like this:

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType


options = webdriver.ChromeOptions()  
options.add_argument('headless')
desired_caps = options.to_capabilities()

prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = "ip_addr:port"
prox.socks_proxy = "ip_addr:port"
prox.ssl_proxy = "ip_addr:port"
prox.add_to_capabilities(desired_caps)


browser = webdriver.Chrome(desired_capabilities=desired_caps)

Upvotes: 1

Related Questions