CODEWITHSUNDEEP
pythonweb-scrapingproxy
Renato Prado
Renato Prado

Reputation: 4150

How to set proxy To hide my IP address for scraping the webpage using dryscrape

I have a lot of active public proxys that i want to use to scrape some pages.

Can anyone give me an axample of how can i set a proxy in dryscrape?

https://github.com/niklasb/dryscrape

I tried setting sys env http_proxy with python and didnt work.

Upvotes: 2

Views: 4202

Answers (1)

avi
avi

Reputation: 9636

You can set proxy using dryscrape.Session.set_proxy method.

For example:

my_session = dryscrape.Session(base_url = 'https://stackoverflow.com')
my_session.set_proxy('127.0.0.1', 9000)

Upvotes: 5

Related Questions