CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Vincent
Vincent

Reputation: 61

Scrape password protected directory (with python)

I have a password protected directory with a lot of files which I am trying to download by using Python's BeautifulSoup.

enter image description here

The problem is that as it is just a password protected directory and there is no login form, I can't seem to use Mechanize like this example?

Anyone suggestions?

Upvotes: 0

Views: 479

Answers (1)

Soviut
Soviut

Reputation: 91595

There is no need to do screen automation to log in. This is using HTTP Authentication which HTTP libraries like urllib2, urllib3 and request (a library built on top of urllib) all support.

Even simpler, you could just prefix your URL with the username and password, which is what the HTTP libraries do under the hood.

https://username:[email protected]

Upvotes: 2

Related Questions