CODEWITHSUNDEEP
pythonbeautifulsoup
Mike
Mike

Reputation: 409

can I use integrated windows authentication with BeautifulSoup?

I am trying to scrape an internal web page with BeautifulSoup. The web page itself is only open to certain users on our network and uses integrated Windows authentication. I have a simple script below that works just fine, but I have to provide credentials.

import requests
from requests_ntlm import HttpNtlmAuth
from bs4 import BeautifulSoup

url = "internal page"
content = requests.get(url, auth=HttpNtlmAuth('domain\\myusername', 'mypw'))

result = BeautifulSoup(content.text, 'html.parser')

print(result)

Is it possible for me to run this in such a way that I don't have to provide my credentials, using integrated Windows authentication?

Thank you.

Upvotes: 1

Views: 449

Answers (1)

Alex Kosh
Alex Kosh

Reputation: 2554

There is a closed issue at requests-ntlm repo with a question:

Is there a way to authenticate with currently logged user's credentials instead of providing login/password directly inside the script?

And response from author of the lib:

Not as far as I know....

Also take a look at this question. There are some alternative ways to store your secrets.

Personally I prefer environment variables and something like secrets.py:

import os

user = os.getenv('MY_USER')
password = os.getenv('MY_PASSWORD')

And then you could import your secrets from there:

from secrets import user, password

Another non so lightweight option is to use playwriht or selenium to open real browser, which might be able to load page without hardcoded login and password

Upvotes: 2

Related Questions