CODEWITHSUNDEEP
pythonhttppasswordsrequest
mattyd
mattyd

Reputation: 1683

Python3: Download DOM from webpage with username/password

I am trying to download the DOM from the Yahoo fantasy football page. The data requires a yahoo user. I am looking for the python library where I can add my user/pass to the request.

urllib has HTTPBasicAuthHandler which needs a HTTPPasswordMgr Objects

the add_password field says I am missing an argument, when I try and pass it the 4 it wants. I am not sure what to put for the realm. I am new to Python.

I have found the Requests to look promising, but when I install it, it throws an error and I can not import it properly :\

I was hoping this was a bit easier to do in Python!

import urllib.request
try:
    url = "http://football.fantasysports.yahoo.com/" 
    username = "un"
    password = "pw"

    pwObj = urllib.request.HTTPPasswordMgr.add_password("http://yahoo.com",url, username, password)
    request = urllib.request.HTTPBasicAuthHandler(pwObj)

    result = urllib.request.urlopen(request)

    print(result.read())


except Exception as e:
    print(str(e))
# Error: add_password() missing 1 required positional argument: 'passwd'

The ideal solution would have someone downloading yahoo DOM data from a page that requires credentials :)

Upvotes: 0

Views: 126

Answers (1)

dsgdfg
dsgdfg

Reputation: 1520

Like this:

import urllib.request
try:
    url = "http://football.fantasysports.yahoo.com/" 
    username = "_username"
    password = "_password"

    password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
    password_mgr.add_password(None, url, username, password)
    handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
    opener = urllib.request.build_opener(handler)
    opener.open("http://football.fantasysports.yahoo.com/f1/leaderboard")
    urllib.request.install_opener(opener)

    result = urllib.request.urlopen(url)

    print(result.read())


except Exception as e:
    print(str(e))

But, how to run java events ?

Upvotes: 1

Related Questions