CODEWITHSUNDEEP
python
Euisoo Yoo
Euisoo Yoo

Reputation: 41

Python reading url that requires logging in

I use

player_id = re.compile('<td><a href="(.+?)" onclick=')
url = 'http://www.csfbl.com/freeagents.asp?leagueid=2237'
sock = urllib.request.urlopen(url).read().decode("utf-8")

or 

import pandas as pd
pd.read_html(url)

The website in question displays two different values based on whether I am logged in or not. How can I get the values as if I was logged in?

http://www.csfbl.com

is the website.

Thanks in advance!

Upvotes: 1

Views: 106

Answers (1)

Raymond Hettinger
Raymond Hettinger

Reputation: 226376

Use the requests module to submit the credentials on the web-form at: http://www.csfbl.com/logon.asp .

Instructions for submitting web forms using requests can be found here: http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests

Other sites can be more challenging. For those, consider using selenium or mechanize.

Upvotes: 2

Related Questions