CODEWITHSUNDEEP
python-3.xhttpcookies
Prashant Vikram Singh
Prashant Vikram Singh

Reputation: 101

Unable to fetch result from a website using Python

I have a webpage https://freddiemac.embs.com/FLoan/secure/login.php where i need to provide the login information and then after successful login, i will land on a different webpage https://freddiemac.embs.com/FLoan/Data/download.php from where i need to tick the terms & condition tickbox and click on continue and the page refreshes to display a bunch of zip files that i need to download.

I am using below code for logging to website:

import urllib
import urllib.request
import http.cookiejar

payload = {'username': 'username', 'password': 'password','submit': 'Submit Credentials'}

cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
login_data = urllib.parse.urlencode(payload)
binary_data = login_data.encode('utf8')

opener.open('https://freddiemac.embs.com/FLoan/secure/login.php', binary_data)
resp = opener.open('https://freddiemac.embs.com/FLoan/Data/download.php')

print(resp.read())

I am getting below result, which i feel is the content of the https://freddiemac.embs.com/FLoan/secure/login.php page only.

b'\n<!DOCTYPE html>\r\n<html>\r\n<head>\r\n<title>Freddie Mac Loan 
History</title>\r\n<link rel=\'stylesheet\' type=\'text/css\' 
href=\'/FLoan/app_corp.css\'>\r\n<link rel=\'stylesheet\' type=\'text/css\' 
href=\'/FLoan/empty.css\'>\r\n<link rel=\'stylesheet\' type=\'text/css\' 
href=\'/FLoan/empty2.css\'>\r\n</head>\r\n<body>\r\n<H2>Please log 
in</H2>\n\t<form name=\'loginform\' action=\'auth.php\' method=\'post\' 
class=\'form\'>\n\t<div class=\'row\'><div class=\'small-5 column\' 
>\n\t<label for="username" >Enter email address:</label>\n\t<input 
name=\'username\' type=\'email\' placeholder=\'[email protected]\'>
</p>\n\t</div></div>\n\t<div class=\'row\'><div class=\'small-5 column\' 
>\n\t<label for="password" >Enter password:</label>\n\t<input 
name=\'password\' type=\'password\' autocomplete=\'off\'>\n\t</div>
</div>\n\t<div class=\'row\'><div class=\'small-5 column\' >\n\t<input 
type=\'submit\' value=\'Submit Credentials\' class=\'fmSubmit\'>\n\t</div>
</div>\n\t</form>\n\t</p>\n\n</div></div></body></html>\n'

I followed the steps as given under the thread How to use Python to login to a webpage and retrieve cookies for later usage?

Appreciate your help!!

Upvotes: 0

Views: 153

Answers (1)

mjrice04
mjrice04

Reputation: 164

Hi Prashant Vikram Singh. I know you asked this question a while ago but I came across your question when I was searching for the answer of the same question myself. The key is to authenticate with auth.php and then pass in that you accept the terms and conditions. The code is below. Hope this helps you or anybody else who runs into the same question.

import requests

def get_freddie_links(payload1,payload2):
"""
This function takes in the data payloads for two http post to Freddie Mac Website, returns the list of issuance
disclosure files. 
"""
with requests.Session() as sess:
    response = sess.post('https://freddiemac.embs.com/FLoan/secure/auth.php',data=payload1)
    final_url = sess.post('https://freddiemac.embs.com/FLoan/Data/download2.php',data=payload2)
return final_url.text

singlefamily_payload = {'username' : <USERNAME>,'password' : <PASSWORD>}
payload2={'accept': 'Yes','acceptSubmit':'Continue','action':'acceptTandC'}
results = get_freddie_links(singlefamily_payload,payload2)

Upvotes: 1

Related Questions