CODEWITHSUNDEEP
pythonpython-3.xhttpbeautifulsouppython-requests
ylnor
ylnor

Reputation: 4869

Python: Trying to loggin with requests and perform a HTTP request

I am trying to loggin to my account using the following python code without success. The login-process is in two steps on two pages. First enter login, second enter password. I am using Python3:

from bs4 import BeautifulSoup
import requests, lxml.html

with requests.Session() as s:
  #First login page
    login = s.get('https://accounts.ft.com/login')
    login_html = lxml.html.fromstring(login.text)
#getting the form inputs
    hidden_inputs = login_html.xpath(r'//form//input')
    form = {x.name: x.value for x in hidden_inputs}
#filling inputs with email
    form['email'] = '[email protected]'
    response = s.post('https://accounts.ft.com/login', data=form)
# Receive reponse 200

#Second login page
    login_html = lxml.html.fromstring(response.text)
#getting inputs
    hidden_inputs = login_html.xpath(r'//form//input')
    form = {x.name: x.value for x in hidden_inputs}
#filling inputs with email and password
    form['email'] = '[email protected]'
    form['password'] = 'p****word'
    response = s.post('https://accounts.ft.com/login', data=form)
#Receive reponse 200

#Trying to read an article being loggedIn
    page = s.get('https://www.ft.com/content/173695cc-1a98-11e7-a266-12672483791a')
    soup = BeautifulSoup(page.content, 'html.parser')
    print(soup.prettify())
# data-next-is-logged-in="false" => Please Register to read this page...


<div class="js-container" data-component="two-step-login-form" id="content">
  <div class="lgn-box">
    <form action="/login/submitEmail" class="js-email-lookup-form" data-test-id="enter-email-form" method="POST" name="enter-email-form" novalidate="">
      <input name="location" type="hidden" value="" />
      <input name="continueUrl" type="hidden" value="" />
      <input name="readerId" type="hidden" value="" />
      <input name="loginUrl" type="hidden" value="/login" />
      <div class="lgn-box__title">
        <h1 class="lgn-heading--alpha">
          Sign in
        </h1>
      </div>
      <div class="o-forms-group">
        <label class="o-forms-label" for="email">
       Email address
      </label>
        <input autocomplete="off" autofocus="" class="o-forms-text js-email" id="email" maxlength="64" name="email" required="" type="email">
        <input id="password" name="password" style="display:none" type="password">
        <label for="password">
        </label>
        </input>
        </input>
      </div>
      <div class="o-forms-group">
        <button class="o-buttons o-buttons--standout o-buttons--big" name="Next" type="submit">
       Next
      </button>
      </div>
    </form>
  </div>

Upvotes: 0

Views: 290

Answers (1)

zglin
zglin

Reputation: 2919

Try using selenium to simulate the web browser as it appears that FT blocks automated access.

Alternatively you can see if a site has been archived with something like archive.is (which will pull most sites into a more machine friendly setup).

Finally, there is both a datamining API and a headline API that the FT offers at their developer page

Upvotes: 1

Related Questions