Online banking web scraping

Question

I'd like to web scrape my online bank website. I have multiple bank accounts (at different banks) and need to regularly pull the latest transactions in order to see my overall spend and monitor my expenses. Currently I have to go in each bank website, pull the extracts, dump them into an excel file, perform some reformatting, and filtering. I would like to automate the whole process. This starts by having a program which can automatically pull transfer history from my bank account.

I have learned about web scraping a website requiring login data using requests and beautiful soup libraries. I understand you typically need to build a 'payload' dictionary which contains: 1. username 2. password 3. the token value provided by the website

In the case of my Chase bank, I could not find the token value but found the line:

<div id=”securityToken” class=”logon-xs-toggle hidden”>
    <input id=”securityToken-input-field” class=”jpui logon-xs-toggle” min=”0” placeholder=”Token” format=”” aria-describedby=“securityToken-placeHolderAdaText securityToken-helpertext” autocomplete=”off” maxlength=”35” name=”securityToken” data-validate=”securityToken” required=”” value=”” type=”tel”>
    <span id=”securityToken-placeholderAdaText” class=”util accessible-text validation__accessible-text”>Token</span>
</div>

How can I determine the value I need for the securityToken? Thanks

Answer 1

Using selenium will make you get rid of the login requests and all the hassle bypassing their security protection since it is a framework that provides a browser automation that acts like a real person navigating.

Selenium is really easy and once you install it and downloaded the browser driver here how the login process will look like with chrome driver

import selenium.webdriver as webdriver
import selenium.webdriver.support.ui as ui
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import NoSuchElementException
from time import sleep

options = webdriver.ChromeOptions()
options.add_argument('--lang=EN')

driver = webdriver.Chrome(executable_path='assets\chromedriver', chrome_options=options)
driver.get("website loging url")
sleep(2)

driver.find_element_by_id("login").send_keys("username")
driver.find_element_by_id("password").send_keys("passowrd")
driver.find_element_by_id("submit_button").click()