CODEWITHSUNDEEP
javascriptpythonhtmlseleniumweb-scraping
WILLIAM
WILLIAM

Reputation: 485

How to use Selenium to get real-time stock price on website?

I am working on a school project to get the real-time stock price on JPMorgan Website. I would like to get all the data shown in <div class="price_area">. I have tried beautifulsoup and yahoo api but still cannot get what I want. So, it my first time to try selenium but I have no idea how to run the javascript by it. Here is my code:

def getStockPrice():
    driver = webdriver.Chrome()
    driver.get("http://www.jpmhkwarrants.com/en_hk/market-statistics/underlying/underlying-terms/code/1")
    try:
        stock_index = WebDriverWait(driver, 10).until(
            driver.find_element_by_class_name('price_area').find_element_by_class_name('price')
        )
    finally:
        driver.quit()

But it shows the error 'WebElement' is not callable. How can I get the real-time price, % change and the open price. Thanks.

Upvotes: 1

Views: 3511

Answers (3)

Puhtooie
Puhtooie

Reputation: 126

That data isn't real-time.

You have to pay for real time data usually.

If your project involves any type of paper trading/analysis, know that everything you pull from a scrape will probably be delayed by 5-15min. I've heard Bloomberg has a free api, but I don't know if the real time data is free. Check out Interactive Brokers API. I'm pretty sure access to the data is free, and it allows you to connect to a paper trading account to test strategies, and algorithms.

Upvotes: 0

QHarr
QHarr

Reputation: 84465

You can use requests and BeautifulSoup to get the three items you mention using the Ajax query string call

import requests
from bs4 import BeautifulSoup

url= 'http://www.jpmhkwarrants.com/en_hk/ajax/terms_quick_search?_=1543832902362'
res = requests.get(url)
soup = BeautifulSoup(res.content, "lxml")
items = [item.text for item in soup.select('.price,.percentage.rise,li:nth-of-type(3) span')]
print(items)

Result:

enter image description here

The real time box has its own Ajax call of:

http://www.jpmhkwarrants.com/en_hk/ajax/market-terms-real-time-box/code/0700?_=1543832902364

You can use that to retrieve all items in that box.

import requests
from bs4 import BeautifulSoup

url= 'http://www.jpmhkwarrants.com/en_hk/ajax/market-terms-real-time-box/code/0700?_=1543832902364'
res = requests.get(url)
soup = BeautifulSoup(res.content, "lxml")
items = [item.text.strip().split('\n') for item in soup.select('.price_area div')]
tidied = [item for sublist in items for item in sublist if item and item !='Change (%)']
print(tidied)

Result:

enter image description here

Upvotes: 1

ewwink
ewwink

Reputation: 19154

to use .find_element_by_* in WebDriverWait you have to use lambda function like

stock_index = WebDriverWait(driver, 10).until(
     lambda d: d.find_element_by_class_name('price_area').find_element_by_class_name('price')
)

and don't forget to call .text to get the content

def getStockPrice():
    driver = webdriver.Chrome()
    driver.get("http://www.jpmhkwarrants.com/en_hk/market-statistics/underlying/underlying-terms/code/0700")
    try:
        stock_index = WebDriverWait(driver, 10).until(
            lambda x: x.find_element_by_class_name('price_area')
        )
        price = stock_index.find_element_by_class_name('price')
        percentage = stock_index.find_element_by_css_selector('.percentage.rise')
        open_price = stock_index.find_element_by_css_selector('ul li span')

        print('Current price: ' + price.text)
        print('Percentage: ' + percentage.text)
        print('Open price: ' + open_price.text)

    except:
        print('wait timeout, element not found')
    finally:
        driver.quit()

Upvotes: 2

Related Questions