CODEWITHSUNDEEP
javascriptparsingweb-scrapingweb-crawlerdryscrape
finstats
finstats

Reputation: 1409

Python: Javascript rendered webpage not parsing

I want to parse the information in the following url. I want to parse the Name of the trade, the strategy description and the transactions in the "Trading History" and "Open Positions". When I parse the page, I do not get this data. I am new to parsing javascript rendered webpages so I would appreciate some explanation why my code below isn't working.

import bs4 as bs
import urllib
import dryscrape
import sys
import time

url = 'https://www.zulutrade.com/trader/314062/trading'

sess = dryscrape.Session()
sess.visit(url)
time.sleep(10)
sauce = sess.body()
soup = bs.BeautifulSoup(sauce, 'lxml')

Thanks!

Upvotes: 0

Views: 209

Answers (1)

SIM
SIM

Reputation: 22440

Your link in the code doesn't allow you to get anything cause the original url you should play with is the one I'm pasting below .The one you tried to work with automatically gets redirected to the one I mentioned here.

https://www.zulutrade.com/zulutrade-client/traders/api/providers/314062/tradeHistory?

Scraping json data out of the table from that page is as follows:

import requests
r = requests.get('https://www.zulutrade.com/zulutrade-client/traders/api/providers/314062/tradeHistory?')
j = r.json()
items = j['content']
for item in items:
    print(item['currency'],item['pips'],item['tradeType'],item['transactionCurrency'],item['id'])

Upvotes: 0

Related Questions