CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Fornax-A
Fornax-A

Reputation: 1032

Scraping iterative data: empty list

I'm learning how to iterate scraping values from a webpage, I've used bs4 before but I can't figure out why, even if this is a simple example, I'm getting an empty list (or element).

The test I would like to print/save is:

<div data-v-1931433b="" class="mar-m c-r"> 9.1516 </div>

The code I've used is:

 1 import json
 2 import requests
 3 from bs4 import BeautifulSoup
 4
 5
 6 url = 'https://goldencoreex.com/trade/index.html?spAsseType=0#/index'
 7
 8 html_page = BeautifulSoup(requests.get(url).content, 'html.parser')
 9 elements = html_page.find_all("div", class_="mar-m c-r")
10
11 print(elements)

Only guess I have for this issue is that I may have messed-up with the url.

I would be also really much appreciate some hints on how to make the code run continuously, so to capture data whenever the element is updated.

Upvotes: 0

Views: 183

Answers (1)

baduker
baduker

Reputation: 20052

As mentioned in the comment, the page is rendered dynamically by JS, but you can query the API to get the data.

For example:

import requests
from tabulate import tabulate

api_url = "https://www.goldencoreex.com/wap/api/realtime!execute.action?symbol=btc,bts,nkn,etc,gdq,xrp,eth,ltc,akro,qtum,bhd"
response = requests.get(api_url).json()

table = [[item['name'], item['open'], item['change_ratio']] for item in response["data"]]
print(tabulate(table, headers=["Name", "Open", "Change"], tablefmt="pretty"))

Output:

+-----------+----------+--------+
|   Name    |   Open   | Change |
+-----------+----------+--------+
| BTC/USDT  | 17809.35 | -5.88  |
| BTS/USDT  |  0.0232  | -6.03  |
| NKN/USDT  | 0.019426 | -3.58  |
| ETC/USDT  |  6.2325  | -4.51  |
| GDQ/USDT  |  9.2947  | -1.34  |
| XRP/USDT  |  0.5456  | -2.38  |
| ETH/USDT  |  527.37  | -3.79  |
| LTC/USDT  |  77.23   | -11.9  |
| AKRO/USDT | 0.00913  | -8.54  |
| QTUM/USDT |  2.6376  | -6.06  |
| BHD/USDT  |  2.5054  | -2.16  |
+-----------+----------+--------+

Upvotes: 1

Related Questions