CODEWITHSUNDEEP
python
David
David

Reputation: 153

How can I scrape the daily price changes from Coinmarketcap with requests_html?

I want to get the daily price changes from coinmarketcap for all coins available on the website.

I have tried to scrape and put the daily changes into a list but, somehow I'm getting the hourly, daily and weekly changes into the list. The code I used:

import requests
from requests_html import HTML, HTMLSession

r = HTMLSession().get('https://coinmarketcap.com/all/views/all/')

table = r.html.find('tbody')

delta_list = []

for row in table:
    change = row.find('.percent-change')    
    for d in change:
        delta = d.text
        delta_list.append(delta)

print(delta_list)

How can I scrape only the daily changes?

Upvotes: 0

Views: 313

Answers (1)

Trapli
Trapli

Reputation: 1597

Since requests_html supports xpath...

from requests_html import HTMLSession

r = HTMLSession().get('https://coinmarketcap.com/all/views/all/')

# get the table by id
table = r.html.xpath('//*[@id="currencies-all"]')
# filter table rows to tr elements with id
rows = table[0].xpath('//tr[@id]')

# your list of results
delta_list = []

# iterate in the rows result
for row in rows:
    # get the cryptocurrency name
    name = row.xpath('//*[@class="no-wrap currency-name"]')[0].text.replace('\n', ' ')
    # get the element which contains the 24h cahnge data
    val_elem = row.xpath('//*[@data-timespan="24h"]')
    # some currencies are too fresh to have a result in 24h, they contain '?'
    # Such elements don't have the @data-timespan="24h" attribute
    # So if the result is empty something should be done, I decided to add 0
    val = val_elem[0].text if val_elem else 0

    # just debug print
    print(f"Change of {name} in the past 24h is {val}")

    # add the result to your list
    delta_list.append(val)

On sidenote, using a list to store the results is not the best choice. The currencies are sorted by "market cap" and order of some currencies may change on any day. Using a dict/OrderedDict would be a better choice because that way you can pair currencies with values...

Upvotes: 1

Related Questions