CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoupcryptocurrency
Oliver May
Oliver May

Reputation: 23

Web scraping with beautifulsoup not finding anything

I'm trying to scrape coinmarketcap.com just to get an update of a certain currency price, also just to learn how to web scrape. I'm still a beginner and can't figure out where I'm going wrong, because whenever I try to run it, it just tells me there are none. Although I know that line does exist. Any help is appreciated!

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find('data-currency-price data-usd=')
print (price)

Upvotes: 0

Views: 1510

Answers (5)

QHarr
QHarr

Reputation: 84465

If you are going to be doing alot of this consider doing a single call using the official API and get all the prices. Then extract what you want. The following is from the site with an amendment by me to show the desired value for electroneum. The API guidance shows how to retrieve one at a time as well, though that requires a higher plan than the basic.

from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects
import json

url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest'
parameters = {
      'start': '1',
      'limit': '5000',
      'convert': 'USD',
  }
headers = {
      'Accepts': 'application/json',
      'X-CMC_PRO_API_KEY': 'yourKey',
  }

session = Session()
session.headers.update(headers)

try:
    response = session.get(url, params=parameters)
    # print(response.text)
    data = json.loads(response.text)
    print(data['data'][64]['quote']['USD']['price'])
except (ConnectionError, Timeout, TooManyRedirects) as e:
    print(e)

You can always deploy a loop and check against a desired list e.g.

interested = ['Electroneum','Ethereum']
for item in data['data']:
    if item['name'] in interested:
        print(item)

For your current example:

You can use an attribute selector for data-currency-value

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
soup.select_one('[data-currency-value]').text

Upvotes: 1

KunduK
KunduK

Reputation: 33384

You can use the class attribute to get the value.

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find('span' ,attrs={"class" : "h2 text-semi-bold details-panel-item--price__value"})
print (price.text)

Output :

0.006778

Upvotes: 0

timmy turner
timmy turner

Reputation: 98

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
x=soup(id="quote_price").text
print (x)

Look for ID better,or search through soup.find_all(text="data-currency-price data-usd")[1].text

Upvotes: 0

Julian Silvestri
Julian Silvestri

Reputation: 2027

You should try to be more specific in how you want to FIND the item.

you currently are using soup.find('') I am not sure what you have put inside this as you wrote data-currency-price data-usd= Is that an ID a class name?

why not try finding the item using an ID. 

soup.find(id="link3")

or find by tag 

soup.find("relevant tag name like div or a")

or something like this

find_this = soup.find("a", id="ID HERE")

Upvotes: 0

drec4s
drec4s

Reputation: 8077

You can get the value like this:

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find("span", id="quote_price").get('data-usd')
print (price)

Upvotes: 0

Related Questions