CODEWITHSUNDEEP
pythonbeautifulsoup
Harshy
Harshy

Reputation: 13

Web Scraping with BeautifulSoup4 failing

I was making a program that would collect the value of the cryptocurrency verge. This script did the trick:

import urllib2
from bs4 import BeautifulSoup
url=("https://coinmarketcap.com/currencies/verge/")
page=urllib2.urlopen(url)
soup=BeautifulSoup(page,"html.parser")
find_value=soup.find('span',attrs={'class':'text-large2'})
price=find_value.text

Though the issue was that the result was in USD and i lived in Australia. So what i then did was put that value in a USD to AUD converter to find my value. I tried with the following code:

url2="http://www.xe.com/currencyconverter/convert/?
Amount="+price+"&From=USD&To=AUD"
print url2
page2=urllib2.urlopen(url2)
soup2=BeautifulSoup(page2,"html.parser")
find_value2=soup.find('span',attrs={'class':'uccResultAmount'})
print find_value2

The result was that i would get the right url though i would get the wrong result. Could anybody tell me where i am going wrong?Thank You

Upvotes: 0

Views: 155

Answers (1)

Ajax1234
Ajax1234

Reputation: 71471

You can use regular expressions to scrape the currency converter:

import urllib
from bs4 import BeautifulSoup
import re
def convert(**kwargs):
   url = "http://www.xe.com/currencyconverter/convert/?Amount={amount}&From={from_curr}&To={to_curr}".format(**kwargs)
   data = str(urllib.urlopen(url).read())
   val = map(float, re.findall("(?<=uccResultAmount'>)[\d\.]+", data))
   return val[0]

url="https://coinmarketcap.com/currencies/verge/"
page=urllib.urlopen(url)
soup=BeautifulSoup(page,"html.parser")
find_value=soup.find('span',attrs={'class':'text-large2'})
print convert(amount = float(find_value.text), from_curr = 'USD', to_curr = 'AUD')

Output:

0.170358

Upvotes: 1

Related Questions