CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Lipaaaaaa
Lipaaaaaa

Reputation: 27

Python, Web Scraping a bar graph

I am currently trying to webscrape the bar graph/chart from this page, but am unsure what specific BeautifulSoup features are needed to extract these types of bar charts. Additionally, if anyone has a link to what BeautifulSoup features are used for scraping what types of charts/graphs, that would be greatly appreciated. https://www.statista.com/statistics/215655/number-of-registered-weapons-in-the-us-by-state/

Here is the code I have so far

import pandas as pd
import requests
from bs4 import BeautifulSoup

dp = 'https://www.statista.com/statistics/215655/number-of-registered-weapons-in-the-us-by-state/'
page = requests.get(dp).text
soup = BeautifulSoup(page, 'html.parser')

#This is what I am trying to figure out
new = soup.find("div", id="bar") 
print(new)

Upvotes: 0

Views: 771

Answers (2)

Andrej Kesely
Andrej Kesely

Reputation: 195408

This script will get all data from the bar graph:

import requests
import pandas as pd
from bs4 import BeautifulSoup

url = 'https://www.statista.com/statistics/215655/number-of-registered-weapons-in-the-us-by-state/'

soup = BeautifulSoup(requests.get(url).content, 'html.parser')

tds = soup.select('#statTableHTML td')
data = []
for td1, td2 in zip(tds[::2], tds[1::2]):
    data.append({'State':td1.text, 'Number': td2.text})

df = pd.DataFrame(data)
print(df)

Prints:

                    State   Number
0                   Texas  725,368
1                 Florida  432,581
2              California  376,666
3                Virginia  356,963
4            Pennsylvania  271,427
5                 Georgia  225,993
6                 Arizona  204,817
7          North Carolina  181,209
8                    Ohio  175,819
9                 Alabama  168,265
10               Illinois  147,698
11                Wyoming  134,050
12                Indiana  133,594
13               Maryland  128,289
14              Tennessee  121,140
15             Washington  119,829
16              Louisiana  116,398
17               Colorado  112,691
18               Arkansas  108,801
19             New Mexico  105,836
20         South Carolina   99,283
21              Minnesota   98,585
22                 Nevada   96,822
23               Kentucky   93,719
24                   Utah   93,440
25             New Jersey   90,217
26               Missouri   88,270
27               Michigan   83,355
28               Oklahoma   83,112
29               New York   82,917
30              Wisconsin   79,639
31            Connecticut   74,877
32                 Oregon   74,722
33   District of Columbia   59,832
34          New Hampshire   59,341
35                  Idaho   58,797
36                 Kansas   54,409
37            Mississippi   52,346
38          West Virginia   41,651
39          Massachusetts   39,886
40                   Iowa   36,540
41           South Dakota   31,134
42               Nebraska   29,753
43                Montana   23,476
44                 Alaska   20,520
45           North Dakota   19,720
46                  Maine   17,410
47                 Hawaii    8,665
48                Vermont    7,716
49               Delaware    5,281
50           Rhode Island    4,655
51  *Other US Territories      866

Upvotes: 2

Osadhi Virochana
Osadhi Virochana

Reputation: 1302

Maybe you can find more about Web Scraping from this web site https://www.datacamp.com/community/tutorials/web-scraping-using-python

Upvotes: 1

Related Questions