Davide
Reputation: 15
Problem with 'find' method (BeautifulSoup) when parsing xml webpage
I’m trying to extract some XML data from WASDE (“World Agricultural Supply and Demand Estimates) webpage using BeautifulSoup and using the ‘find’ method and ‘lxml’ parser but I have an error and I cannot find a solution. See the code below.
Data1, Data2 and Data4 are exactly in the same path. I can extract Data1 and Data2 but python gives me an attribute error on Data4: AttributeError: 'NoneType' object has no attribute 'cell'.
The only difference I found is that the attribute1 of Data1 and Data2 are single words “Production” (Data1) and “Imports” (Data2), nevertheless, the attribute1 of Data4 is composed of two separated words “Ending Stocks”. This seems to be the problem because if you replace “Ending Stocks” with “Domestic Feed” which is in the same path, you will find the same error but if you replace it with “Exports” which is also in the same path, the code will work fine.
But if you look now to Data3, which is in a different path but has a long attribute1 “Yield per Harvested Acre”, the code will work fine. So, the question is: Why ‘find’ method of Beutifulsoup doesn’t work sometimes when there is more than one word in an attribute? Any solution or better coding for this?
# Import libraries to be used
from urllib.request import urlopen
import requests
from bs4 import BeautifulSoup
# URL to be parsed
URL = 'https://www.usda.gov/oce/commodity/wasde/wasde1119.xml'
# Downloading URL and creating BeautifulSoup object
html = requests.get(URL)
soup = BeautifulSoup(html.text, "lxml")
# Get Data1 and print it
Data1 = soup.sr19.matrix1.find("m1_month_group",{"forecast_month1" :"Oct"}).find("m1_attribute_group",{"attribute1": "Production"}).cell["cell_value1"]
print("Data1 = ", Data1)
# Get Data2 and print it
Data2 = soup.sr19.matrix1.find("m1_month_group",{"forecast_month1" :"Oct"}).find("m1_attribute_group",{"attribute1": "Imports"}).cell["cell_value1"]
print("Data2= ", Data2)
# Get Data3 and print it
Data3 = soup.sr11.matrix1.find("attribute1",{"attribute1":"Yield per Harvested Acre"}).find("m1_month_group",{"forecast_month1": "Nov"}).cell["cell_value1"]
print("Data3= ", Data3)
# Get Data4 and print it
Data4 = soup.sr19.matrix1.find("m1_month_group",{"forecast_month1" :"Oct"}).find("m1_attribute_group",{"attribute1": "Ending Stocks"}).cell["cell_value1"]
print("Data4= ", Data4)
Upvotes: 0
Views: 135
Answers (2)
αԋɱҽԃ αмєяιcαη
Reputation: 11515
import requests
from bs4 import BeautifulSoup
r = requests.get(
'https://www.usda.gov/oce/commodity/wasde/wasde1119.xml')
soup = BeautifulSoup(r.text, 'lxml')
production = []
imports = []
nov = []
stock = []
for item in soup.findAll('m1_attribute_group', attrs={'attribute1': 'Production'}):
for cell in item.findAll('cell'):
cell = cell.get('cell_value1')
if cell is not None:
production.append(cell)
for item in soup.findAll('m1_attribute_group', attrs={'attribute1': 'Imports'}):
for cell in item.findAll('cell'):
cell = cell.get('cell_value1')
if cell is not None:
imports.append(cell)
for item in soup.findAll('m1_month_group', attrs={'forecast_month1': 'Nov'}):
for cell in item.findAll('cell'):
cell = cell.get('cell_value1')
if cell is not None:
nov.append(cell)
for item in soup.findAll('m1_attribute_group'):
for cell in item.findAll('cell', attrs={'cell_value1': True}):
cell = cell.get('cell_value1')
if cell is not None:
stock.append(cell)
for a, b, c, d in zip(production, imports, nov, stock):
print(
"Production: {:<10} Imports: {:<10} Nov: {:<10} Stock: {:<10}".format(a, b, c, d))
Output:
Production: 762.31 Imports: 180.03 Nov: 2658.19 Stock: 2618.41
Production: 627.98 Imports: 176.09 Nov: 3456.69 Stock: 3416.96
Production: 47.38 Imports: 4.30 Nov: 428.02 Stock: 414.83
Production: 714.93 Imports: 175.72 Nov: 2664.54 Stock: 2600.42
Production: 333.09 Imports: 6.96 Nov: 792.15 Stock: 816.54
Production: 18.50 Imports: 0.01 Nov: 765.55 Stock: 2629.43
Production: 20.94 Imports: 0.16 Nov: 1043.45 Stock: 3445.97
Production: 30.38 Imports: 0.45 Nov: 180.68 Stock: 428.40
Production: 151.13 Imports: 5.82 Nov: 755.17 Stock: 2647.47
Production: 85.17 Imports: 0.47 Nov: 288.28 Stock: 798.49
Production: 26.98 Imports: 0.06 Nov: 1394.89 Stock: 2659.68
Production: 202.93 Imports: 115.67 Nov: 1742.19 Stock: 3460.07
Production: 1.15 Imports: 6.15 Nov: 201.56 Stock: 426.56
Production: 4.26 Imports: 7.02 Nov: 1415.37 Stock: 2663.17
Production: 134.33 Imports: 3.94 Nov: 326.83 Stock: 796.90
Production: 0.97 Imports: 5.88 Nov: 497.76 Stock: 2658.19
Production: 19.25 Imports: 27.45 Nov: 671.05 Stock: 3456.69
Production: 0.07 Imports: 5.16 Nov: 45.78 Stock: 428.02
Production: 18.36 Imports: 18.46 Nov: 494.01 Stock: 2664.54
Production: 0.00 Imports: 26.06 Nov: 177.04 Stock: 792.15
Production: 98.51 Imports: 1.17 Nov: 2,239.15 Stock: 762.31
Production: 14.80 Imports: 0.10 Nov: 2,941.47 Stock: 1025.03
Production: 765.23 Imports: 175.68 Nov: 349.26 Stock: 182.47
Production: 765.55 Imports: 176.53 Nov: 2,309.22 Stock: 742.31
Production: 633.23 Imports: 172.48 Nov: 711.00 Stock: 282.73
Production: 633.55 Imports: 173.33 Nov: 713.29 Stock: 731.28
Production: 53.39 Imports: 3.27 Nov: 958.54 Stock: 1014.01
Production: 52.26 Imports: 3.27 Nov: 154.82 Stock: 172.89
Production: 711.84 Imports: 172.41 Nov: 723.71 Stock: 736.11
Production: 713.29 Imports: 173.27 Nov: 260.68 Stock: 277.90
Production: 324.70 Imports: 6.66 Nov: 1,034.06 Stock: 765.23
Production: 326.20 Imports: 6.66 Nov: 1,320.21 Stock: 1042.91
Production: 20.50 Imports: 0.01 Nov: 151.67 Stock: 179.68
Production: 20.00 Imports: 0.01 Nov: 1,095.67 Stock: 755.11
Production: 18.00 Imports: 0.15 Nov: 274.43 Stock: 287.80
Production: 17.20 Imports: 0.15 Nov: 491.79 Stock: 765.55
Production: 33.00 Imports: 0.45 Nov: 662.72 Stock: 1043.45
Production: 33.00 Imports: 0.45 Nov: 42.76 Stock: 180.68
Production: 152.00 Imports: 5.50 Nov: 489.85 Stock: 755.17
Production: 153.00 Imports: 5.50 Nov: 175.89 Stock: 288.28
Production: 72.50 Imports: 0.48 Nov: 571.30 Stock: 1361.22
Production: 74.00 Imports: 0.48 Nov: 699.06 Stock: 1747.20
Production: 28.70 Imports: 0.08 Nov: 172.55 Stock: 185.23
Production: 29.00 Imports: 0.08 Nov: 494.66 Stock: 1375.95
Production: 203.07 Imports: 114.99 Nov: 109.99 Stock: 371.25
Production: 203.29 Imports: 115.49 Nov: 337.76 Stock: 1398.80
Production: 1.10 Imports: 6.00 Nov: 352.64 Stock: 1770.05
Production: 1.10 Imports: 6.00 Nov: 93.64 Stock: 211.36
Production: 5.30 Imports: 7.70 Nov: 336.39 Stock: 1422.75
Production: 5.30 Imports: 7.70 Nov: 13.79 Stock: 347.30
Production: 132.00 Imports: 3.20 Nov: 206.47 Stock: 1396.68
Production: 132.00 Imports: 3.20 Nov: 227.55 Stock: 1747.53
Production: 0.95 Imports: 5.90 Nov: 88.69 Stock: 201.01
Production: 0.95 Imports: 5.90 Nov: 204.54 Stock: 1413.52
Production: 18.87 Imports: 27.40 Nov: 19.09 Stock: 334.01
Production: 18.87 Imports: 27.40 Nov: 45.2 Stock: 1394.89
Production: 0.06 Imports: 5.10 Nov: 37.2 Stock: 1742.19
Production: 0.06 Imports: 5.10 Nov: 51.7 Stock: 201.56
Production: 22.46 Imports: 17.29 Nov: 1,080 Stock: 1415.37
Production: 22.68 Imports: 17.09 Nov: 1,920 Stock: 326.83
Production: 0.00 Imports: 27.10 Nov: 120 Stock: 494.88
Production: 0.00 Imports: 27.10 Nov: 3,120 Stock: 644.73
Production: 102.19 Imports: 0.02 Nov: 955 Stock: 47.13
Production: 102.19 Imports: 0.02 Nov: 61 Stock: 482.17
Production: 11.50 Imports: 0.06 Nov: 140 Stock: 162.56
Production: 11.50 Imports: 0.06 Nov: 1,156 Stock: 499.34
Production: 1,361.22 Imports: 187.74 Nov: 950 Stock: 661.90
Production: 1,095.50 Imports: 171.31 Nov: 2,106 Stock: 44.15
Production: 384.39 Imports: 2.93 Nov: 1,014 Stock: 488.61
Production: 976.83 Imports: 184.81 Nov: 4.60 Stock: 173.29
Production: 252.15 Imports: 3.69 Nov: 100.7 Stock: 497.77
Production: 39.32 Imports: 0.01 Nov: 89.5 Stock: 669.62
Production: 12.19 Imports: 0.00 Nov: 4.03 Stock: 45.88
Production: 85.06 Imports: 1.46 Nov: 57.8 Stock: 494.54
Production: 26.24 Imports: 1.77 Nov: 360.6 Stock: 175.09
Production: 41.72 Imports: 0.13 Nov: 3.1 Stock: 497.76
Production: 34.07 Imports: 0.05 Nov: 421.4 Stock: 671.05
Production: 245.11 Imports: 133.71 Nov: 139.7 Stock: 45.78
Production: 152.12 Imports: 19.40 Nov: 179.5 Stock: 494.01
Production: 0.19 Imports: 17.58 Nov: 319.1 Stock: 177.04
Production: 33.19 Imports: 16.36 Nov: 49.9 Stock: 2,180.98
Production: 29.07 Imports: 39.13 Nov: 369.0 Stock: 2,875.67
Production: 0.26 Imports: 11.99 Nov: 52.4 Stock: 320.25
Production: 30.00 Imports: 14.54 Nov: 5.3 Stock: 2,242.47
Production: 0.20 Imports: 10.19 Nov: 4.7 Stock: 727.78
Production: 265.72 Imports: 16.43 Nov: 75.9 Stock: 2,191.07
Production: 1,396.68 Imports: 199.73 Nov: 64 Stock: 2,911.62
Production: 1,394.89 Imports: 201.43 Nov: 358 Stock: 345.01
Production: 1,134.21 Imports: 185.33 Nov: 0 Stock: 2,285.10
Production: 1,132.41 Imports: 187.03 Nov: 421 Stock: 709.92
Production: 363.65 Imports: 3.43 Nov: 160 Stock: 2,236.65
Production: 360.82 Imports: 3.39 Nov: 100 Stock: 2,940.82
Production: 1,033.03 Imports: 196.30 Nov: 260 Stock: 346.78
Production: 1,034.06 Imports: 198.05 Nov: 110 Stock: 2,306.25
Production: 302.69 Imports: 3.03 Nov: 370 Stock: 714.34
Production: 302.48 Imports: 3.05 Nov: 51 Stock: 2,239.15
Production: 57.73 Imports: 0.01 Nov: 3.40 Stock: 2,941.47
Production: 57.63 Imports: 0.01 Nov: 277.90 Stock: 349.26
Production: 11.50 Imports: 0.00 Nov: 765.55 Stock: 2,309.22
Production: 11.00 Imports: 0.00 Nov: 176.53 Stock: 711.00
Production: 104.23 Imports: 1.70 Nov: 148.47 Stock: 714.93
Production: 104.35 Imports: 1.70 Nov: 755.17 Stock: 941.22
Production: 28.56 Imports: 1.03 Nov: 180.68 Stock: 157.81
Production: 28.56 Imports: 1.03 Nov: 288.28 Stock: 713.06
Production: 39.88 Imports: 0.09 Nov: 138.13 Stock: 252.82
Production: 40.38 Imports: 0.09 Nov: 633.55 Stock: 679.98
Production: 46.20 Imports: 0.03 Nov: 173.33 Stock: 929.12
Production: 46.04 Imports: 0.03 Nov: 127.47 Stock: 147.41
Production: 256.34 Imports: 144.88 Nov: 627.17 Stock: 706.09
Production: 253.80 Imports: 146.05 Nov: 179.38 Stock: 248.51
Production: 158.75 Imports: 22.01 Nov: 142.61 Stock: 711.84
Production: 158.55 Imports: 22.01 Nov: 29.39 Stock: 956.87
Production: 0.18 Imports: 17.37 Nov: 52.26 Stock: 153.82
Production: 0.18 Imports: 17.77 Nov: 3.27 Stock: 723.32
Production: 33.10 Imports: 18.01 Nov: 3.81 Stock: 259.40
Production: 30.60 Imports: 18.16 Nov: 31.46 Stock: 713.29
Production: 32.81 Imports: 42.92 Nov: 25.86 Stock: 958.54
Production: 32.96 Imports: 42.24 Nov: 27.59 Stock: 154.82
Production: 0.25 Imports: 12.51 Nov: 248.51 Stock: 723.71
Production: 0.27 Imports: 12.51 Nov: 713.29 Stock: 260.68
Production: 30.96 Imports: 17.26 Nov: 173.27 Stock: 976.83
Production: 30.96 Imports: 18.26 Nov: 144.66 Stock: 1,297.72
Production: 0.20 Imports: 10.61 Nov: 723.71 Stock: 118.08
Production: 0.20 Imports: 10.91 Nov: 154.82 Stock: 1,051.55
Production: 262.48 Imports: 14.40 Nov: 260.68 Stock: 313.33
Production: 262.48 Imports: 14.40 Nov: 32.21 Stock: 1,018.87
Production: 1,080.02 Imports: 149.96 Nov: 326.20 Stock: 1,329.57
Production: 820.94 Imports: 146.50 Nov: 6.66 Stock: 156.41
Production: 371.10 Imports: 0.92 Nov: 85.85 Stock: 1,094.98
Production: 708.92 Imports: 149.05 Nov: 200.45 Stock: 289.54
Production: 164.42 Imports: 1.18 Nov: 131.00 Stock: 1,033.03
Production: 32.00 Imports: 0.01 Nov: 33.62 Stock: 1,322.68
Production: 82.00 Imports: 0.92 Nov: 1.93 Stock: 150.10
Production: 13.20 Imports: 0.05 Nov: 20.00 Stock: 1,092.55
Production: 13.10 Imports: 0.17 Nov: 0.01 Stock: 281.03
Production: 24.12 Imports: 0.04 Nov: 0.05 Stock: 1,034.06
Production: 126.08 Imports: 88.35 Nov: 6.05 Stock: 1,320.21
Production: 6.40 Imports: 9.46 Nov: 14.00 Stock: 151.67
Production: 62.02 Imports: 18.47 Nov: 1.89 Stock: 1,095.67
Production: 0.00 Imports: 15.67 Nov: 4.96 Stock: 274.43
Production: 27.57 Imports: 16.13 Nov: 17.20 Stock: 489.23
Production: 29.95 Imports: 14.20 Nov: 0.15 Stock: 636.73
Production: 0.07 Imports: 10.02 Nov: 5.00 Stock: 44.37
Production: Imports: Nov: 8.50 Stock: 477.87
Production: 14.10 Imports: 1.70 Nov: 9.00 Stock: 161.63
Production: 259.07 Imports: 3.46 Nov: 4.81 Stock: 492.22
Production: 1,104.01 Imports: 166.09 Nov: 5.92 Stock: 652.93
Production: 1,102.16 Imports: 167.44 Nov: 33.00 Stock: 41.18
Production: 850.01 Imports: 159.09 Nov: 0.45 Stock: 484.03
Production: 848.16 Imports: 160.44 Nov: 4.50 Stock: 171.87
Production: 350.01 Imports: 1.27 Nov: 9.50 Stock: 491.78
Production: 347.01 Imports: 1.27 Nov: 24.50 Stock: 661.27
Production: 754.00 Imports: 164.82 Nov: 5.37 Stock: 42.86
Production: 755.16 Imports: 166.17 Nov: 10.00 Stock: 490.38
Production: 214.50 Imports: 1.17 Nov: 153.00 Stock: 173.91
Production: 214.50 Imports: 1.17 Nov: 5.50 Stock: 491.79
Production: 50.00 Imports: 0.01 Nov: 56.00 Stock: 662.72
Production: 50.00 Imports: 0.01 Nov: 127.50 Stock: 42.76
Production: 101.00 Imports: 1.00 Nov: 29.00 Stock: 489.85
Production: 101.00 Imports: 1.00 Nov: 12.00 Stock: 175.89
Production: 13.50 Imports: 0.04 Nov: 7.82 Stock: 580.77
Production: 14.00 Imports: 0.04 Nov: 74.00 Stock: 690.45
Production: 14.00 Imports: 0.10 Nov: 0.48 Stock: 176.30
Production: 14.00 Imports: 0.10 Nov: 17.00 Stock: 484.13
Production: 36.00 Imports: 0.02 Nov: 39.50 Stock: 116.52
Production: 35.50 Imports: 0.02 Nov: 34.50 Stock: 596.69
Production: 129.27 Imports: 95.80 Nov: 8.29 Stock: 713.20
Production: 127.03 Imports: 97.20 Nov: 1.59 Stock: 171.18
Production: 6.40 Imports: 10.20 Nov: 29.00 Stock: 489.90
Production: 6.40 Imports: 9.90 Nov: 0.08 Stock: 127.77
Production: 64.80 Imports: 21.00 Nov: 3.30 Stock: 574.85
Production: 64.56 Imports: 21.00 Nov: 9.40 Stock: 703.02
Production: 0.00 Imports: 15.60 Nov: 20.00 Stock: 172.38
Production: 0.00 Imports: 16.00 Nov: 1.26 Stock: 498.21
Production: 27.00 Imports: 17.50 Nov: 182.10 Stock: 109.75
Production: 25.00 Imports: 17.50 Nov: 203.29 Stock: 571.30
Production: 30.91 Imports: 16.90 Nov: 115.49 Stock: 699.06
Production: 30.91 Imports: 17.90 Nov: 39.66 Stock: 172.55
Production: 0.08 Imports: 10.50 Nov: 299.72 Stock: 494.66
Production: 0.08 Imports: 10.80 Nov: 13.05 Stock: 109.99
Production: 14.00 Imports: 1.00 Nov: 188.11 Stock: 331.71
Production: 14.00 Imports: 1.00 Nov: 0.94 Stock: 349.08
Production: 254.00 Imports: 7.00 Nov: 1.10 Stock: 89.12
Production: 254.00 Imports: 7.00 Nov: 6.00 Stock: 328.60
Production: 494.88 Imports: 46.70 Nov: 0.30 Stock: 16.56
Production: 346.01 Imports: 41.20 Nov: 7.20 Stock: 333.79
Production: 5.66 Imports: 0.87 Nov: 0.00 Stock: 350.34
Production: 489.23 Imports: 45.83 Nov: 0.84 Stock: 93.42
Production: 181.69 Imports: 0.76 Nov: 1.06 Stock: 331.69
Production: 13.20 Imports: 0.01 Nov: 5.30 Stock: 14.88
Production: 112.76 Imports: 0.00 Nov: 7.70 Stock: 340.32
Production: 7.50 Imports: 0.00 Nov: 0.50 Stock: 355.04
Production: 20.58 Imports: 0.25 Nov: 12.20 Stock: 94.21
Production: 27.66 Imports: 0.50 Nov: 0.60 Stock: 338.20
Production: 241.09 Imports: 21.35 Nov: 1.26 Stock: 14.27
Production: 148.87 Imports: 5.50 Nov: 139.77 Stock: 337.76
Production: 2.01 Imports: 2.01 Nov: 132.00 Stock: 352.64
Production: 37.00 Imports: 2.35 Nov: 3.20 Stock: 93.64
Production: 4.73 Imports: 2.00 Nov: 21.00 Stock: 336.39
Production: 12.24 Imports: 1.30 Nov: 128.00 Stock: 13.79
Production: 2.22 Imports: 3.62 Nov: 1.30 Stock: 198.67
Production: Imports: Nov: 145.67 Stock: 219.33
Production: 8.20 Imports: 0.56 Nov: 0.97 Stock: 79.87
Production: 1.64 Imports: 1.66 Nov: 0.95 Stock: 192.25
Production: 4.30 Imports: 0.09 Nov: 5.90 Stock: 22.22
Production: 7.79 Imports: 0.69 Nov: 0.70 Stock: 203.24
Production: 0.18 Imports: 0.81 Nov: 6.50 Stock: 225.46
Production: 3.97 Imports: 0.40 Nov: 0.28 Stock: 86.19
Production: 497.77 Imports: 43.85 Nov: 1.04 Stock: 200.14
Production: 497.76 Imports: 43.35 Nov: 15.46 Stock: 21.09
Production: 351.77 Imports: 40.75 Nov: 18.87 Stock: 207.50
Production: 351.76 Imports: 40.85 Nov: 27.40 Stock: 228.78
Production: 5.99 Imports: 0.94 Nov: 2.25 Stock: 88.61
Production: 5.97 Imports: 0.94 Nov: 46.75 Stock: 205.21
Production: 491.78 Imports: 42.91 Nov: 0.89 Stock: 19.66
Production: 491.79 Imports: 42.41 Nov: 14.09 Stock: 206.47
Production: 183.60 Imports: 0.66 Nov: 0.20 Stock: 227.55
Production: 184.60 Imports: 0.66 Nov: 0.06 Stock: 88.69
Production: 13.30 Imports: 0.01 Nov: 5.10 Stock: 204.54
Production: 13.30 Imports: 0.01 Nov: 0.05 Stock: 19.09
Production: 114.00 Imports: 0.00 Nov: 4.76 Stock: 46.1
Production: 115.00 Imports: 0.00 Nov: 0.40 Stock: 47.8
Production: 7.50 Imports: 0.00 Nov: 0.20 Stock: 45.2
Production: 7.50 Imports: 0.00 Nov: 11.77 Stock: 45.2
Production: 20.50 Imports: 0.25 Nov: 22.68 Stock: 37.6
Production: 20.50 Imports: 0.25 Nov: 17.09 Stock: 39.6
Production: 28.30 Imports: 0.40 Nov: 3.92 Stock: 38.1
Production: 28.30 Imports: 0.40 Nov: 39.04 Stock: 37.2
Production: 241.70 Imports: 16.03 Nov: 0.72 Stock: 46.4
Production: 240.60 Imports: 15.43 Nov: 11.78 Stock: 47.6
Production: 146.00 Imports: 3.10 Nov: 4.92 Stock: 51.6
Production: 146.00 Imports: 2.50 Nov: 0.00 Stock: 51.7
Production: 2.01 Imports: 2.05 Nov: 27.10 Stock: 1,181
Production: 2.01 Imports: 2.05 Nov: 8.04 Stock: 1,099
Production: 37.40 Imports: 0.60 Nov: 25.82 Stock: 1,080
Production: 36.50 Imports: 1.40 Nov: 1.07 Stock: 1,080
Production: 4.90 Imports: 2.20 Nov: 5.14 Stock: 1,741
Production: 4.90 Imports: 1.80 Nov: 16.99 Stock: 1,885
Production: 12.20 Imports: 2.70 Nov: 102.19 Stock: 1,962
Production: 12.00 Imports: 2.50 Nov: 0.02 Stock: 1,920
Production: 2.29 Imports: 3.68 Nov: 6.00 Stock: 158
Production: 2.29 Imports: 3.68 Nov: 98.00 Stock: 135
Production: 7.14 Imports: 1.10 Nov: 0.50 Stock: 120
Production: 7.14 Imports: 1.10 Nov: 20.70 Stock: 120
Production: 1.55 Imports: 1.90 Nov: 1.68 Stock: 3,080
Production: 1.55 Imports: 2.00 Nov: 11.50 Stock: 3,119
Production: 4.30 Imports: 0.30 Nov: 0.06 Stock: 3,161
Production: 4.30 Imports: 0.30 Nov: 1.80 Stock: 3,120
Production: 7.80 Imports: 0.69 Nov: 6.60 Stock: 964
Production: 7.80 Imports: 0.69 Nov: 5.20 Stock: 955
Production: 0.20 Imports: 0.79 Nov: 1.44 Stock: 960
Production: 0.20 Imports: 0.79 Nov: 347.30 Stock: 955
Production: 3.88 Imports: 0.41 Nov: 1,394.89 Stock: 63
Production: 3.88 Imports: 0.41 Nov: 201.43 Stock: 59
Upvotes: 0
balderman
Reputation: 23815
The code below uses python ElementTree lib to read 'Data1' and 'Data3'.
since the XML is very big I am not sure those results are correct but if it is - you can extend it and use it for 'Data2' and 'Data4',
import requests
import xml.etree.ElementTree as ET
xml = requests.get('https://www.usda.gov/oce/commodity/wasde/wasde1119.xml').content.decode('utf-8')
root = ET.fromstring(xml)
# Data1
cells = root.findall(".//sr19//matrix1//m1_month_group[@forecast_month1='Oct']//m1_attribute_group[@attribute1='Production']//Cell")
values = [cell.attrib['cell_value1'] for cell in cells]
print(values)
# Data3
cells = root.findall(".//sr11//matrix1//attribute1[@attribute1='Yield per Harvested Acre']//m1_month_group[@forecast_month1='Nov']//Cell")
values = [cell.attrib['cell_value1'] for cell in cells]
print(values)
output
['765.23', '633.23', '53.39', '711.84', '324.70', '20.50', '18.00', '33.00', '152.00', '72.50', '28.70', '203.07', '1.10', '5.30', '132.00', '0.95', '18.87', '0.06', '22.46', '0.00', '102.19', '11.50']
['51.7']
Upvotes: 0
Related Questions
- Web scrapping XML page but structure is incorrect using BeautifulSoup
- Problem with find a specific part of XML file in Python
- Python BeautifulSoup XML Parsing
- Python BeautifulSoup Won't Return Tag from XML
- XML not parsing as expected with BeautifulSoup
- BeautifulSoup not finding xml tag, how do i fix this?
- Beautifulsoup xml parsing failure: find all returns only one result
- Python xml parsing with beautifulsoup
- BeautifulSoup can't find tags inside XML block
- Trouble parsing XML with python