CODEWITHSUNDEEP
pythonpandas
Xin
Xin

Reputation: 674

How to use pd.read_csv() on this web page?

I am having difficulties using pd.read_csv() on the web page to use the "Download Data" button since I do not see the typical .zip or .csv at the end. What would be the correct url to use to directly download the data with pd.read_csv()?

Link:

https://climate.weather.gc.ca/climate_data/daily_data_e.html?hlyRange=2008-12-22%7C2020-05-24&dlyRange=1999-05-01%7C2020-05-24&mlyRange=2000-06-01%7C2007-11-01&StationID=27211&Prov=AB&urlExtension=_e.html&searchType=stnProx&optLimit=yearRange&StartYear=2000&EndYear=2020&selRowPerPage=25&Line=5&txtRadius=25&optProxType=city&selCity=51%7C2%7C114%7C4%7CCalgary&selPark=&txtCentralLatDeg=&txtCentralLatMin=0&txtCentralLatSec=0&txtCentralLongDeg=&txtCentralLongMin=0&txtCentralLongSec=0&txtLatDecDeg=&txtLongDecDeg=&timeframe=2&Day=24&Year=2019&Month=5#

Upvotes: 1

Views: 81

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195543

When you open Firefox developer tools -> Network tab, you will see the URL when you click the download button. (Chrome has something similar too)

import pandas as pd

url = 'https://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=27211&Year=2019&Month=5&Day=1&timeframe=2&submit=Download+Data'

df = pd.read_csv(url)
print(df)

Prints:

     Longitude (x)  Latitude (y)      Station Name  Climate ID   Date/Time  ...  Snow on Grnd Flag  Dir of Max Gust (10s deg)  Dir of Max Gust Flag  Spd of Max Gust (km/h)  Spd of Max Gust Flag
0           -114.0         51.11  CALGARY INT'L CS     3031094  2019-01-01  ...                NaN                       29.0                   NaN                    44.0                   NaN
1           -114.0         51.11  CALGARY INT'L CS     3031094  2019-01-02  ...                NaN                       27.0                   NaN                    70.0                   NaN
2           -114.0         51.11  CALGARY INT'L CS     3031094  2019-01-03  ...                NaN                       27.0                   NaN                    62.0                   NaN
3           -114.0         51.11  CALGARY INT'L CS     3031094  2019-01-04  ...                NaN                       23.0                   NaN                    66.0                   NaN
4           -114.0         51.11  CALGARY INT'L CS     3031094  2019-01-05  ...                NaN                        NaN                   NaN                     NaN                   NaN
..             ...           ...               ...         ...         ...  ...                ...                        ...                   ...                     ...                   ...
360         -114.0         51.11  CALGARY INT'L CS     3031094  2019-12-27  ...                NaN                       30.0                   NaN                    46.0                   NaN
361         -114.0         51.11  CALGARY INT'L CS     3031094  2019-12-28  ...                NaN                        NaN                   NaN                     NaN                   NaN
362         -114.0         51.11  CALGARY INT'L CS     3031094  2019-12-29  ...                NaN                        NaN                   NaN                     NaN                   NaN
363         -114.0         51.11  CALGARY INT'L CS     3031094  2019-12-30  ...                NaN                       27.0                   NaN                    50.0                   NaN
364         -114.0         51.11  CALGARY INT'L CS     3031094  2019-12-31  ...                NaN                       28.0                   NaN                    55.0                   NaN

[365 rows x 31 columns]

Upvotes: 2

Related Questions