CODEWITHSUNDEEP
pythonpandascsvdataframe
Nic
Nic

Reputation: 186

Python pandas create datafrane from csv embeded within a web txt file

I am trying to import CSV formatted data to Pandas dataframe. The CSV data is located within a .txt file the is located at a web URL. The issue is that I only want to import a part (or parts) of the .txt file that is formatted as CSV (see image below). Essentially I need to skip the first 9 rows and then import rows 10-16 as CSV.

My code 

import csv
import pandas as pd
import io

url = "http://www.bom.gov.au/climate/averages/climatology/windroses/wr15/data/086282-3pmMonth.txt"
df = pd.read_csv(io.StringIO(url), skiprows = 9, sep =',', skipinitialspace = True)
df

I get a lengthy error msg that ultimately says "EmptyDataError: No columns to parse from file"

I have looked at similar examples Read .txt file with Python Pandas - strings and floats but this is different.

Snippet of .txt file with CSV part highlighted

Upvotes: 1

Views: 87

Answers (1)

bartonstanley
bartonstanley

Reputation: 1307

The code above attempts to read a CSV file from the URL itself rather than the text file fetched from that URL. To see what I mean take out the skiprows parameter and then show the data frame. You'll see this:

Empty DataFrame
Columns: [http://www.bom.gov.au/climate/averages/climatology/windroses/wr15/data/086282-3pmMonth.txt]
Index: []

Note that the columns are the URL itself.

Import requests (you may have to install it first) and then try this:

content = requests.get(url).content
df = pd.read_csv(io.StringIO(content.decode('utf-8')),skiprows=9)

Upvotes: 1

Related Questions