CODEWITHSUNDEEP
pythondataframe
litmus
litmus

Reputation: 107

Python: error reading and manipulating DataFrame data

I have a DataFrame variable called "obsData", which has the structure: enter image description here I then use this variable as an input to a code (with much help from Stackoverflow) that sorts all hourly data into one row for each day using:

f = obsData

data = {}

for line in f:
    if 'Date' not in line or 'Temp' not in line:
        k, v, = line.split()                       # split line in 2 parts, v and k
        temperature = v.split(';')[1]
        if k not in data:
            data[k] = [temperature]
        else:
            data[k].append(temperature)


for k, v in data.items():
    outPut = "{} ;{}".format(k, ";".join(v))

My issue it that the variable "line" never manages to get past the first row of the data in "obsData". It only manages to read 'Date' but not the second column 'Temp'. As a consequence the split function tries to split 'Date' but since its only one value I get the error:

ValueError: not enough values to unpack (expected 2, got 1)

I have tried to redefine "f" (i.e. "obsData") from a DataFrame into a ndarray or string using to make it easier for the code to work with the data:

f = f.values                                             # into ndarry                 

f = f.astype(str)                                        # into string try 1

f[['Date', 'Temp']] = f[['Date', 'Temp']].astype(str)    # into string try 2

But for some reason I don't understand I cant convert it. What am I doing wrong? Any help is much appreciated!

EDIT for clarification: I get the error at the line with

k, v, = line.split()

Upvotes: 0

Views: 52

Answers (1)

Kenan
Kenan

Reputation: 14094

When importing csv data it's best to use pandas

import pandas as pd

df = pd.read_csv('obsData.csv')

if you still need to loop check itertuples

Upvotes: 1

Related Questions