CODEWITHSUNDEEP
python-3.xpandasnumpy
iceboing
iceboing

Reputation: 13

Reading data from csv file with different number of columns and mixed datatype - python

i've got an CSV-file with the following input:

Title;High Data

Date save;01.01.2000;00:00

Comment;

Magnification;1;[m]

Counts;4931

Length;5583;[m]

Start 1;0;1475

End 1;4931;1475

Profil 1[µm]

529

528

and so on ...

I want to read the counts and length into variables. The problem seems to be that there are different numbers of columns. I've tried different things to load it into an numpy array or an pandas dataframe but nothing really worked out. Please help me! Thank you!

Edit: This is the code i used to load it into the pandas dataframe:

fin = pd.read_csv('Temp.csv', sep = ';')
df = pd.DataFrame(fin)

But after that i can't read the data out of the dataframe...

Upvotes: 1

Views: 1248

Answers (1)

jfbeltran
jfbeltran

Reputation: 1818

This is not really a CSV file. If you want to parse a file into a pandas dataframe, you usually want to be looking at something like a table (for example: each column is one feature, each row is one sample/item/person).

It seems like you have three types of data annotation formats here:

1

[NAME OF FEATURE];[VALUE FOR FEATURE]

2

[NAME OF FEATURE];[VALUE FOR FEATURE];[UNIT]

3

[NAME OF FEATURE][UNIT]
[VALUE]
[VALUE]
[VALUE]

I would recommend writing a parser yourself if there are a lot of these files, or reshaping the data to be in a consistent format if it is just a single file. Preferably something like:

Title, Date Save, Comment, Magnification, ..., Profil 1, ...
High Data, 01.01.2000;00:00,,1,...,"529,528",...
...

EDIT: If you only care about count and length.

I want to read the counts and length into variables. 

relevant_lines = [line.split(';')[:1] for line in open(your_file_name).read().split('\n') if line.startswith('Counts') or line.startswith('Length')]
df = pd.DataFrame([dict(relevant_lines)])

Makes a df that looks like this:

  Count Length
0  4931  5583

EVEN SIMPLER EDIT: If you just want them as variables and don't care about the dataframe at all:

lines = open(your_file).read().split('\n')
count = None
length = None
for line in lines:
    if(line.startswith('Length')):
        length=int(line.split(';')[1])
    if(line.startswith('Count')):
        count=int(line.split(';')[1])

Upvotes: 1

Related Questions