CODEWITHSUNDEEP
pythonpandasimporttext-files
Emma
Emma

Reputation: 453

Importing an irregular sized text file

I would like to import a text file that has 11576 rows and 7 columns into a pandas dataframe and then reshape it so that it has 229 rows and 351 columns.

In the text file, every 34 lines or so, there are 2 whitespaces (i.e the 6th and 7 th column of that row have no values).

I want to slice the data at this point and assign all the previous data as being in row one but, I do not know how to cut the data at that point and place it all in one row.

I have tried pd.read_csv but I can`t make it into the dimensions that I need.

Any suggestions would be really great. Thanks Emma

Here is a link to the text file.

Upvotes: 2

Views: 69

Answers (1)

jezrael
jezrael

Reputation: 862911

I believe need read_csv for create DataFrame and then numpy.reshape with select all columns without 2 last:

df = pd.read_csv('fieldgen_out1.txt', skiprows=1, header=None, sep='\s+')
#print (df)

#[351 rows x 229 columns]
df = pd.DataFrame(df.values.reshape(351, -1)[:, :-2])
#print (df)

And then if necessary transpose:

#[229 rows x 351 columns]
df = pd.DataFrame(df.values.reshape(351, -1)[:, :-2].T)
#print (df)

Upvotes: 1

Related Questions