CODEWITHSUNDEEP
pythonarraysfile
Emotional Damage
Emotional Damage

Reputation: 148

Open multiple files and store the data in arrays in python

import numpy as np
import matplotlib.pyplot as plt
import csv

file = 'random_file'
with open(file + '.txt') as datafile:
    for line in range(7):
        next(datafile)
    for line in datafile:
        lines = datafile.readlines()
        x0_1  = [float(line.split()[0]) for line in lines]
        y1_1  = [float(line.split()[1]) for line in lines]
        Z_1   = [float(line.split()[2]) for line in lines]
        u_1   = [float(line.split()[3]) for line in lines]
        v_1   = [float(line.split()[4]) for line in lines]
        V0_1  = [float(line.split()[5]) for line in lines]
        V1_1  = [float(line.split()[6]) for line in lines]
        Yf_1  = [float(line.split()[7]) for line in lines]
        Yo_1  = [float(line.split()[8]) for line in lines]
        T_1   = [float(line.split()[9]) for line in lines]
        Yp0_1 = [float(line.split()[10]) for line in lines]
        Yp1_1 = [float(line.split()[11]) for line in lines]
        D_1   = [float(line.split()[12]) for line in lines]
        rho_1 = [float(line.split()[13]) for line in lines]
#Do something with the data

Currently, I have this code to open a single file, skip the first seven lines in the file and then store the data in the file to arrays. But I have multiple files that I would like to read and store the data in the file to arrays in a similar fashion. As of now, I am just repeating the above code for different files. How can I read all the files at once and store the data in arrays?

The data file looks something like this,

#First seven lines ignore #

  0.00000E+00  5.00000E-04  9.28613E-01  6.58135E-02  4.05154E-01  6.58135E-02  4.05154E-01  9.24508E-01  2.01228E-16  4.44076E+02  1.12880E-02  9.23567E-03  2.46667E-05  4.53488E-01
  0.00000E+00  1.50000E-03  8.14544E-01  9.64630E-02  3.82685E-01  9.64630E-02  3.82685E-01  8.03881E-01  0.00000E+00  6.74294E+02  2.93252E-02  2.39933E-02  2.46667E-05  3.15181E-01
  0.00000E+00  2.50000E-03  5.46555E-01  1.40368E-01  1.23704E-01  1.40368E-01  1.23704E-01  5.20482E-01  0.00000E+00  1.21516E+03  7.17009E-02  5.86644E-02  2.46667E-05  2.01024E-01
  0.00000E+00  3.50000E-03  1.23207E-01  1.08260E-01 -2.56808E-01  1.08260E-01 -2.56808E-01  7.27916E-02  0.00000E+00  2.06958E+03  1.38643E-01  1.13435E-01  2.46667E-05  1.54495E-01

Upvotes: 1

Views: 1198

Answers (2)

Serge Ballesta
Serge Ballesta

Reputation: 148910

Should be a comment since it is not a direct answer, but formatting code in comment is not possible.

Beware: your code actually skips 8 lines and is terribly inefficient. The problem is here:

for line in range(7):
    next(datafile)               # ok, 7 lines skipped
for line in datafile:            # read 8th line 
    lines = datafile.readlines() # lines only starts at the 9th line

Additionally you repeat the splitting of the line for each field which is not efficient.

As you already import the csv module, you could do:

for line in range(7):
    next(datafile)               # ok, 7 lines skipped
x0_1, y1_1, Z_1, u_1, v_1, V0_1, V1_1, Yf_1, Yo_1, T_1, Yp0_1, Yp1_1, D_1, rho_1 = [
    [float(i) for i in row] for row in zip(*csv.reader(datafile, delimiter=' ',
                                                       skipinitialspace=True))][:14]

Upvotes: 0

Vosem
Vosem

Reputation: 106

You can create an array with all of the filenames and iterate through those. As long as the files have the data stored the same way and you want to read the same thing in them.

Something like this:

files = ['file1', 'file2', 'file3']
    
for file in files:
  with open(file + '.txt') as datafile:
  # Skip the first seven lines, Read the rest of the file

Upvotes: 1

Related Questions