CODEWITHSUNDEEP
pythontextformatblock
akashwani
akashwani

Reputation: 105

Read, Slice and Re-structure data file block-by-block in Python

A text file generated by a Fortran program contains "blocks" data, that need to be reformatted (Python script).

Each "block" of data in this file corresponds to the "Time:" specified in the beginning of the block. All "blocks" have a fixed size and structure.

I need to extract the data from "Head" and "Moisture" columns corresponding to different "Depths" (0, -1, and -2) for each "Time:".

Note: The header at the beginning is not part of the repeating "blocks" of data.

Sample input file:

 ******* Program Simulation
 ******* 
 This is initial header information for Simulation                              
 Date:   1. 6.    Time:  15: 3:39
 Units: L = cm   , T = min  , M = mmol 

 Time:        0.0000

 Node      Depth      Head Moisture       K         
           [L]        [L]    [-]        [L/T]      

   1     0.0000     -37.743 0.0630   0.5090E-05  
   2    -1.0000     -36.123 0.0750   0.5090E-05  
   3    -2.0000     -33.002 0.0830   0.5090E-05  
end

 Time:      360.0000

 Node      Depth      Head Moisture       K         
           [L]        [L]    [-]        [L/T]     

   1     0.0000 -0.1000E+07 0.0450   0.1941E-32  
   2    -1.0000    -253.971 0.0457   0.4376E-10  
   3    -2.0000     -64.510 0.0525   0.2264E-06  
end

 Time:      720.0000

 Node      Depth      Head Moisture       K         
           [L]        [L]    [-]        [L/T]     

   1     0.0000 -0.1000E+07 0.0550   0.1941E-32  
   2    -1.0000    -282.591 0.0456   0.2613E-10 
   3    -2.0000     -71.829 0.0513   0.1229E-06  
end

Desired output:

Time        Head(Depth=0)   Head(Depth=-1)  Head(Depth=-2)  Moisture(Depth=0)   Moisture(Depth=-1)  Moisture(Depth=-2)
0.0000      -37.743         -36.123         -33.002         0.0630              0.0750              0.0830
360.0000    -0.1000E+07     -253.971        -64.510         0.0450              0.0457              0.0525
720.0000    -0.1000E+07     -282.591        -71.829         0.0550              0.0456              0.0513

How I read the input file block-by-block from each "Time:" to "end" keywords and reformat to the desired output?

Upvotes: 2

Views: 1702

Answers (3)

Hugh Bothwell
Hugh Bothwell

Reputation: 56654

Edit: I have made a couple of changes so it actually runs.

from itertools import chain

def get_lines(f, n=1):
    return [f.next() for i in xrange(n)]

class BlockReader(object):
    def __init__(self, f, n=1):
        self.f = f
        self.n = n
    def __iter__(self):
        return self
    def next(self):
        return [self.f.next() for i in xrange(self.n)]

fmt = "{:<12}" + "{:<16}"*6 + "\n"
cols = [
    "Time",
    "Head(Depth=0)",
    "Head(Depth=-1)",
    "Head(Depth=-2)",
    "Moisture(Depth=0)",
    "Moisture(Depth=-1)",
    "Moisture(Depth=-2)"
]

def main():
    with open("simulation.txt") as inf, open("result.txt","w") as outf:
        # throw away input header
        get_lines(inf, 5)
        # write output header
        outf.write(fmt.format(*cols))

        # read input file in ten-line chunks
        for block in BlockReader(inf, 10):
            # grab time value
            time = float(block[1].split()[1])

            # grab head and moisture columns
            data = (line.split()[2:4] for line in block[6:9])
            values = (map(float,dat) for dat in data)
            h,m = zip(*values)

            # write data to output file
            outf.write(fmt.format(*chain([time],h,m)))

if __name__=="__main__":
    main()

Output is

Time        Head(Depth=0)   Head(Depth=-1)  Head(Depth=-2)  Moisture(Depth=0)Moisture(Depth=-1)Moisture(Depth=-2)
0.0         -37.743         -36.123         -33.002         0.063           0.075           0.083           
360.0       -1000000.0      -253.971        -64.51          0.045           0.0457          0.0525          
720.0       -1000000.0      -282.591        -71.829         0.055           0.0456          0.0513

Upvotes: 1

georg
georg

Reputation: 214969

Here's the parsing part:

import re

data = []

with open(xxxx) as f:
    for line in f:
        m = re.match(r'^\s+Time:\s+([\d.]+)', line)
        if m:
            data.append([float(m.group(1))])
        elif re.match(r'^\s+\d+', line):
            data[-1].append(map(float, line.strip().split()))

produces:

[[0.0,
  [1.0, 0.0, -37.743, 0.063, 5.09e-06],
  [2.0, -1.0, -36.123, 0.075, 5.09e-06],
  [3.0, -2.0, -33.002, 0.083, 5.09e-06]],
 [360.0,
  [1.0, 0.0, -1000000.0, 0.045, 1.941e-33],
  [2.0, -1.0, -253.971, 0.0457, 4.376e-11],
  [3.0, -2.0, -64.51, 0.0525, 2.264e-07]],
 [720.0,
  [1.0, 0.0, -1000000.0, 0.055, 1.941e-33],
  [2.0, -1.0, -282.591, 0.0456, 2.613e-11],
  [3.0, -2.0, -71.829, 0.0513, 1.229e-07]]]

it should be easy to print the desired table from this.

Upvotes: 1

Justin Blank
Justin Blank

Reputation: 1878

If the file isn't too large, you can do:

f = open('somefile')
file = f.read()
blocks = file.split('Time:')[1:]

Upvotes: 0

Related Questions