CODEWITHSUNDEEP
bashcsvunix
quenchlemon
quenchlemon

Reputation: 41

Using bash to extract numbers and convert to CSV file

I am quite new in using bash to extract but I am not what search terms to look for my problem. I like to extract data for some variables from a very large log file.

Sample of logfile

temp[min,max]=[   24.0000000000000      ..   834.230000000000      ]
CHANGE working on TEMS
RMS(TEMS)=  6.425061887244621E-002   DIFMAX:   0.896672707535103     
               765                     1                   171
CHANGE working on PHI 
RMS(PHI )=   1.92403467949391        DIFMAX:    62.3113693145351     
               765                     1                   170
CHANGE working on TEMP
RMS(TEMP)=  6.425061887244621E-002   DIFMAX:   0.896672707535103     
               765                     1                   171
PMONI working
TIMSTP working

COPEQE working : INFO
DELT =    630720000.000000      sec       

Courant-Number in x,y,z:
Max. :   5.05    ,      0.00    ,      6.93    
Min. :   0.00    ,      0.00    ,      0.00    
Avg. :  0.568E-02,      0.00    ,     0.383    
PROBLEM: Courant-Number(s) greater than 1 :   11.9802093558263     
max. TEMP-Peclet in X:                     653                     1
               170
max. TEMP-Peclet in Y:                     653                     1
               170
Temperature-Peclet-Number in x,y,z:
Max. :  0.357    ,      0.00    ,     0.313E-01
Min. :   0.00    ,      0.00    ,      0.00    
Avg. :  0.307E-03,      0.00    ,     0.435E-03
Temperature-Neumann-Number in x,y,z:
Max.:   64.9    ,    64.9    ,    64.9    
Min.:  0.619E-02,   0.619E-02,   0.619E-02
Avg.:   35.5    ,    35.5    ,    35.5    
PROBLEM: Temp-Neumann-Number greater than 0.5 :   194.710793368065     
(Dominating: Courant-Number)
DRUCK working
KOPPX working
#########################################################################
STRESS PERIOD:                      1                        1   
                 1  of                    100   <<<<<
Time Step:      50 (  1.0% of 0.315E+13 sec       )(0.631E+09 sec       )
#########################################################################

### Continues on ###

I managed to extract the lines relating to the variables I am looking for using bash.

grep -A 3 'Courant-Number in x,y,z' logfile.log > courant.txt
grep -A 2 'Max.' courant.txt > courant.txt

to get this...

  Max. :  0.146E+04,      0.00    ,     0.169E+04
  Min. :   0.00    ,      0.00    ,      0.00    
  Avg. :   1.15    ,      0.00    ,     0.986    
--
  Max. :  0.184E+04,      0.00    ,     0.175E+04
  Min. :   0.00    ,      0.00    ,      0.00    
  Avg. :   1.13    ,      0.00    ,      1.05    
--
  Max. :  0.163E+04,      0.00    ,     0.172E+04
  Min. :   0.00    ,      0.00    ,      0.00    
  Avg. :   1.13    ,      0.00    ,      1.17

I would like to convert this data to a CSV file with the following columns, thus making a total of 9 columns.

Max_x | Max_y | Max_z | Min_x | Min_y | Min_z | Avg_x | Avg_y | Avg_z

I would like to continue to use bash to get this data. Any inputs will be most appreciated.

Thanks!

Upvotes: 3

Views: 260

Answers (1)

22degrees
22degrees

Reputation: 645

You've got a good start. I had a much worse solution a bit earlier, but then I learned about paste -d. 

grep -A 3 'Courant-Number in x,y,z' logfile.log |
    grep -A 2 'Max.' |
    grep -v -- '--' |
    sed 's/^.*://' |
    paste -d "," - - - |
    sed 's/ *//g'
  • find courant number + 3 lines
  • find max + 2 following lines
  • get rid of lines that have '--'
  • get rid of min: max: avg:
  • join every three lines with commas
  • get rid of whitespace

Upvotes: 4

Related Questions