CODEWITHSUNDEEP
bash
feeling
feeling

Reputation: 53

bash script reading lines in every file copying specific values to newfile

I want to write a script helping me to do my work. Problem: I have many files in one dir containing data and I need from every file specific values copied in a newfile. The datafiles can look likes this:

Name   abc   $desV0
Start   MJD56669   opCMS v2
End   MJD56670   opCMS v2
...
valueX   0.0456   RV_gB
...
valueY   12063.23434 RV_gA
...

What the script should do is copy valueX and the following value and also valueY and following value copied into an new file in one line. And the add in that line the name of the source datafile. Additionally the value of valueY should only contain everything before the dot. The result should look like this:

valueX 0.0456 valueY 12063 name_of_sourcefile

I am so far:

for file in $(find -maxdepth 0 -type f -name *.wt); do
    for line in $(cat $file | grep -F vb); do
        cp $line >> file_done
    done
done

But that doesn't work at all. I also have no idea how to get the data in ONE line in the newfile. Can anyone help me?

Upvotes: 1

Views: 91

Answers (3)

Tom Fenech
Tom Fenech

Reputation: 74615

I think you can simplify your script a lot using awk:

awk '/valueX/{x=$2}/valueY/{print "valueX",x,"valueY",$2,FILENAME}' *.wt > file_done

This goes through every file in the current directory. When "valueX" is matched, the value is saved to the variable x. When "valueY" is matched, the line is printed.

This assumes that the line containing "valueX" always comes before the one containing "valueY". If that isn't a valid assumption, the script can easily be changed.

To print only the integer part of "valueY", you can use printf instead of print:

awk '/valueX/{x=$2}/valueY/{printf "valueX %s valueY %d %s\n",x,$2,FILENAME}' *.wt > file_done

%d is the format specifier for an integer.

If your requirements are more complex and you need to use find, you should use -exec rather than looping through the results, to avoid problems with awkward file names:

find -maxdepth 1 -iname "5*.par" ! -iname "*_*" -exec \
awk '/valueX/{x=$2}/valueY/{printf "valueX %s valueY %d %s\n",x,$2,"{}"}' '{}' \; > file_done

Upvotes: 2

SMA
SMA

Reputation: 37023

Try something like below :

egrep "valueX|valueY" *.wt | awk -vRD="\n" -vORS=" " -F':| ' '{if (NR%2==0) {print $2, $3, $1} else {print $2, $3}}' > $file.new.txt

Upvotes: 0

feeling
feeling

Reputation: 53

don't fight. I'm really thankful for your help and exspecially the fast answers. This is my final solution I think:

#!/bin/bash

for file in $(find * -maxdepth 1 -iname "5*.par" ! -iname "*_*"); do
awk '/TASC/{x=$2}/START/{printf "TASC %s MJD %d %s",x,$2, FILENAME}' $file > mjd_vs_tasc
done

Very thanks again to you guys.

Upvotes: 0

Related Questions