CODEWITHSUNDEEP
bashunixawkparametersgrep
MJM
MJM

Reputation: 87

using parameter text file in bash

How can I change 3rd field of records by depending on another file? Is it possible to use awk?

Sorry I am new to this

Example: Records

xxxx xxxx 1234 xxxx
xxxx xxxx 5678 xxxx

Parameter (another file)

1234,9001
5678,9020

My expected output is

xxxx xxxx 9001 xxxx
xxxx xxxx 9020 xxxx

Glimpse of my code

 #!/usr/bin/ksh

 SRC=/home

 FILE_LIST=`sqlplus -s idmp_stg/idmp_stg@DTPMPDR07_SUDB << EOF
 set echo off head off feed off pagesize 0 trimspool on linesize 1000 colsep ,
 spool output.csv
 SELECT * from USAGE_TYPE_PARAM;             
 spool off;
 exit;
 EOF`

#Using while loop read values into variables from CSV file and create flat file 
 for    each records
 counter=1
 while IFS=, read V1 V2
 do
 echo "${V1} ${V2}" > param_${counter}.txt
 counter=$(( counter + 1 ))
 done < output.csv


 cd $SRC

 ls D* | while read FILES
 do

  #--this supposed to change the 3rd field of the file but it doesn't show 
  #--an output,       just zero byte file
  awk 'NR==FNR{a[$1]=$2;next}{$3=a[$3]}1' FS="," output.csv FS=" " $FILES >  
  final_output.txt

  done`

Upvotes: 2

Views: 250

Answers (2)

John1024
John1024

Reputation: 113924

$ awk 'NR==FNR{a[$1]=$2;next} {$3=a[$3];print}' <(tr , " " <Parameter) Records 
xxxx xxxx 9001 xxxx
xxxx xxxx 9020 xxxx

Explanation:

Taking it one piece at a time:

  • NR==FNR{a[$1]=$2;next}

    awk is processing two files, one after the other, one line at a time. NR is the total number of lines read and FNR is the number of lines read in the current file. So, when NR==FNR, we are in the first file which, in this case, is Parameter. These commands are therefore executed only while reading Parameter. a[$1]=$2 creates a dictionary whose keys are the first field and whose corresponding values are the second field of Parameter. The next command tells awk to ignore the remaining awk commands and skip to the next line.

  • {$3=a[$3];print}

    Because of the next statement above, these commands are only executed when reading the second file. They change the third field to its new value and print the line.

  • <(tr , " " <Parameter)

    Unlike Records, the file Parameter is comma-separated. Here, the translate command, tr, is used to convert it from comma-separated to space-separated before awk reads it. The <(...) construct is known as process substitution.

POSIX or mksh or pdsh

Process substitution is a bash/ksh/zsh extension not supported by all shells. To run this without process substitution:

$ tr , " " <Parameter | awk 'NR==FNR{a[$1]=$2;next} {$3=a[$3];print}' - Records 
xxxx xxxx 9001 xxxx
xxxx xxxx 9020 xxxx

In this command, the first file argument to awk is - which means stdin. The output of tr is piped into awk to provide this stdin.

This works the same as the previous solution but avoids process substitution.

Upvotes: 4

jaypal singh
jaypal singh

Reputation: 77145

Using awk:

$ cat Records
xxxx xxxx 1234 xxxx
xxxx xxxx 5678 xxxx

$ cat Parameter
1234,9001
5678,9020

$ awk 'NR==FNR{a[$1]=$2;next}{$3=a[$3]}1' FS="," Parameter FS=" " Records
xxxx xxxx 9001 xxxx
xxxx xxxx 9020 xxxx

Set the Field Separator variable at the end before the file name to set it for that particular file.

Upvotes: 1

Related Questions