CODEWITHSUNDEEP
shellsh
GURU SINGH
GURU SINGH

Reputation: 149

Removing Duplicates from CSV in Shell

I am looking for help in below file. There are clients in column 1 which can be part of one or multiple groups. Their status can be either Failed, succeeded or interrupted. I want only those client who is not having any entry of succeeded.

Example

My file is as below

RBCSREXC04 AUTO_RERUN_RBC_DAILY succeeded
RBCSRTM03 AUTO_RERUN_RBC_DAILY succeeded
RBCVMAPPPROD01 AUTO_RERUN_RBC_DAILY succeeded
RBCVVMAPPDEV02 AUTO_RERUN_RBC_DAILY succeeded
E6-RBC-SQL-06 AUTO_RERUN_RBC_DAILY succeeded
E6-ODI-Prod-01 AUTO_RERUN_RBC_DAILY succeeded
GSIERBC2004 AUTO_RERUN_RBC_DAILY succeeded
GSIERBC3008 AUTO_RERUN_RBC_DAILY succeeded 
RBCSRTM03 D_RBC_VM_DUBLIN_E6 failed
RBCSREXC04 D_RBC_VM_DUBLIN_E6 failed
GSIERBC3008 D_RBC_VM_DUBLIN_E6_1 interrupted
E6-ODI-Prod-01 D_RBC_VM_DUBLIN_E6_1 failed
RBCVVMAPPDEV02 D_RBC_VM_DUBLIN_E6_1 failed
E6-RBC-SQL-06 D_RBC_VM_DUBLIN_E6 failed
RBCVMAPPPROD01 D_RBC_VM_DUBLIN_E6 failed
RBCSRCV01 D_RBC_VM_DUBLIN_E6 failed

Below is the Expected Output

RBCSRCV01 D_RBC_VM_DUBLIN_E6 failed

Upvotes: 2

Views: 79

Answers (2)

uzsolt
uzsolt

Reputation: 6037

sed -n '/succeeded/ s,\(.*\) .* succeeded$,\1,p' status.csv | grep -v -f - status.csv

Explanation: the sed produces list of 'x' which has success status (in any group). The grep will drop (because of -v) them from status.csv (the -f - means grep obtains patterns from stdin (equals sed's stdout)).

Upvotes: 1

Freddy
Freddy

Reputation: 4708

You could maintain two arrays with awk for the "good" and the "bad" entries where the array index is the first column and only print the "bad" ones for which no entry in the "good" array exists.

awk '
  $3=="succeeded"{ good[$1] }  # we only need the index here
  $3=="failed" || $3=="interrupted"{
    if ($1 in bad){ 
      bad[$1]=bad[$1] ORS $0 # append this line to existing entry
    } else {
      bad[$1]=$0             # save the line
    }
  }
  END{
    for (i in bad)
      if (!(i in good))print bad[i]
  }
' file

Upvotes: 1

Related Questions