CODEWITHSUNDEEP
bashsed
bluethundr
bluethundr

Reputation: 1345

Replace line in aws cli output with sed

I have a script that I wrote that deletes AWS snapshots. I want to remove part of the output when a snapshot doesn't exist.

In this output:

Deleting unused snapshots in AWS Gov Non Prod
*****************************************************************
deleting snapshot: snap-0b571b64784bac904

An error occurred (InvalidSnapshot.NotFound) when calling the DeleteSnapshot operation: The snapshot 'snap-0b571b64784bac904' does not exist.
*****************************************************************


*****************************************************************
Verify that snapshot: snap-0b571b64784bac904 is gone:

An error occurred (InvalidSnapshot.NotFound) when calling the DescribeSnapshots operation: The snapshot 'snap-0b571b64784bac904' does not exist.
*****************************************************************

I want to remove just the phrase: "An error occurred (InvalidSnapshot.NotFound) when calling the DescribeSnapshots operation:" and leave the phrase: The snapshot 'snap-0b571b64784bac904' does not exist.

In my script I tried adding sed to remove what I don't want:

aws ec2 delete-snapshot --snapshot-id=$i --profile=govcloud-nonprod | sed -e 's/An error occurred (InvalidSnapshot.NotFound) when calling the DeleteSnapshot operation: //g'

This is my script in its entirety:

    #!/bin/bash

echo "Deleting unattached volumes in AWS Lab"

for i in $(cat aws_lab_volumes.txt)
do
  echo "*****************************************************************"
  echo "deleting volume: $i"
  aws ec2 delete-volume --volume-id=$i --profile=lab | sed -e 's/An error occurred (InvalidVolume.NotFound) when calling the DeleteVolume operation: //g' 2>/dev/null
  echo "*****************************************************************"
  echo; echo; echo; echo; echo
  sleep 5
  echo "*****************************************************************"
  echo "Verify that volume: $i is gone:"
  aws ec2 describe-volumes --volume-ids=$i --profile=lab | sed -e 's/An error occurred (InvalidVolume.NotFound) when calling the DescribeVolumes operation: //g' 2>/dev/null
  echo "*****************************************************************"
  echo; echo; echo; echo; echo
done

So basically the sed line is failing. How can I use sed to remove just the part I don't want?

Upvotes: 3

Views: 2603

Answers (2)

Walter A
Walter A

Reputation: 20032

EDIT: deleted first improvement, that has been used by OP.

This problem shouldn't be so difficult, but we have to try it in small steps. You have followed my advice to add 2>/dev/null, but wrote this on the wrong place:

aws ... --profile=lab | sed -e 's/...//g' 2>/dev/null

should be

aws ... --profile=lab 2>/dev/null | sed -e 's/...//g'

When this works you will be missing the whole line, so even better is

aws ... --profile=lab 2>&1 | sed -e 's/...//g'

When this doesn't help, try the next approaches:

Review your code and what you have written in the code
The example output shows Deleting unused snapshots in AWS Gov Non Prod, the code shows echo "Deleting unattached volumes in AWS Lab".
Are you testing the same file that you have been editing?

Make sure #!/bin/bash is on the first line without additional spaces

Debug with a temp file
Redirect the aws output to a file and look at that file.

aws ... --profile=lab > /tmp/aws.out
vi /tmp/aws.out

(look voor special characters, TAB characters, ..)

Try simple commands with /tmp/aws.out 

sed 's/An error/===/' /tmp/aws.out
sed 's/An error.*: /===/' /tmp/aws.out # something special in the part you try to match
sed 's/ /=/' /tmp/aws.out # Are all white spaces a space ?
sed 's/.*/===/' /tmp/aws.out # Can sed do anything at all or are you missing an EOL 
grep -v "An error" /tmp/aws.out # sed should be fine, but will this remove the complete line?

Upvotes: 1

Josiah DeWitt
Josiah DeWitt

Reputation: 1812

Make sure you don't have a character that requires an escape, the dot "." and likely the parens "()" are going to need special attention in the sed search string, however; In this case there doesn't seem to be a need for large specific strings to search. Try dumbing it down like so and increase string complexity from there. This worked in my tests which I cat a file to stdout, however your command may send to stderr if so the second line should work for you.

aws ec2 delete-snapshot --snapshot-id=$i --profile=govcloud-nonprod | sed 's/.*DescribeSnapshots.*: //'

aws ec2 delete-snapshot --snapshot-id=$i --profile=govcloud-nonprod 2>&1 | sed 's/.*DescribeSnapshots.*: //'

Upvotes: 3

Related Questions