CODEWITHSUNDEEP
bashshellawksedgrep
Teju Priya
Teju Priya

Reputation: 665

grep with delimiters in shell script

Hi I have a log file and I am doing grep operations like this.

cat logfile | grep select

2018-03-20T15:26:34,397 INFO  [2da4e66f-6092-46a7-9542-60afc0611205 HiveServer2-Handler-Pool: Thread-32([])]: ql.Driver (Driver.java:compile(429)) - Compiling command(queryId=hive_20180320152634_a6ef02a1-e018-4085-8ceb-8a8d2733b427): select * from reportingperiod limit 5
2018-03-20T15:26:37,761 INFO  [HiveServer2-Background-Pool: Thread-35([])]: ql.Driver (Driver.java:execute(1735)) - Executing command(queryId=hive_20180320152634_a6ef02a1-e018-4085-8ceb-8a8d2733b427): select * from reportingperiod limit 5

I am trying to extract the query part based on the delimiter like this

delimiter based extract, first delimiter ': ' upto '\n'

So that I can get the expect output like this.

select * from reportingperiod limit 5

Initially the query extract I tried with regex and grep -OE methods. Right now I am trying to implement a generic method, So that any query will be captured.

I tried like this.

IFS=$': '
for i in `cat logfile`; do    echo $i; 
done

The above code is not working as expected, Since I don't know how to pass the second delimiter to extract the query. Any help will be appreciated.

Upvotes: 3

Views: 4447

Answers (2)

mjz19910
mjz19910

Reputation: 242

You could probably use cut as well.

My opinion is that you should use the most simple tool that satisfies your requirements, so this is probably a bit better.

print all, after first :

cut -d ':' -f 2- <file>

print stuff after first 2 fields.

echo xxx:yyy:zzz | cut -d ':' -f -2 --complement

Upvotes: 2

kvantour
kvantour

Reputation: 26521

I believe what you are interested in is :

awk -F ':' '/select/{print $NF}' <logfile>

be aware that you write from the first : till the end of line. However, in your example there are multiple : per line.

You could also just use sed to accomplish the same :

sed '/select/!{d};s/.*: *//' <logfile>

Upvotes: 1

Related Questions