CODEWITHSUNDEEP
bash
Sergey
Sergey

Reputation: 199

How count line by minutes in big log file?

Pls Help! I`m have such log file:

[AJPRqu92] 2019-11-04 00:01:31,872 INFO   rervice -  signer.setObject(SiocumntObject.getSgnedocumetbect(xmDoctr.getBytes("UTF-8")));, 0 .
[AJPeqe2] 2019-11-04 00:01:31,872 INFO   r.ervice -  signatureFactory.newSignedDocumentSerializer(), 0 .
[AJPR90] 2019-11-04 00:01:31,965 INFO   roController - <evc:1; cat:4>       . {'ip': 0000, 'size(64)': 1208 }
[AJPReq] 2019-11-04 00:01:31,965 INFO   re -  signer.getSignedDocument(), 93 .
[AJPequesHaHandlr-10] 2019-11-04 00:01:31,965 INFO   rce -   SignatureFactory, 0 .
[AJPequetHandler-RM0] 2019-11-04 00:01:31,965 INFO   ctoService -   UnificationFactory, 0 .
[AuesHandlr-R0] 2019-11-04 00:01:31,965 INFO   ua-   ptoPiderParams, CryptoProvider, 0 .
[AetHander-Rler-2] 2019-11-04 00:01:31,965 INFO   u.c.tse.scypservices.CryptoService -  signer.getSignedDocument(), 93 .
[AJPRequer-RMIallHad-8] 2019-11-04 00:01:31,965 INFO   ru.es.CrSece -  serializer.serialize(sineDocment, bytOtputStream), 0 .

I need to count how many lines per minutes each days generated, example:

2019-11-04 00:01 - 9
2019-11-04 00:02 - 10
2019-11-05 10:21 - 3

How can I do it using bash?

Upvotes: 0

Views: 670

Answers (2)

fra
fra

Reputation: 852

I would extract all the dates first and then use a for loop to count the entries with the same date.

 dates=`cat ~/myLogFile.log | cut -f2 -d" " | uniq`
 for f in $dates ; do value=`grep $f ~/myLogFile.log  | wc -l`; echo "$f - $value" ; done

2019-11-04 - 9
2019-11-05 - 1

If you want the minutes, you can change the cut statement based on your log

dates=`cat ~/myLogFile.log | cut -f2-3 -d" " | cut -f1 -d"," | uniq

Upvotes: 1

JNevill
JNevill

Reputation: 50119

You can use awk.

Something like: 

awk -F"[ :]" '{++a[$2" "$3":"$4]}END{for (item in a){print item, a[item]}}'

This will

  1. Split each line by space or colon -F"[ :]"
  2. For each line in an array called a, increment the value for a key that is made up from the datetime: {++a[$2" "$3":"$4]}
  3. Once the file is processed it will iterate the array and print the key(datetime) and value(count) END{for (item in a){print item, a[item]}}

Upvotes: 1

Related Questions