CODEWITHSUNDEEP
javaregexlogstashlogstash-configurationlogstash-file
Hamzeh Abu Ajamieh
Hamzeh Abu Ajamieh

Reputation: 47

What is the best way to extract request and response data from a log file using Logstash and regex?

I have application log file which contains the application requests and responses, the complete request and response looks like the below, I tried different patterns using RegEx but unfortunately without any luck, can some one suggest what should I change :

pattern =>

"
(?m)\b\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}\b.?\bRequest given by the user\b.?\brspDesc is\b(?!.\n)..?(?=\R|$)
"

2023-05-11 00:20:26,103 [http-apr-7777-exec-46] INFO com.welcome.ws.AccountWebServiceImpl. - Request given by the user

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:GetCustomerBalanceReq xmlns="url" xmlns:ns2="url">

testt
**
xls

ns2:ID/EID/5000000004/123456****1234</ns2:ID>
</ns2:GetCustomerBalanceReq>

2023-05-11 00:20:26,144 [http-apr-7777-exec-46] INFO com.welcome.svc.AccountService. - rspCode is:1001 and rspDesc is:Account or Media does not Exist

I was trying this regex but it wasn't matching anything

"(?m)\b\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}\b.?\bRequest given by the user\b.?\brspDesc is\b(?!.\n)..?(?=\R|$)

Upvotes: 0

Views: 440

Answers (1)

Reilas
Reilas

Reputation: 6266

To reiterate, you're saying that within the log-file there are sections which start and end with the example text you provided.

To capture these sections, you can use the following regular expression pattern.

(?s)(\d{4}-\d\d-\d\d .+?rspDesc.+?$)
  • Use the s flag to enable single-line mode, causing a . to additionally match new-line characters.
  • The beginning of the pattern will match the time-stamp.
  • Then, match all characters up to "rspDesc".
  • Finally, continue until the end of the line is reached.

Upvotes: 0

Related Questions