CODEWITHSUNDEEP
regexlogstash
Shawn Sim
Shawn Sim

Reputation: 555

Logstash if statement with regex example

Can anyone show me what an if statement with a regex looks like in logstash?

My attempts:

if [fieldname] =~ /^[0-9]*$/

if [fieldname] =~ "^[0-9]*$"

Neither of which work.

What I intend to do is to check if the "fieldname" contains an integer

Upvotes: 17

Views: 48497

Answers (5)

marcingo
marcingo

Reputation: 73

Your first format works (for me at the time of writing).

Check the current logstash version in the below excerpt, and also watch for the uuid field present in the output upon match. As expected, empty field matches too, but otherwise it is perfect.

I suggest you to test stuff with such short stdin-stdout configurations. Logstash and Elastic stuff is great, but all too often the corner cases are not properly discussed in the documentation. They develop code faster than the docs as we are all tempted.

============= logstash @ logstash.host.example.com : ~ ============
$ cfg="$(cat)"
input { stdin {} }
filter { if [message] =~ /^[0-9]*$/ { uuid { target => "uuid" } } }
output { stdout { codec => "rubydebug" } }
============= logstash @ logstash.host.example.com : ~ ============
$ /usr/share/logstash/bin/logstash --config.string "$cfg" --pipeline.workers 1 --log.format json --path.data /tmp/kadmar
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2018-11-26 14:50:36.434 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-11-26 14:50:37.646 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.3.0"}
[INFO ] 2018-11-26 14:50:44.490 [Converge PipelineAction::Create<main>] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-11-26 14:50:44.840 [Converge PipelineAction::Create<main>] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4620459c run>"}
The stdin plugin is now waiting for input:
[INFO ] 2018-11-26 14:50:45.048 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2018-11-26 14:50:45.457 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601}
hello
{
       "message" => "hello",
    "@timestamp" => 2018-11-26T13:50:56.293Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
ab123cd
{
       "message" => "ab123cd",
    "@timestamp" => 2018-11-26T13:51:13.648Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
123
{
       "message" => "123",
          "uuid" => "3cac8b35-6054-4e14-b7d0-0036210c1f2b",
    "@timestamp" => 2018-11-26T13:51:18.100Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
1
{
       "message" => "1",
          "uuid" => "1d56982f-421a-4ccd-90d6-6c2c0bcf267d",
    "@timestamp" => 2018-11-26T13:51:25.631Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}

{
       "message" => "",
          "uuid" => "747ac36f-8679-4c66-8050-9bd874aef4c5",
    "@timestamp" => 2018-11-26T13:51:27.614Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
012 456
{
       "message" => "012 456",
    "@timestamp" => 2018-11-26T13:52:09.614Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}

Upvotes: 1

Bohemian
Bohemian

Reputation: 424983

You need this regex (and brackets, I think):

if ([fieldname] =~ /^[0-9]+$/)

Upvotes: 0

Will Barnwell
Will Barnwell

Reputation: 4089

To combine the other answers into a cohesive answer.

Your first format looks correct, but your regex is not doing what you want.

/^[0-9]*$/ matches:

^: the beginning of the line

[0-9]*: any digit 0 or more times

$: the end of the line

So your regex captures lines that are exclusively made up of digits. To match on the field simply containing one or more digits somewhere try using /[0-9]+/ or /\d+/ which are equivalent and each match 1 or more digits regardless of the rest of the line.

In total you should have:

if [fieldname] =~ /\d+/ {
   # do stuff
}

Upvotes: 24

Val
Val

Reputation: 217274

The simplest way is to check for \d

if [fieldname] =~ /\d+/ {
   ...
}

Upvotes: 4

Kerwin
Kerwin

Reputation: 1212

^ asserts position at start of the string

$ asserts position at the end of the string

Your regexp just match the number string, and check contains an integer need remove ^ and $.

Upvotes: 3

Related Questions