CODEWITHSUNDEEP
logstashlogstash-configuration
Dropout
Dropout

Reputation: 13866

Referencing field from input in a Logstash filter

I have the following input, which is generated by FileBeat

{
  "@timestamp": "2018-12-04T09:21:33.360Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.5.0"
  },
  "message": "the message",
  "prospector": {"type": "log"},
  "input": {"type": "log"},
  "beat": {
    "name": "linuxkit-025000000001",
    "hostname": "linuxkit-025000000001",
    "version": "6.5.0"
  },
  "host": {"name": "linuxkit-025000000001"},
  "source": "/opt/foo/logs/bar.log",
  "offset": 9893715,
  "log": {"flags": ["multiline"]}
}

I'd like to use the message field inside my script, so I tried using the following filter

filter {
    mutate {
        add_field => {
            "decoded_base64" => ruby {
                path => "scripts/my_script.rb"
                script_params => { "msg" => "${[message]}" }
            }
        }
    }
}

But I keep getting NoMethodError referencing the method "get".

logstash    | [2018-12-11T14:05:47,765][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:cep_logs, 
:exception=>"NoMethodError", 
:message=>"undefined method `get' for #<Java
::OrgLogstashConfigIrImperative::PluginStatement:0x29fc9676>
\nDid you mean?  getClass\n               get_class\n               gem", 
:backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:210:in `expr'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:202:in `expr'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:133:in `expr'", "org/jruby/RubyArray.java:2486:in `map'",
"/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:97:in `expr_attributes'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:75:in `expr'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/log
stash-core/lib/logstash/compiler/lscl.rb:68:in `expr'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:47:in `block in compile'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstas
h/compiler/lscl.rb:45:in `compile'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:45:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/usr/share/logstash/logsta
sh-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineE
xt.java:149:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:22:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/usr/share/logstash/logstash-core/lib/logsta
sh/pipeline_action/create.rb:38:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}

Can someone please shed some light on why this is happening, please?

I don't think it's relevant, but just in case the script that I'm using is

require "base64"

def register(params)
    @msg = params["msg"]
end

def filter(event)
    if @msg.matches(/<DataB64>(.*)<\/DataB64>/)
        return Base64.decode64($1)
    end
end

Upvotes: 0

Views: 120

Answers (1)

Alain Collins
Alain Collins

Reputation: 16362

Unless something has changed that I'm not seeing in the doc, you shouldn't be using mutate->add_field to run your ruby. Try:

filter {
    ruby {
        ...
    }
}

and have your ruby code add the field with [event.set][1].

Upvotes: 1

Related Questions