CODEWITHSUNDEEP
javaelasticsearchlogstash
ps0604
ps0604

Reputation: 1071

Defining constants in Logstash 7.0

My intent is to index in ElasticSearch N tables from an Oracle database. For that, I define one jdbc for each table in the input section. I don't want to repeat, for example, the location of the JDBC jar in each jdbc entry, so I tried to mutate/add_field to set the value once but this doesn't work. I get

Couldn't find any input plugin named 'mutate'

I tried stdin instead of mutate and it didn't work either. Any ideas how to set and use a constant? 

input {
  mutate {
      add_field => { "[@metadata][lib]" => "/path/to/lib/ojdbc8.jar" }
  }

  jdbc {
    jdbc_driver_library => [@metadata][lib]
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    jdbc_connection_string => "jdbc:oracle:thin:@//localhost:1521/XE"
    jdbc_user => "user"
    jdbc_password => "pwd"
    schedule => "* * * * *"
    statement => "SELECT col1 from table1"
  }

  jdbc {
    jdbc_driver_library => [@metadata][lib]
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    jdbc_connection_string => "jdbc:oracle:thin:@//localhost:1521/XE"
    jdbc_user => "user"
    jdbc_password => "pwd"
    schedule => "* * * * *"
    statement => "SELECT col2 from table2"
  }

}
output {
     elasticsearch {
        hosts => ["localhost:9200"]
        index => "my_index"
        user => "kibana"
        password => "changeme"
    }
}

Upvotes: 0

Views: 274

Answers (2)

Furhan S.
Furhan S.

Reputation: 1524

As @Alain Collins mentioned, you can use the environment variables. Here is an example:

input {
  jdbc {
    jdbc_driver_library => "${JDBC_DRIVER_LIB}"
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    jdbc_connection_string => "jdbc:oracle:thin:@//localhost:1521/XE"
    jdbc_user => "user"
    jdbc_password => "pwd"
    schedule => "* * * * *"
    statement => "SELECT col1 from table1"
  }
}

and then you can set the environment variable when you are starting up the Logstash process. For instance, if you are using command line to start Logstash process:

JDBC_DRIVER_LIB=/path/to/lib/ojdbc8.jar /PATH_TO_LOGSTASH_BIN_DIR/logstash -f YOUR_CONFIG.yaml

Upvotes: 2

Alain Collins
Alain Collins

Reputation: 16362

The error message gives you a clue - mutate is a filter plugin, and is not available in the input or output sections.

You might try using environment variables, which you can use in the input section.

Upvotes: 0

Related Questions