CODEWITHSUNDEEP
logstashlogstash-configuration
Ishan Hettiarachchi
Ishan Hettiarachchi

Reputation: 1704

Logstash jdbc input plugin with multiple tables

We are using Logstash to get data from RDBMS into Elastic search. At the moment I have multiple config files, each having a single query for each table we want to index. While this seems to be working fine is there a better way to do it.

NOTE: We cannot use stored procedures.

Upvotes: 1

Views: 2818

Answers (1)

Tomáš Poch
Tomáš Poch

Reputation: 890

You can use several input plugins (of same type) in one config - like 

input {
  jdbc {
  jdbc_driver_library => "mysql-connector-java-5.1.15.jar"
  ... remaining config ... 
  statement => "SELECT * FROM t1;"
  type => "score-history"
  }
}

jdbc {
  jdbc_driver_library => "mysql-connector-java-5.1.15.jar"
  ... remaining config ... 
  statement => "SELECT * from table2"
  type => "score"
  }
}

I am afraid that there is no way to share the connection string as well as other config options between two plugins. The rest of the processing can be different depending on the type set in the input plugin - for instance 

filter{
  if [type] == "score-history" {
  date {
    match => [ "sctimestamp", "UNIX" ]
    }
  }
}

Upvotes: 2

Related Questions