CODEWITHSUNDEEP
elasticsearchlogstashelklogstash-jdbc
Jason
Jason

Reputation: 55

How can I manually trigger timing jdbc input in logstash?

I have one timing jdbc logstash input and load data into elasticsearch.

input {
    jdbc {
        jdbc_connection_string => "XXX"
        jdbc_user => "XXX"
        jdbc_password => "XXX"
        jdbc_validate_connection => true
        jdbc_driver_library => "XXX"
        jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
        statement => "XXX"
        schedule => "0 3 * * * America/New_York"
        type => "XXX"
        add_field => { "[@metadata][indexname]" => "XXX" }
    }
}

output {
    elasticsearch {
        hosts=> ["XXX","XXX","XXX"]
        index=>"%{[@metadata][indexname]}"
        user => XXX
        password => XXX
        ssl => true
        ssl_certificate_verification => false
        cacert => 'XXX'
        document_id => "%{uniqueid}"
    }
    stdout {}
}

I wonder how can I manually trigger it running, if there have some jdbc connection failed or elasticsearch connection issue?

Upvotes: 0

Views: 134

Answers (2)

Ashish Tiwari
Ashish Tiwari

Reputation: 2277

The best you can do is to put monitoring on Logstash. From monitoring data you can setup different types of alert like

  1. Number of events processed. If number is less than expectation it will generate alert and you can simply refer log file what wrong.
  2. You can read logstash error log and push to Elasticsearch using Elastic agent. In that log, logstash will log error if something comes like db connection etc. You can write alert rule for that also.
  3. Monitoring agent remains active even if the Logstash instance does not. So if your logtsash stopped, you will able to catch in logs and it could be the another alert.

Upvotes: 0

Musab Dogan
Musab Dogan

Reputation: 3680

You can run the logstash on command line like the following.

bin/logstash -f mypipeline.conf

Because you have stdout you will also see the output on the terminal. https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html

Note: if it's slow don't worry the stdout is a performance killer :) Recommended to remove it on prod after testing.

Note2: Remove schedule in logstash.conf during test otherwise you should wait until the schedule time.

Upvotes: 1

Related Questions