CODEWITHSUNDEEP
elasticsearchjdbclogstash
AlisonGrey
AlisonGrey

Reputation: 507

Logstash JDBC: Update row issue

Im using the Below JDBC code in Logstash for updating the already existing index in Elasticsearch, without duplicating rows or adding the updated row as another new row. Versions: Elasticsearch, Logstash and Kibana are v7.1.0.

input {
    jdbc {
        jdbc_connection_string => "jdbc:sqlserver://DB01:1433;databasename=testdb;integratedSecurity=true"
        jdbc_driver_class =>  "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_driver_library => "C:\Program Files\sqljdbc_6.2\enu\mssql-jdbc-6.2.2.jre8.jar"
        jdbc_user => nil
        statement => "SELECT * from data WHERE updated_on > :sql_last_value ORDER BY updated_on"
    use_column_value =>true
        tracking_column =>updated_on
        tracking_column_type => "timestamp"
    }
}
output {
          elasticsearch { hosts => ["localhost:9200"] 
        index => "datau" 
        action=>update
            document_id => "%{id}"
            doc_as_upsert =>true}
          stdout { codec => rubydebug }
       }

when i run the above in logstash (logstash -f myfile.conf) the below error appars.

[2019-08-21T10:46:33,864][ERROR][logstash.outputs.elasticsearch] Failed to insta ll template. {:message=>"Got response code '400' contacting Elasticsearch at URL  'http://localhost:9200/_template/logstash'", :class=>"LogStash::Outputs::Elasti cSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["D:/ELK 6.4.0/log stash-6.4.0/logstash-6.4.0/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasti csearch-9.2.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adap ter.rb:80:in `perform_request'", "D:/ELK 6.4.0/logstash-6.4.0/logstash-6.4.0/ven dor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.0-java/lib/logstas h/outputs/elasticsearch/http_client/pool.rb:291:in `perform_request_to_url'"...

Where am i gong wrong?

Upvotes: 0

Views: 347

Answers (2)

Luis Celestino
Luis Celestino

Reputation: 11

Remove } from this line:

doc_as_upsert =>true}

Upvotes: 1

AlisonGrey
AlisonGrey

Reputation: 507

The issue was with the version compatibility. I had used 6.4 logstash on 7.1 Elasticsearch. Once my logstash was upgraded, the issue was resolved.

Thanks!

Upvotes: 0

Related Questions