CODEWITHSUNDEEP
elasticsearchlogstashelastic-stack
Fafenback
Fafenback

Reputation: 23

Logstash / Elasticsearch: "Failed to install template" / " Got response code '400' "

I'm new with ELK stack, i'm trying to install a template from logstash output.elasticsearch but when i'm put a "mappings" key in my JSON i have this issue:

[2020-09-12T15:19:04,321][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Got response code '400' contacting Elasticsearch at URL 'http://elasticsearch:9200/_template/maillog'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:278:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:277:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:352:in `template_put'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in `template_install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:28:in `install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:16:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:130:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:51:in `block in setup_after_successful_connection'"]}

Here my JSON template:

{
  "index_patterns": "*-maillog-*",
  "settings": {
    "index": {
      "refresh_interval": "10s",
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  },
  "mappings": {
    "maillog": {
      "properties": {
        "ip": { "type": "ip" }
      }
    }
  }
}

Here my output.elasticsearch:

output {
     elasticsearch {
        id => "test"
        index => "%{[product]}-maillog-%{+YYYY.MM.dd}"
        hosts => ["###ELASTIC_HOST###:9200"]
        document_type => "maillog"
        manage_template => true
        template_overwrite => true
        template => "${CONF_PATH}/mapping/maillog.json"
        template_name => "maillog"
      }
}

With this conf elastic can't create my template but if i remove "mappings" key from the template, like this:

{
  "index_patterns": "*-maillog-*",
  "settings": {
    "index": {
      "refresh_interval": "10s",
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  }
}

There is no issue anymore.

My stack is composed from 3 containers :

elasticsearch 7.4.2

logstash 7.4.2

kibana 7.4.2

I'm probably missing something, but many hours spent and no clue to resolve this issue...

Thanks for your help

Upvotes: 2

Views: 11492

Answers (1)

leandrojmp
leandrojmp

Reputation: 7463

You are using elasticsearch version 7.X which does not have types anymore.

The mailog after your mappings declaration would be your type in versions before 7.X, but this does not work on version 7.X anymore, you need to change your mappings to the one bellow.

"mappings": {
    "properties": {
        "ip": { "type": "ip" }
    }  
}

Also, you can remove the document_type in your elasticsearch output in Logstash, this does not work anymore.

Upvotes: 4

Related Questions