CODEWITHSUNDEEP
elasticsearchlogstashlogstash-configurationelk
Santhosh Kumar
Santhosh Kumar

Reputation: 53

Logstash throws [message=>"undefined method `update' for nil:NilClass"] when Elasticsearch index template defined in logstash

I am trying to load data from sql server to elastic search using logstash. For index creation i am using index template that is specified in the config file of logstash.

Output field of logstash.config :

 output{ 
    stdout { codec => json_lines } 
    elasticsearch { 
                    hosts => ["localhost:9200"] 
                                             document_id => "%{[id]}" 
                                             manage_template => true                     
                                             template => "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json" 
                                             template_name => "logstash_test" 
                                             template_overwrite => true 
            } 
    }

Error while running logstash :

    [2021-01-25T19:48:28,232][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} 
[2021-01-25T19:48:28,298][INFO ][logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"} 
[2021-01-25T19:48:28,370][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/sampleconf.conf"], :thread=>"#<Thread:0x797ce1a8 run>"} 
[2021-01-25T19:48:28,509][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:42:in `add_ilm_settings_to_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]} 
[2021-01-25T19:48:28,546][INFO ][logstash.outputs.elasticsearch][main] Creating rollover alias <logstash-{now/d}-000001> 
[2021-01-25T19:48:29,463][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.08} 
[2021-01-25T19:48:29,802][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"} 
[2021-01-25T19:48:29,888][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

Index template configuration:

{ 
   "index_patterns":"*-logstash_test-*", 
   "priority":1, 
   "template":{ 
      "settings":{ 
            "analysis":{ 
               "analyzer":{ 
                  "test_autocomplete":{ 
                     "tokenizer":"test_tokenizer" 
                  } 
               }, 
               "tokenizer":{ 
                  "test_tokenizer":{ 
                     "custom_token_chars":"'-", 
                     "max_gram":"3", 
                     "min_gram":"3", 
                     "token_chars":[ 
                         
                     ], 
                     "type":"ngram" 
                  } 
               } 
            }, 
         "mappings":{ 
            "properties":{ 
               "test_id":{ 
                  "type":"text", 
                  "analyzer":"test_autocomplete" 
               } 
            } 
         } 
      } 
   } 
}

Although it fails the index gets created with the data, but the analyzers specified in the settings{..} does not reflect. I am new to ELK stack and not sure on why this is happening.

Elasticsearch Version : 7.10.1

Logstash Version : 7.10.1

Upvotes: 1

Views: 1494

Answers (1)

Val
Val

Reputation: 217254

Make sure that your logstash_test.json file has the following format:

{
  "index_patterns": "*-logstash_test-*",
  "order": 1,
  "settings": {
    "analysis": {
      "analyzer": {
        "test_autocomplete": {
          "tokenizer": "test_tokenizer"
        }
      },
      "tokenizer": {
        "test_tokenizer": {
          "custom_token_chars": "'-",
          "max_gram": "3",
          "min_gram": "3",
          "token_chars": [],
          "type": "ngram"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "test_id": {
        "type": "text",
        "analyzer": "test_autocomplete"
      }
    }
  }
}

You had mappings and settings enclosed within the template section, but this is only for the new index templates which the elasticsearch Logstash output doesn't support yet. You need to use the legacy index templates.

Upvotes: 3

Related Questions