CODEWITHSUNDEEP
elastic-stackelk
techzone4all
techzone4all

Reputation: 143

logstash Issue sending data to elasticseach cluster

I have upgraded ELK stack into 7.4 version ( filebeat, logstash, elasticalert, kibana). I am using elasticsearch cloud.

Once after the upgrade, following error shows in logstash log file. but few of records can see in kibana.

[2019-10-25T15:22:01,578][ERROR][logstash.outputs.elasticsearch][main] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [https://user:xxxxxx@:9243/][Manticore::ConnectTimeout] connect timed out", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2019-10-25T15:22:01,595][WARN ][logstash.outputs.elasticsearch][main] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://user:xxxxxx@:9243/][Manticore::ConnectTimeout] connect timed out {:url=>https://user:xxxxxx@/, :error_message=>"Elasticsearch Unreachable: [https://user:xxxxxx@:9243/][Manticore::ConnectTimeout] connect timed out", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}

How i can solve this issue?

Upvotes: 2

Views: 465

Answers (1)

techzone4all
techzone4all

Reputation: 143

fixed the issue by optimizing the pipelines. in my setup each filebeat servers sending different logs. So configured separate pipeline and different workers according to logs size. See the pipeline.yml file as below. 

- pipeline.id: intake
  config.string: |
    input { beats { port => 5043 } }
    output {
    if [log][file][path] =~ "request-response-logger" {
        pipeline { send_to => requestResponseLogger }
    } else if [host][name] =~ "wso2telcohubgateway" and [log][file][path] =~ "wso2carbon"  {
        pipeline { send_to => wso2gateway }
    } else if [host][name] =~ "wso2esb" and [log][file][path] =~ "wso2carbon" {
        pipeline { send_to => wso2esb }
    } else if [host][name] =~ "wso2ei" and [log][file][path] =~ "wso2carbon" {
        pipeline { send_to => wso2ei }
    } else if [host][name] =~ "wso2telcomig" and [log][file][path] =~ "wso2carbon" {
        pipeline { send_to => wso2telcomig }
    }
    }


- pipeline.id: requestResponseLogger
  pipeline.workers: 1
  path.config: "/etc/logstash/conf.d/requestResponseLogger.conf"

- pipeline.id: wso2gateway
  pipeline.batch.delay: 20
  pipeline.workers: 15
  path.config: "/etc/logstash/conf.d/wso2gateway.conf"

- pipeline.id: wso2esb
  pipeline.workers: 20
  pipeline.batch.delay: 10
  path.config: "/etc/logstash/conf.d/wso2esb.conf"

- pipeline.id: wso2ei
  pipeline.workers: 2
  path.config: "/etc/logstash/conf.d/wso2ei.conf"

- pipeline.id: wso2telcomig
  pipeline.workers: 10
  path.config: "/etc/logstash/conf.d/wso2telcomig.conf"

Upvotes: 1

Related Questions