CODEWITHSUNDEEP
elasticsearchlogstashkibanafilebeatelk
Devendra
Devendra

Reputation: 305

Kibana shows only 1 servers logs among 2 servers sent via Logstash

I have 2 servers with filebeat installed on it and on another server I have ELK stack installed.

On ELK server under Logstash conf directory, I have created 2 files representing 2 servers

On both servers In filebeat configuration file, I have specified different ports for each servers (Logstash as a Output)

And here's my Logstash file for the server 1,

input {
  beats {
    port => 5044
  }
}

output {
  if "api_logs" in [tags] {
  elasticsearch {
    hosts => [ "es:9200" ]
    index => "api_logs-%{+YYYY.MM.dd}"
  }
 }
 else if "error_logs" in [tags] {
 elasticsearch {
    hosts => [ "es:9200" ]
    index => "error_logs-%{+YYYY.MM.dd}"
  }
 }
}

And here's my Logstash file for the server 2,

input {
  beats {
    port => 5045
  }
}

output {
  if "api_logs" in [tags] {
  elasticsearch {
    hosts => [ "es:9200" ]
    index => "api_logs-%{+YYYY.MM.dd}"
  }
 }
 else if "error_logs" in [tags] {
 elasticsearch {
    hosts => [ "es:9200" ]
    index => "error_logs-%{+YYYY.MM.dd}"
  }
 }
}

My filebeat file for server 1

filebeat.inputs:

- type: log
  tags: ["api_logs"]
  enabled: true
  paths:
    - logs/api*

- type: log
  tags: ["error_logs"]
  enabled: true
  paths:
    - logs/error*
output.logstash:
  hosts: ["es:5044"]

My filebeat file for server 2

filebeat.inputs:

- type: log
  tags: ["api_logs"]
  enabled: true
  paths:
    - logs/api*

- type: log
  tags: ["error_logs"]
  enabled: true
  paths:
    - logs/error*
output.logstash:
  hosts: ["es:5045"]

But when I create an index in Kibana, It only shows one server logs, not another server (Under host.name field).

please suggest some suggestions.

Upvotes: 0

Views: 288

Answers (1)

justkind
justkind

Reputation: 149

No obvious errors in the configuration, and if it works for one server, then it should be working for the other server too so I doubt it's a misconfiguration issue.

I think it would be a good idea to look at the differences between the servers and ensure that they are configured exactly the same.

Check things like:

  • connectivity from both servers (especially the failing server) with filebeat to the Logstash server and ports
  • compare file permissions of log files on both servers
  • enable INFO logging on both filebeat and logstash and try to see if anything stands out
  • ensure versions are compatible with beats and ELK on both servers (https://www.elastic.co/support/matrix#matrix_compatibility)

Upvotes: 0

Related Questions