CODEWITHSUNDEEP
elasticsearchfilebeat
Surfingjoe
Surfingjoe

Reputation: 9

Filebeat is not sending a continuous stream to Logstash

I'm running a development set of servers (not production). I have an ELK server, a web server and a reverse proxy server. Data from the web server and the reverse proxy has successfully been sent into Logstash on the ELK server. Kibana's filebeat discovery does show the host name of all three servers. So I know, the configuration is correct.

Running the following command: sudo filebeat -e -c /etc/filebeat/filebeat.yml test output on both remote servers (the web server and the reverse proxy server) has the following response: logstash: 192.168.1.6:5044... connection... parse host... OK dns lookup... OK addresses: 192.168.1.6 dial up... OK TLS... WARN secure connection disabled talk to server... OK

Running the command to see the status of filebeat service (on both remote servers) reflects the service is active and has recent timestamp of log data that is being collected. I did this a few times over a span of 15 minutes and I can continously see new time stamps of log collection by file beat on the remote servers.

But for some reason, the only time filebeat actually sends data to Logstash on the ELK server is upon reboot of the web server and reverse proxy server. And only just once, filebeat data is not a continuous stream of data into Logstash.

So my question is: How is it possible that testing filebeat output shows a succesful connection, and that filebeat data is succesfully seen in elasticsearch, and Kibana discovery has valid data for my servers. But, filebeat is not sending continuously into logstash?

Oh, one more thing. The ELK server does in fact have a continuous stream of filebeat data from itself (the ELK server's filebeat is continuously streaming data to logstash). It's the other servers that are not feeding continuously.

Running Elasticstack version 7.17.13 on Ubuntu servers. ELK server is ubuntu server 20.04 (minimal), web server is ubuntu server 20.04 (minimal), and the reverse proxy server is ubuntu server 22.04 (minimal). All are on the same subnet. Running as virtual servers in an ESXi VMware platform.

I've tried purging filebeat from the web server and the reverse proxy server and making sure the filebeat configurations are deleted.

Then reinstalling filebeat and putting in a fresh configuration to send all data to logstash on the ELK server and test the filebeat outputs.

Everything looks OK, but I see only the one stream of data into logstash only upon reboot of the servers! Ughhhhh!

Upvotes: 0

Views: 294

Answers (1)

Surfingjoe
Surfingjoe

Reputation: 9

I have determined why the Filebeat service is not streaming to ELK logstash. It is as simple as the logs do not exist to be sent. Apparently, a Ubuntu server (minimized) doesn’t have “rsyslog” installed, and therefore "auth.log, kern.log, and syslog don’t exist. Therefore, there are no logs that would change over time in the path /var/log/. This makes sense, I suppose, because a minimized system is not expecting users to log into the system.

However, I must ensure that no rogue users are getting into the system, and therefore, I must see user logins, user sudo commands, etc., especially on the reverse proxy.

So I ran sudo apt-get install rsyslog on the web server and the reverse proxy server. Now, I see a regular stream of data into logstash from those servers because now the data actually exists.

Still don’t see sudo commands yet. Hopefully, I can figure that one out. At least I have streaming data from remote servers into Logstash now.

Upvotes: 0

Related Questions