CODEWITHSUNDEEP
dockerdocker-swarmelastic-stack
Sunil Agarwal
Sunil Agarwal

Reputation: 4277

Redirect docker daemon logs to elasticsearch

I have a docker swarm cluster and am able to get all docker "container" logs to ELK stack.

But am unable to get docker daemon logs. Can someone please guide me to achieve this.

FYI : My stack is in Linux.

Upvotes: 1

Views: 2003

Answers (2)

Paul Rey
Paul Rey

Reputation: 1337

You can use Filebeat plugin to send the logs from the daemon logs file to your ELK (plugin presentation page.

There is an article on this point on the elasic.co blog. Your configuration will be different since you don't want containers logs but Docker daemon logs found at the path /var/log/docker.log or /var/log/daemon.log.

EDIT 1:

Since in your environment, the logs are readable with journalctl, I digged up the internet and I have found an ELK plugin that allows you to send the logs from the journald: https://github.com/logstash-plugins/logstash-input-journald

I Hope it'll help.

Upvotes: 1

Joao Cunha
Joao Cunha

Reputation: 772

1st: you'd need to find out where your docker daemon is saving the logs, which depends on linux distribution. See this response with a list of possible places:

https://stackoverflow.com/a/30970134/3165889

2nd: you can use the suggestion of Paul Rey and use Filebeat. As an alternative, I also suggest the use of Fluentd, which usually you can use in place of Logstash, then having EFK instead of ELK, or simply as an extra tool to your ELK environment.

It can also read from a file using the tail input plugin

It can also insert data to Elasticsearch using the elasticsearch out plugin

This tutorial teaches how to log containers, but then you'd need to change your input plugin to tail from that file: Docker logging via EFK

I'd also like to add that, if you're interested in logging the daemon, you probably want to log even if docker is failing to start. So I'd install Fluentd directly on the host. NOT in a container.

Upvotes: 1

Related Questions