CODEWITHSUNDEEP
dockerdocker-composefluentdefk
leeman24
leeman24

Reputation: 2889

Side Effects When Using fluentd logging driver with Docker

I am setting up an EFK stack which works as expected but came with a few drawbacks that I would like to work around.

  1. If my fluentd instance is not running, other containers using that use the fluetnd log driver will fail to start. This is a pretty big concern since the main application is now dependent on my log aggregator. 
ERROR: for sharedcontainers_mongo_s3_backup_1  Cannot start service mongo_s3_backup: b'failed to initialize logging driver: dial tcp [::1]:24224: connect: connection refused'
  1. When using the fluentd logging driver, I am no longer able to view the logs in the command-line when running docker-compose logs. While this is okay more times than not, there could be an issue with the log aggregator (maybe a situation which broke all the containers) and you are now unable to easily view your logs. Are you able to use multiple logging drivers or another workaround?
redis_1               | WARNING: no logs are available with the 'fluentd' log driver

docker-compose.yml:

logging:
  driver: "fluentd"
  options:
    fluentd-address: localhost:24224

Upvotes: 0

Views: 3363

Answers (1)

vmax33
vmax33

Reputation: 683

In docker community version you have only few logging drivers that support 'dual logging': local, json-file, journald.

So, there are several options:

  1. Use journald log driver for dockers, this way you can see the logs using docker logs command and also using journalctl. In order to forward the logs to fluentd, we configure syslog on the host to forward all the logs to fluentd port.

  2. Use docker Enterprise version. This version provides 'dual' logging for other drivers including fluentd log driver.

Check this link for more details docs.docker.com/config/containers/logging/configure

Upvotes: 2

Related Questions