CODEWITHSUNDEEP
loggingdockerelastic-stack
herm
herm

Reputation: 16405

Docker: effect of logging driver on logs

I want to use the elk stack (Elastic search, kibana, logstash) for logging of an application built with multiple microservices running in docker (currently in a swram).

We want to log messages from stdout / stderr. Our system should add the serviceName, ContainerID, timestamp (if possible to do it automatically and precise), hostname.... to each logentry without any work for the developer.

Docker supports multiple logging drivers such as Json, syslog and Gelf (only UDP) which can be directly shipped to logstash or through a shipping system (like logspout or others).

My question: How does the logging driver and log shipping choice effect the logs? Does docker always include the same data (like container id, timestamp, actual log message) packaged in different ways or does it actually effect the content?

My colleagues used logspout to collect the logs from the standard docker json logs. Will I get the exact same information into logstash if I use --log-driver=gelf (assuming no packages are lost)?

Upvotes: 0

Views: 415

Answers (1)

herm
herm

Reputation: 16405

I found out that the Json logs only contain: {"log":"log message","stream":"stderr","time":"2017-04-20T07:05:19.584571658Z"} simply by viewing the logfile found with docker inspect. Gelf logs however have a lot of additional fields as described here:

fields := gelfFields{
hostname:      hostname,
containerID:   ctx.ContainerID,
containerName: string(containerName),
imageID:       ctx.ContainerImageID,
imageName:     ctx.ContainerImageName,
command:       ctx.Command(),
tag:           ctx.Config["gelf-tag"],
created:       ctx.ContainerCreated, }

Unfortunately docker only transmits gelf log messages with udp so packets might be lost. However over localhost udp packages are not lost.

Logspout which ships the json logs to Logstash collects additional information about the containter and ships that too so the end result is not so different from the gelf entries. Logspout is however not maintaned by docker and changes to docker might break or change the functionality since the data is not in the actual json log entry.

Upvotes: 1

Related Questions