Finding a string in docker logs of container

Question

What is the best way to find a specific string in the logs of a docker container? Let's say I want to see all requests, that are made in the "nginx" docker image that came from a ip starting with "127."

grep wont work as expected on docker logs command:

docker logs nginx | grep "127."

It prints all logs, but does not filter the result!

Answer 1

docker logs nginx | grep -a '127.'

Answer 2

this can happen if the container is logging to stderr, piping works only for stdout, so try:

docker logs nginx 2>&1 | grep "127."

• "2>&1" will tell the shell to redirect stderr to stdout
• "|" to give the "unified" output to grep using pipe

Answer 3

The top-rated solution didn't work for me - grep printed this output and I couldn't properly find what I was searching for (I'm trying this with a custom docker container, not with nginx):

docker logs collector 2>&1 | grep "ERROR"
grep: (standard input): binary file matches

The solution for me was to use grep -a 'pattern'.

from grep's man page:

-a, --text Process a binary file as if it were text; this is equivalent to the --binary-files=text option.

So this works properly for me:

docker logs collector | grep -a 'ERROR'

Answer 4

you can use these:

1. On all shells:

   docker logs nginx 2>&1 | grep "127."
   

2. Only on newer versions of bash works:

   docker logs nginx | &> grep "127"
   

---

What is this syntax?

• Standard input (0)
• Standard output (1)
• Standard error (2)

The > operator actually defaults to using 1 as the file descriptor number, which is why we don't need to specify 1> to redirect standard output: (date 1> now.txt = date > now.txt)

all together

We can redirect multiple streams at once! In this example, we are concatenating two files, redirecting standard output to a file called insects.txt, and redirecting standard error to a file called error.txt.

cat bees.txt ants.txt > insects.txt 2> error.txt

getting fancy

If we wanted to redirect both standard output and standard error to the same file, we could do ls docs > output.txt 2> output.txt

Or we could instead use 2>&1 which is a fancy syntax for saying "redirect standard error to the same location as standard output.

ls docs > output.txt 2>&1

getting fancier

Newer versions of bash also support a fancier syntax for redirecting both standard output and standard error to the same file: the &> notation

ls docs &> output.txt

Answer 5

As vim fan I prefer to use less and search with / (or ? for backwards search)

docker logs nginx 2>&1 | less  

More about vim search here

Answer 6

I generally use it with -f option as well, when I am debugging the issue

docker logs -f nginx 2>&1 | grep "127."

It will show us, what we are expecting in real-time.

To include timestamps, add -t

docker logs -ft nginx 2>&1 | grep "127."

Answer 7

You could also use an anonymous pipe:

docker logs nginx 2> >(grep '127.')

Answer 8

docker logs <container_name> 2>&1 | grep <string>

Answer 9

Additionally, I found it usefull to highlight some terms that I'm searching for in the logs. Especially on productive installations where a lot of log output is generated. In my case I want to highlight COUNT(*) statements. But with a simple grep I can't see the entire SQL statement since it's a multi line statement. This is possible with -E switch and some regex:

For example, the following snippet search for all queries that contain COUNT(*) as well as count(*):

docker logs <containerName> -f | grep --line-buffered -i -E --color "select count\(\*\)|$"

Some explanation:

• docker logs -f tell docker to follow the logs, so the filter applys also to new entrys like when viewing it using tail -f
• greps --line-buffered switch flushes the output on every line, which is required to grep in real time when docker logs -f is used
• -E is an extended regex pattern, required to apply our pattern that allow us returning also the non matching results
• --color highlights the matched parts (seems the default behaviour on my Ubuntu 16.04 LTS, but maybe not on other distributions, so I included it here to be safe)
• * is escaped to disable its special glob functionality, where (, and ) are masked to avoid their regex meaning as group, which is enabled by -E switch

If the container logs to stderr, you can pipe them as Edoardo already wrote for a simple grep:

docker logs <containerName> -f 2>&1 | grep --line-buffered -i -E --color "select count\(\*\)|$"

The -f switch could be omitted if no live grep is wanted. In both cases, you see the entire log buth with highlighted search term like this:

Answer 10

First, use this command ( b1e3c456f07f is the container id ):

docker inspect --format='{{.LogPath}}' b1e3c456f07f

The result will be something like this:

/var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log

Second, use this command ( you can use vim if you like ):

nano /var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log

Answer 11

Run following command to extract container name for image nginx -

docker ps --filter ancestor=nginx

Copy container ID from last command & then extract log path for your container through below command

grep "127." `docker inspect --format={{.LogPath}} <ContainerName>`

Answer 12

To follow up on the comments and clarify this for anyone else hitting this issue. Here is the simplest way I can see to search an nginx container log.

docker logs nginx > stdout.log 2>stderr.log
cat stdout.log | grep "127."

IMO its kinda messy because you need to create and delete these potentially very large files. Hopefully we'll get some tooling to make it a bit more convenient.