Why doesn't Python app print anything when run in a detached docker container?

Question

I have a Python (2.7) app which is started in my dockerfile:

CMD ["python","main.py"]

main.py prints some strings when it is started and goes into a loop afterwards:

print "App started"
while True:
    time.sleep(1)

As long as I start the container with the -it flag, everything works as expected:

$ docker run --name=myapp -it myappimage
> App started

And I can see the same output via logs later:

$ docker logs myapp
> App started

If I try to run the same container with the -d flag, the container seems to start normally, but I can't see any output:

$ docker run --name=myapp -d myappimage
> b82db1120fee5f92c80000f30f6bdc84e068bafa32738ab7adb47e641b19b4d1
$ docker logs myapp
$ (empty)

But the container still seems to run;

$ docker ps
Container Status ...
myapp     up 4 minutes ... 

Attach does not display anything either:

$ docker attach --sig-proxy=false myapp
(working, no output)

Any ideas whats going wrong? Does "print" behave differently when ran in background?

Docker version:

Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.2
Git commit (client): a8a31ef
OS/Arch (client): linux/arm
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.2
Git commit (server): a8a31ef

Answer 1

For me none of the answers here worked.

In order to see messages my Python program was printing to stdout within a Docker container I had to add:

tty: True

for my Python service in my docker compose.yaml

https://docs.docker.com/compose/compose-file/05-services/#tty

Answer 2

For anyone who ends up here, I tried a bunch of these solutions but none worked for me. In the end I needed to propertly configure my logger by adding

logging.basicConfig(level=logging.INFO)

before I instantiated my logger instance

logger = logging.getLogger()

This worked perfectly and I didn't have to make any other changes.

Answer 3

I use visual studio code with docker extension to create Dockerfile in python automatically.

By default it have this line

# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1

it works and show logs when container run in detached mode, so I think that's the preferred way. I use python 3.11

Answer 4

Finally I found a solution to see Python output when running daemonized in Docker, thanks to @ahmetalpbalkan over at GitHub. Answering it here myself for further reference :

Using unbuffered output with

CMD ["python","-u","main.py"]

instead of

CMD ["python","main.py"]

solves the problem; you can see the output now (both, stderr and stdout) via

docker logs myapp

---

why -u ref

- print is indeed buffered and docker logs will eventually give you that output, just after enough of it will have piled up
- executing the same script with python -u gives instant output as said above
- import logging + logging.warning("text") gives the expected result even without -u

what it means by python -u ref. > python --help | grep -- -u

-u     : force the stdout and stderr streams to be unbuffered;

Answer 5

In my case, running Python with -u didn't change anything. What did the trick, however, was to set PYTHONUNBUFFERED=1 as environment variable:

docker run --name=myapp -e PYTHONUNBUFFERED=1 -d myappimage

[Edit]: Updated PYTHONUNBUFFERED=0 to PYTHONUNBUFFERED=1 after Lars's comment. This doesn't change the behavior and adds clarity.

Answer 6

If anybody is running the python application with conda you should add --no-capture-output to the command since conda buffers to stdout by default.

ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "my-app", "python", "main.py"]

Answer 7

When using python manage.py runserver for a Django application, adding environment variable PYTHONUNBUFFERED=1 solve my problem. print('helloworld', flush=True) also works for me.

However, python -u doesn't work for me.

Answer 8

If you aren't using docker-compose and just normal docker instead, you can add this to your Dockerfile that is hosting a flask app

ARG FLASK_ENV="production"
ENV FLASK_ENV="${FLASK_ENV}" \
    PYTHONUNBUFFERED="true"

CMD [ "flask", "run" ]

Answer 9

Try to add these two environment variables to your solution PYTHONUNBUFFERED=1 and PYTHONIOENCODING=UTF-8

Answer 10

If you want to add your print output to your Flask output when running docker-compose up, add the following to your docker compose file.

web:
  environment:
    - PYTHONUNBUFFERED=1

https://docs.docker.com/compose/environment-variables/

Answer 11

I had to use PYTHONUNBUFFERED=1 in my docker-compose.yml file to see the output from django runserver.

Answer 12

Since I haven't seen this answer yet:

You can also flush stdout after you print to it:

import time

if __name__ == '__main__':
    while True:
        print('cleaner is up', flush=True)
        time.sleep(5)

Answer 13

Usually, we redirect it to a specific file (by mounting a volume from host and writing it to that file).

Adding a tty using -t is also fine. You need to pick it up in docker logs.

Using large log outputs, I did not have any issue with buffer storing all without putting it in dockers log.

Answer 14

You can see logs on detached image if you change print to logging.

main.py:

import time
import logging
print "App started"
logging.warning("Log app started")
while True:
    time.sleep(1)

Dockerfile:

FROM python:2.7-stretch
ADD . /app
WORKDIR /app
CMD ["python","main.py"]

Answer 15

See this article which explain detail reason for the behavior:

There are typically three modes for buffering:

• If a file descriptor is unbuffered then no buffering occurs whatsoever, and function calls that read or write data occur immediately (and will block).
• If a file descriptor is fully-buffered then a fixed-size buffer is used, and read or write calls simply read or write from the buffer. The buffer isn’t flushed until it fills up.
• If a file descriptor is line-buffered then the buffering waits until it sees a newline character. So data will buffer and buffer until a \n is seen, and then all of the data that buffered is flushed at that point in time. In reality there’s typically a maximum size on the buffer (just as in the fully-buffered case), so the rule is actually more like “buffer until a newline character is seen or 4096 bytes of data are encountered, whichever occurs first”.

And GNU libc (glibc) uses the following rules for buffering:

Stream               Type          Behavior
stdin                input         line-buffered
stdout (TTY)         output        line-buffered
stdout (not a TTY)   output        fully-buffered
stderr               output        unbuffered

So, if use -t, from docker document, it will allocate a pseudo-tty, then stdout becomes line-buffered, thus docker run --name=myapp -it myappimage could see the one-line output.

And, if just use -d, no tty was allocated, then, stdout is fully-buffered, one line App started surely not able to flush the buffer.

Then, use -dt to make stdout line buffered or add -u in python to flush the buffer is the way to fix it.

Answer 16

As a quick fix, try this:

from __future__ import print_function
# some code
print("App started", file=sys.stderr)

This works for me when I encounter the same problems. But, to be honest, I don't know why does this error happen.