CODEWITHSUNDEEP
pythondocker
tambel
tambel

Reputation: 166

Python docker container gets empty file after bind mount in ~50% of cases

I am trying to mount the file with the bind option into the container but it works very unstable.

The example below creates a file, mounts it into the container and prints the content of this file.

import docker
import docker.types
import tempfile

def foo():

    temp_dir = tempfile.TemporaryDirectory()
    file_path = temp_dir.name + "/file.txt"
    with open(file_path, "w") as f:
        f.write("Hello")

    docker_client = docker.DockerClient()
    output = docker_client.containers.run(
        image="alpine",

        mounts=[
            docker.types.Mount("/file.txt", file_path, type="bind")
        ],
        command="cat /file.txt"
    )
    print(output)

for _ in range(10):
    foo()

Here are the results of the execution: 

b''
b'Hello'
b''
b''
b''
b''
b'Hello'
b'Hello'
b'Hello'
b'Hello'

This random behaviour really confuses me. I have tried various options but it stil has the same results.

what am I doing wrong?

Thanks in advance.

UPD:

To be more clear, I should say that I have tried:

Upvotes: 0

Views: 553

Answers (2)

tambel
tambel

Reputation: 166

Actually, the mounted file is not empty and containers works fine, the problem is in the unstable output of the DockerClient.containers.run function. What a surprise:)

Here is an example. It is not so elegant but should work.

import docker
import docker.types
import tempfile
import os


def foo():
    temp_dir = tempfile.TemporaryDirectory()
    file_path = temp_dir.name + "/file.txt"
    with open(file_path, "w") as f:
        f.write("Hello")

    output_path = temp_dir.name + "/output"

    os.mkdir(output_path)

    docker_client = docker.DockerClient()
    docker_client.containers.run(
        image="alpine",

        mounts=[
            docker.types.Mount("/file.txt", file_path, type="bind"),
            docker.types.Mount("/output", output_path, type="bind")  # mount new directory to store copy result.
        ],
        command="cp /file.txt /output/checkfile.txt",  # copy first file into mounted directory
    )

    checkfile_path = output_path + "/checkfile.txt"

    with open(checkfile_path) as f:
        print(f.read())


for _ in range(20):
    foo()

Output: 

Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello

Upvotes: 0

jkr
jkr

Reputation: 19310

It looks like the file is not being written fully to disk before you run the container. If you write the file once initially and then run the container, it will print the contents every time. Here is the output in ipython:

In [7]: !echo "Hello" > /tmp/file.txt                                                                                                                                           

In [8]: import docker 
   ...: import docker.types 
   ...:  
   ...: def foo(file_path): 
   ...:     docker_client = docker.DockerClient() 
   ...:     output = docker_client.containers.run( 
   ...:         image="alpine", 
   ...:         mounts=[ 
   ...:             docker.types.Mount("/file.txt", file_path, type="bind") 
   ...:         ], 
   ...:         command="cat /file.txt" 
   ...:     ) 
   ...:     print(output) 
   ...:  
   ...: for _ in range(10): 
   ...:     foo("/tmp/file.txt") 
   ...:                                                                                                                                                                         
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'
b'Hello\n'

This is odd, however, because forcing a write to disk does not solve the problem. See https://docs.python.org/3/library/os.html#os.fsync

In [1]: import docker 
   ...: import docker.types 
   ...: import tempfile 
   ...: import os 
   ...:  
   ...: def foo(): 
   ...:  
   ...:     temp_dir = tempfile.TemporaryDirectory() 
   ...:     file_path = temp_dir.name + "/file.txt" 
   ...:     with open(file_path, "w") as f: 
   ...:         f.write("Hello") 
   ...:         f.flush()  # Added. 
   ...:         os.fsync(f.fileno())  # Added. 
   ...:  
   ...:     docker_client = docker.DockerClient() 
   ...:     output = docker_client.containers.run( 
   ...:         image="alpine", 
   ...:  
   ...:         mounts=[ 
   ...:             docker.types.Mount("/file.txt", file_path, type="bind") 
   ...:         ], 
   ...:         command="cat /file.txt" 
   ...:     ) 
   ...:     print(output) 
   ...:  
   ...: for _ in range(10): 
   ...:     foo() 
   ...:                                                                                                                                                                         
b''
b''
b''
b'Hello'
b''
b''
b''
b''
b''
b'Hello'

Upvotes: 1

Related Questions