CODEWITHSUNDEEP
docker
shubhamagarwal92
shubhamagarwal92

Reputation: 85

Can I increase shared memory after launching a docker session

If I have a docker session already running, is there any way to increase the shared memory?

  1. I know that we can do docker run -it --shm-size=13g dockertag /bin/bash as shown in https://stackoverflow.com/a/35884597/3776827

or change the docker-compose.yml file.

But if I have a docker session already running, is it still possible to do that? (I have one process running and don't want to stop it)

  1. Where do I put this run command? 
docker build -f ./Dockerfile -t <tag> .
docker tag <tag> <repo>
docker push <repo>

If I do docker run -it --shm-size=13g <tag> /bin/bash , I get inside the docker. Doing docker push after (exiting the docker) this didn't create any effect.

I am trying to solve these errors on pytorch:

Pardon my understanding of docker. I am a newbie to it.

Upvotes: 1

Views: 11804

Answers (1)

mdo123
mdo123

Reputation: 1897

  1. But if I have a docker session already running, is it still possible to do that

The answer is no and yes.

The answer is yes because this is possible if you created a volume when you created the container. You then can increase the size of the mounted /dev/shm and those changes will be reflected in the container without a restart

To demonstrate, in the example below /dev/shm from my machine is mounted as /dev/shm in the container.

First, let's check the size of /dev/shm on my machine

anon@anon-VirtualBox:~$ df -h /dev/shm

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           2.9G   48M  2.9G   2% /dev/shm

Now let's create a docker container mounting in /dev/shm as a volume in the container and check the size of the container's /dev/shm

create the container: 

anon@anon-VirtualBox: docker run -d -v /dev/shm:/dev/shm bash sleep 100000
bdc043c79cf8b1ba64ee8cfc026f8d62f0b609f63cbca3cae9f5d321fe47b0e0

check size of /dev/shm in container:

anon@anon-VirtualBox: docker exec -it bdc043c df -h /dev/shm

Filesystem                Size      Used Available Use% Mounted on
tmpfs                     2.9G     47.9M      2.8G   2% /dev/shm

You can see the size the container matches the size on my machine which verifies we've properly mounted /dev/shm into the container.

Now I'll increase the size of /dev/shm on my machine

anon@anon-VirtualBox:~$ sudo mount -o remount,size=4G /dev/shm
anon@anon-VirtualBox:~$ df -h /dev/shm

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           4.0G   56M  4.0G   2% /dev/shm

Now we can verify the container has been adjusted (without being restarted)

anon@anon-VirtualBox:~$ sudo docker exec -it bdc043c79cf8 df -h /dev/shm
Filesystem                Size      Used Available Use% Mounted on
tmpfs                     4.0G     55.9M      3.9G   1% /dev/shm

The answer is no for you because you've already created a container. The configuration for the container can be modified in /var/lib/docker/containers/<container_id>/hostconfig.json by adjusting the ShmSize, but this requires a restart of the container to take effect. At that point there's no difference in creating a new container and specifying a new size using docker run..

  1. Where do I put this run command?

docker build -f ./Dockerfile -t <tag> .
docker tag <tag> <repo>
docker push <repo>

docker build: this builds the docker image

docker tag: give the image an optional tag (note - this is redundant since you specify a tag in the prior command)

docker push: this pushes the image to a remote registry (an image repository, i.e. https://hub.docker.com/)

These steps are all independent of each other and are used when they need to be used for their purpose. It is optional to tag an image just as it's optional to push an image to a registry. The only requirement to run a docker container is that an image exists to run the container from, hence why you specify the image name in the docker run command. Therefore, to satisfy your answer the docker run command would go after you built the image. It's worth noting though that by default when you run for example docker run bash it looks locally for that image and if it doesn't exist (by default) it will attempt to pull that image from docker hub, i.e. https://hub.docker.com/_/bash. This is why from a fresh install of docker you can run docker run bash and it works without building the bash image first. Your output would look similar to 

docker run bash

Unable to find image 'bash:latest' locally
latest: Pulling from library/bash
050382585609: Pull complete 
7bf5420b55e6: Pull complete 
1beb2aaf8cf9: Pull complete 
Digest: sha256:845d500de4879819b189934ec07de15e8ff8de94b3fd16d2744cbe7eb5ca3323
Status: Downloaded newer image for bash:latest
e4c594907b986af923afe089bdbbac057712b3e27589d12618b3d568df21d533

Upvotes: 8

Related Questions