Docker data-only container and dealing with new releases

Question

I've been researching a lot of practices around Docker and have come quite far. But there's one thing that keeps on puzzling me, working with data-only containers.

Here's a brief overview of my current setup:

# nginx
web:
  extends:
    file: _common.yml
    service: web
  ports:
    - "80:80"
  environment:
    APPLICATION_ENV: prod
  volumes_from:
    - data
  links:
    - db
    - redis

# php5-cli
app:
  extends:
    file: _common.yml
    service: app
  environment:
    APPLICATION_ENV: prod
  volumes_from:
    - data
  links:
    - db
    - redis

data:
  image: <censored-url>
  volumes:
    - "/var/lib/mysql"
    - "/app"

# percona
db:
  extends:
    file: _common.yml
    service: db
  volumes_from:
    - data

# redis
redis:
  extends:
    file: _common.yml
    service: redis

The <censored-url> that you see is an image build with this Dockerfile:

FROM busybox
COPY . /app

Now this setup works but I just can't figure out how to handle a new release. My source is in git, when I want to deploy to production I imagine I create a new image (FROM busybox should probably be replaced with my existing image url) and pull in the new image on my production server.

But how do I get the data to update for my web container and such? I also have to make sure my persistent data (/var/lib/mysql) remains.

I hope the question is clear, I'd be happy to clarify when necessary.

Answer 1

Now this setup works but I just can't figure out how to handle a new release. My source is in git, when I want to deploy to production I imagine I create a new image (FROM busybox should probably be replaced with my existing image url) and pull in the new image on my production server.

Apart from the statement about busybox (which I don't follow), that seems pretty much right. Normally, you re-build the images, push to a registry, then pull from the production server. And as @Mario Marin suggests, it's worth being clever about tags, so that you can easily roll-back if needs be and you know exactly which version of your app is deployed.

But how do I get the data to update for my web container and such? I also have to make sure my persistent data (/var/lib/mysql) remains.

I assume this is referring to your data container, which you've done in a bit of an unusual way. To begin with, I would pull out the mysql directory and put it in its own data container. I would use the percona image for this so that all the permissions are set correctly. When you create the data-container, you don't leave it running, so there's no need to worry about the container getting out-of-date; it's really just a namespace for the directory.

The next step is to deal with the app directory, which I assume isn't data, but code? In this case I would include it in your web image (don't use a volume at all). In the Dockerfile I would normally do a git clone to keep the image up-to-date. During development you can mount a volume over the top of the app directory with code from the host so you can make changes instantly.

For more info on data containers, have a look at http://container42.com/2014/11/18/data-only-container-madness/

Answer 2

I would drop the /app directory from the data container and build it with docker-compose:

web:
  build: .
  extends:
    file: _common.yml
    service: web
  ports:
    - "80:80"
  environment:
    APPLICATION_ENV: prod
  links:
    - db
    - redis

app:
  extends:
    file: _common.yml
    service: app
  environment:
    APPLICATION_ENV: prod
  volumes_from:
    - data
  links:
    - db
    - redis

data:
  volumes:
    - "/var/lib/mysql"

db:
  extends:
    file: _common.yml
    service: db
  volumes_from:
    - data

redis:
  extends:
    file: _common.yml
    service: redis

Dockerfile

FROM busybox

ADD . /app

WORKDIR /app

You can use tags for your different releases, here is a script that I use in my deployments

DOCKER_HUB_USER="therightplace"
DOCKER_COMPOSE_IMAGE="projectname_web_1"
APP_IMAGE="nicer_name"
REMOTE_IMAGE=${DOCKER_HUB_USER}/${APP_IMAGE}
IMAGE_TAG=$(date -u +"%Y-%m-%dT%H:%M:%SZ" |sed 's/-\|:/_/g')
TAGGED_IMAGE=${REMOTE_IMAGE}:${IMAGE_TAG}
LATEST_IMAGE=${REMOTE_IMAGE}:latest

build_image () {
    echo "Building image: ${TAGGED_IMAGE}"
    docker-compose build web
}

push_tagged_image () {
    echo ${TAGGED_IMAGE}
    # change docker-compose image tag for a nicer one
    docker tag ${DOCKER_COMPOSE_IMAGE} ${TAGGED_IMAGE}
    # push image out to docker hub
    docker push ${TAGGED_IMAGE} && echo "${TAGGED_IMAGE} image pushed to docker hub" \
    || echo "Failed to push ${TAGGED_IMAGE} image to docker hub"
}

push_latest_image () {
    echo ${LATEST_IMAGE}
    # push image out to docker hub
    docker tag ${TAGGED_IMAGE} ${LATEST_IMAGE}
    docker push ${LATEST_IMAGE} && echo "${LATEST_IMAGE} image pushed to docker hub" \
    || echo "Failed to push ${LATEST_IMAGE} image to docker hub"
}

The script will build the service web and push it to the docker hub. You can omit the pushes and just tag the images.

Instead of using a timestamp for your releases you could use a git hash:

IMAGE_TAG=$(git rev-parse --short HEAD)