NestJS in Docker can't do a Prisma Migrate on Postgres in another Docker Container

Question

I have a problem with creating a Docker Image for a NestJS application talking via Prisma to a Postgress database already running in another Container. The problem is that is can't reach the database during the 'prisma generate' phase in the Docker Build.

That's the short version. :-)

docker-compose Database

First, the docker-compose of the database which is running fine after a 'docker-compose up -d':

version: '3.9'

services:
  db:
    image: postgres:latest
    restart: "no"
    container_name: hwpostgres
    volumes:
      - ./database:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: root
      POSTGRES_USER: root
      POSTGRES_DB: taskDb

networks:
  default:
    external:
      name: my-network

docker-compose of the API

There is another docker-compose file building the NestJS API application:

version: '3.9'

services:
  api:
    build:
      context: ./build
      dockerfile: Dockerfile_api
    image: hwapi
    restart: "no"
    container_name: hwapi
    environment:
      DATABASE_URL: postgresql://root:root@hwpostgres:5432/tasksDb?schema=public
    ports:
      - "8080:3001"
    command: ["node", "dist/main.js"]

networks:
  default:
    external:
      name: cops-net

Dockerfile_api

The Dockerfile_api looks like this:

FROM node:latest As development
ARG DATABASE_URL=postgresql://root:root@hwpostgres:5432/tasksDb?schema=public
ENV DATABASE_URL $DATABASE_URL

WORKDIR /usr/src/app

COPY . .

RUN npm install -g npm@7.6.3
RUN npm install
RUN npx prisma migrate dev --name init --preview-feature
RUN npm run build

This Dockerfile_api shown here is obviously the first stage of a multi-part Dockerfile, but the second part is not interesting for this problem description.

The problem is that the 'npx prisma migrate' command fails because it can't find the database. Output of that section of the build process:

Step 8/9 : RUN npx prisma migrate dev --name init --preview-feature
 ---> Running in 938d6538806a
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "tasksDb", schema "public" at "hwpostgres:5432"

Error: P1001: Can't reach database server at `hwpostgres`:`5432`

Please make sure your database server is running at `hwpostgres`:`5432`.

So, it says there's something wrong with the database

Same thing interactively

When I change the Dockerfile_api to the following:

FROM node:latest As development
ARG DATABASE_URL=postgresql://root:root@hwpostgres:5432/tasksDb?schema=public
ENV DATABASE_URL $DATABASE_URL

WORKDIR /usr/src/app

COPY . .

RUN npm install -g npm@7.6.3
RUN npm install
# RUN npx prisma migrate dev --name init --preview-feature
# RUN npm run build

And change the command in the docker-compose.yml for this Dockerfile to

    command: ["sleep", "3650d"]

Then the container keeps on running after finishing the Build.

And then I go into the created Docker Container (docker exec -it hwapi /bin/bash), and then do the 'npm prisma migrate' command, it all works !!

Output of that:

/usr/src/app# echo $DATABASE_URL
postgresql://root:root@hwpostgres:5432/tasksDb?schema=public
/usr/src/app# npx prisma migrate dev --name init --preview-feature
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "tasksDb", schema "public" at "hwpostgres:5432"

Already in sync, no schema change or pending migration was found.
/usr/src/app#

So, here it looks like it is able to find the Container running the database.

Why is it that it can't find the database during the Build phase and it can find the Database when I do the Prisma Migrate in the interactive version of the same Container?

Answer 1

A Docker image build isn't able to see most of the things in the Compose setup. In particular, the Docker network or the other containers described in docker-compose.yml may not exist yet, and an image build can't connect to them. You need to run migrations at some other point, either manually or during container startup.

One capability Docker has in general is to run a container but replace the image's CMD with another command. In Compose, you can docker-compose run a temporary container based on something in the Compose file but replacing its command. You could use this to run migrations manually:

docker-compose run api \
  npx prisma migrate dev --name init --preview-feature
docker-compose up -d

If you wanted to unconditionally run migrations on every container startup, I'd use an entrypoint wrapper script. This is a short shell script that does first-time setup, then uses exec "$@" to run the main container command. Make sure the script is executable (chmod +x entrypoint.sh) before checking it in to source control.

#!/bin/sh

# Run database migrations
npx prisma migrate dev --name init --preview-feature

# Run the main container command
exec "$@"

In your Dockerfile, make this script be the ENTRYPOINT (it must use JSON-array syntax), and the last line will run the CMD (or a command: or docker-compose run override).

COPY . . # includes entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
CMD ["node", "dist/main.js"]

Answer 2

also the answer is correct, i just wanted to add that in my case, with the same problem, i just had to add the image to the host network, by writing this in the docker-compose.yml file:

network_mode: "host"

then, we can connect to the host database, and we can do the rest of the script. so, to conclude, if you dont want to use postgres as a separate image, but in the host linux machine, then you can use the code provided.

Answer 3

The trick is that when you run the docker container inside of your docker-compose setup, you run it inside of the same docker network as your database.

When you run docker build, you're not inside of the same docker network as your database, so it can't find it.

Personally, I would suggest not running the migrate step as part of your image build but instead running it as part of the container startup process - perhaps in a custom ENTRYPOINT script.

If you want to keep the migration as part of your image build, you can try running docker build --network=my-network, where my-network is the name of the network you've attached your running database to.