CODEWITHSUNDEEP
dockerdockerfileollama
Sun Bee
Sun Bee

Reputation: 1820

Ollama in Docker pulls models via interactive shell but not via RUN command in the dockerfile

I have Ollama running in a Docker container that I spun up from the official image. I can successfully pull models in the container via interactive shell by typing commands at the command-line such as:

ollama pull nomic-embed-text

This command pulls in the model: nomic-embed-text.

Now I try to do the same via dockerfile:

FROM ollama/ollama

RUN ollama pull nomic-embed-text

# Expose port 11434
EXPOSE 11434

# Set the entrypoint
ENTRYPOINT ["ollama", "serve"]

and get

Error: could not connect to ollama app, is it running?
------
dockerfile:9
--------------------
   7 |     COPY . .
   8 |     
   9 | >>> RUN ollama pull nomic-embed-text
  10 |     
  11 |     # Expose port 11434
--------------------
ERROR: failed to solve: process "/bin/sh -c ollama pull nomic-embed-text" did not complete successfully: exit code: 1

As far as I know, I am doing the same thing but it works in one place and not another. Any help?

Based on the error message, I updated the dockerfile to launch the ollama service before pulling the model. No change.

FROM ollama/ollama

# Expose port 11434
EXPOSE 11434

RUN ollama serve &
RUN ollama pull nomic-embed-text

It gave same error message.

Upvotes: 7

Views: 10864

Answers (4)

mcdaqc
mcdaqc

Reputation: 41

This entrypoint.sh script worked for me:

#!/bin/bash

# Start Ollama in the background
/bin/ollama serve &
# Record Process ID
pid=$!

# Pause for Ollama to start
sleep 5

# Extract model name from MODEL variable (removing quotes if present)
MODEL_NAME=$(echo $MODEL | tr -d '"')

# Check if MODEL_NAME has a value
if [ -z "$MODEL_NAME" ]; then
    echo "❌ No model specified in MODEL environment variable"
else
    # Check if model exists
    if ollama list | grep -q "$MODEL_NAME"; then
        echo "🟢 Model ($MODEL_NAME) already installed"
        touch /tmp/ollama_ready  # Creates a temporary file to signal readiness
    else
        echo "🔴 Retrieving model ($MODEL_NAME)..."
        # Attempt to pull model and verify before creating the ready flag
        if ollama pull "$MODEL_NAME" 2>/dev/null && ollama list | grep -q "$MODEL_NAME"; then
            echo "🟢 Model download complete!"
            touch /tmp/ollama_ready  # Mark readiness after successful download
        else
            echo "❌ Error downloading model ($MODEL_NAME)"
        fi
    fi
fi

# Wait for Ollama process to finish
wait $pid

And in my docker-compose.yml:

ollama:
    image: ollama/ollama:${OLLAMA_HARDWARE:-latest}
    ports:
      - "11434:11434"
    env_file:
      - .env
    environment:
      - OLLAMA_BASE_URL=${OLLAMA_BASE_URL:-}
    volumes:
      - ollama_data:/root/.ollama
      - ./docker/ollama/entrypoint.sh:/entrypoint.sh
    networks:
      - app-network
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    tty: true
    restart: always
    entrypoint: ["/usr/bin/bash", "/entrypoint.sh"]
    healthcheck:
      test: 
        - "CMD-SHELL"
        - |
          test -f /tmp/ollama_ready && \
          bash -c '</dev/tcp/localhost/11434'  # Checks if Ollama is accepting connections
      interval: 10s
      timeout: 10s
      retries: 100
      start_period: 10s
  • Temporary file (/tmp/ollama_ready): Used as a flag to indicate that the model is ready, so the health check knows when Ollama is initialized and the model is listed/downloaded.
  • /dev/tcp/localhost/11434: This tests if Ollama's API is accessible on port 11434.

Hope this helps!

Upvotes: 1

JspBack
JspBack

Reputation: 11

My solution are almost same with some difference

entrypoint.sh:

#!/bin/bash

echo "Starting Ollama server..."
ollama serve &
SERVE_PID=$!

echo "Waiting for Ollama server to be active..."
while ! ollama list | grep -q 'NAME'; do
  sleep 1
done

ollama pull llama3.2
ollama pull llama3.2-vision

wait $SERVE_PID

Dockerfile:

FROM ollama/ollama:latest

COPY ./entrypoint.sh /entrypoint.sh

RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

Docker-compose(if needed):

    ollama:
    container_name: ollama
    build:
      context: .
    pull_policy: always
    restart: unless-stopped
    ports:
      - "11434:11434"
    environment:
      - OLLAMA_NUM_PARALLEL=4
      - OLLAMA_MAX_LOADED_MODELS=2
      - OLLAMA_MODELS=/usr/share/ollama/.ollama/models
    healthcheck:
      test: "ollama --version && ollama ps || exit 1"
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    volumes:
      - ollama-models:/usr/share/ollama/.ollama/models

volumes:
  ollama-models:

Upvotes: 1

Miguel
Miguel

Reputation: 407

Before pulling any model, Ollama needs to be running; otherwise, you won't be able to pull any model. This simple workaround worked for me:

#!/bin/bash

echo "Starting Ollama server..."
ollama serve &


echo "Waiting for Ollama server to be active..."
while [ "$(ollama list | grep 'NAME')" == "" ]; do
  sleep 1
done


ollama pull nomic-embed-text
ollama pull phi3

You can run that script from your Dockerfile

Upvotes: 4

kmehow
kmehow

Reputation: 81

Try this:

Dockerfile:

FROM ollama/ollama

COPY ./run-ollama.sh /tmp/run-ollama.sh

WORKDIR /tmp

RUN chmod +x run-ollama.sh \
    && ./run-ollama.sh

EXPOSE 11434

run-ollama.sh:

#!/usr/bin/env bash

ollama serve &
ollama list
ollama pull nomic-embed-text

ollama serve &
ollama list
ollama pull qwen:0.5b-chat

Upvotes: 8

Related Questions