How to install pyodbc on Dockerfile

Question

I'm trying to install pyodbc on Django to access Sql Server but the Docker image had no be built.

The Dockerfile:

FROM --platform=linux/amd64 python:3.8-slim-buster
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -y && apt-get install -y gcc curl gnupg
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update -y && apt-get install -y unixodbc unixodbc-dev tdsodbc freetds-common freetds-bin freetds-dev postgresql
RUN apt-get update && ACCEPT_EULA=Y apt-get -y install mssql-tools msodbcsql17
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN apt-get update
RUN mkdir /djangonoguero
COPY ./project /djangonoguero/
COPY ./requirements.txt /djangonoguero/
COPY odbcinst.ini /etc/
COPY odbc.ini /etc/
COPY freetds.conf /etc/freetds/
WORKDIR /djangonoguero
RUN pip install --no-cache-dir -r requirements.txt
EXPOSE 8000

I get the output error through docker cli terminal

=> ERROR [16/16] RUN pip install --no-cache-dir -r requirements.txt                                                                                                                                                                      155.7s
------
> [16/16] RUN pip install --no-cache-dir -r requirements.txt:
...
#20 93.38 Building wheels for collected packages: pyodbc, dj-static, static3, django-ckeditor, odfpy, anyjson, django-celery, pyftpdlib
#20 93.38   Building wheel for pyodbc (setup.py): started
#20 96.49   Building wheel for pyodbc (setup.py): finished with status 'error'
#20 96.49   ERROR: Command errored out with exit status 1:
#20 96.49    command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',     '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_ty0puya
#20 96.49        cwd: /tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/
#20 96.49   Complete output (13 lines):
#20 96.49   running bdist_wheel
#20 96.49   running build
#20 96.49   running build_ext
#20 96.49   building 'pyodbc' extension
#20 96.49   creating build
#20 96.49   creating build/temp.linux-x86_64-3.8
#20 96.49   creating build/temp.linux-x86_64-3.8/tmp
#20 96.49   creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm
#20 96.49   creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138
#20 96.49   creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src
#20 96.49   gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPYODBC_VERSION=3.0.10 -DPYODBC_UNICODE_WIDTH=4 -DSQL_WCHART_CONVERT=1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include -I/usr/local/include/python3.8 -c /tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src/sqlwchar.cpp -o build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src/    sqlwchar.o -Wno-write-strings
#20 96.49   gcc: error trying to exec 'cc1plus': execvp: No such file or directory
#20 96.49   error: command 'gcc' failed with exit status 1
#20 96.49   ----------------------------------------
#20 96.49   ERROR: Failed building wheel for pyodbc
#20 96.49   Running setup.py clean for pyodbc

...

#20 133.2 Failed to build pyodbc
#20 135.2 Installing collected packages: wcwidth, vine, six, prompt-toolkit, click, amqp, urllib3, sqlparse, pytz, python-dateutil, kombu, jmespath, docutils, click-repl, click-plugins, click-didyoumean, billiard, text-unidecode, Django, celery, botocore, static3, s3transfer, pyodbc, pyftpdlib, Pillow, numpy, idna, Faker, et-xmlfile, django-js-asset, django-celery, django-appconf, defusedxml, chardet, certifi, requests, reportlab, psycopg2-binary, ping3, pandas, openpyxl, odfpy, honcho,     gunicorn, factory-boy, djangorestframework, django-widget-tweaks, django-storages, django-queued-storage, django-mssql-backend, django-ftpserver, django-filter, django-environ, django-cors-headers, django-ckeditor, dj-static, boto3, boto, asgiref, anyjson
#20 151.2     Running setup.py install for pyodbc: started
#20 154.3     Running setup.py install for pyodbc: finished with status 'error'
#20 154.3     ERROR: Command errored out with exit status 1:
#20 154.3      command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',     '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-fgdyreqq/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/pyodbc
#20 154.3          cwd: /tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/
#20 154.3     Complete output (13 lines):
#20 154.3     running install
#20 154.3     running build
#20 154.3     running build_ext
#20 154.3     building 'pyodbc' extension
#20 154.3     creating build
#20 154.3     creating build/temp.linux-x86_64-3.8
#20 154.3     creating build/temp.linux-x86_64-3.8/tmp
#20 154.3     creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm
#20 154.3     creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138
#20 154.3     creating build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src
#20 154.3     gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPYODBC_VERSION=3.0.10 -DPYODBC_UNICODE_WIDTH=4 -DSQL_WCHART_CONVERT=1 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include -I/usr/local/include/python3.8 -c /tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src/sqlwchar.cpp -o build/temp.linux-x86_64-3.8/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/src/    sqlwchar.o -Wno-write-strings
#20 154.3     gcc: error trying to exec 'cc1plus': execvp: No such file or directory
#20 154.3     error: command 'gcc' failed with exit status 1
#20 154.3     ----------------------------------------
#20 154.3 ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6hbca4vm/pyodbc_01637cbf594442cbbc4aac8a8305e138/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code =     f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-fgdyreqq/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/pyodbc Check the logs for full command output.
#20 155.0 WARNING: You are using pip version 21.2.4; however, version 22.0.3 is available.
#20 155.0 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c pip install --no-cache-dir -r requirements.txt]: exit code: 1

The problem should be with some library but i don't know if is Dockerfile or python requirements level and i can't solve it.

Anybody could help me please ? Thanks in advance.

Answer 1

My current Dockerfile for Debian 12

FROM python:3.12-slim-bookworm
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        gnupg \
        g++ \
        unixodbc-dev \
        curl \
    && curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg \
    && curl https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql-release.list \
    && apt-get update \
    && ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql18 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

Answer 2

Supposed Minimal Set Of RUN Statements To Install pyodbc
In A Docker Image And To Run Correctly In A Docker Container

This particular question is important to me. Previously, I had used similar run statements to those proposed above, but it seemed that a good number of those statements would be for actually installing mssql-server into a docker image rather than just being able to use a pyodbc import in a python script.

I believe that the code lines below are the minimal needed statements to make sure that pyodbc will install correctly to the docker image and run correctly in the docker container once launched. I'd be happy if someone can reduce these further to an even smaller number of installs to the image.

# Stuff for pyodbc to install correctly in image and run correctly in container
ENV ACCEPT_EULA=Y
RUN apt-get update -y \
  && apt-get install -y --no-install-recommends curl gcc g++ gnupg unixodbc-dev

RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
  && curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list \
  && apt-get update \
  && apt-get install -y --no-install-recommends --allow-unauthenticated msodbcsql17 mssql-tools

Answer 3

On Debian 11 this work for me:

RUN apt install curl
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN exit
RUN apt-get update
RUN echo msodbcsql18 msodbcsql/ACCEPT_EULA boolean true | debconf-set-selections
RUN ACCEPT_EULA=Y apt-get install -y mssql-tools18
RUN echo 'export PATH="$PATH:/opt/mssql-tools18/bin"' >> ~/.bashrc \
    && /bin/bash -c "source ~/.bashrc"

For DRIVER = ODBC 18 Driver for SQL

Answer 4

Compiler is simply complaining about a build time dependency, cc1 tool should be in your system to build pyodbc.

In Ubuntu you can solve this with

sudo apt-get update
sudo apt-get install --reinstall build-essential

For your Dockerfile this seems to work fine, I have added build-essential on line 5

FROM --platform=linux/amd64 python:3.8-slim-buster
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -y && apt-get install -y gcc curl gnupg build-essential
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update -y && apt-get install -y unixodbc unixodbc-dev tdsodbc freetds-common freetds-bin freetds-dev postgresql
RUN apt-get update && ACCEPT_EULA=Y apt-get -y install mssql-tools msodbcsql17
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN apt-get update
RUN mkdir /djangonoguero
COPY ./project /djangonoguero/
COPY ./requirements.txt /djangonoguero/
COPY odbcinst.ini /etc/
COPY odbc.ini /etc/
COPY freetds.conf /etc/freetds/
WORKDIR /djangonoguero
RUN pip install --no-cache-dir -r requirements.txt
EXPOSE 8000

PS: If you feel little adventurous you can, try to install packages one by one, build-essential contains a lot of tools; from here you can squeeze image size.