CODEWITHSUNDEEP
airflowpipenv
jamiet
jamiet

Reputation: 12264

airflow install fails when using pipenv

I'm trying to install python package airflow into a virtualenv that has been created using pipenv, inside a docker container. It fails with an error that I'm clueless about.

Here is my Dockerfile: 

FROM python:3.6-stretch

WORKDIR /tmp

# Define build args
ARG http_proxy
ARG https_proxy
ARG no_proxy

RUN apt-get update && \
    apt-get -y install default-jdk

# Detect JAVA_HOME and export in bashrc.
# This will result in something like this being added to /etc/bash.bashrc
#   export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
RUN echo export JAVA_HOME="$(readlink -f /usr/bin/java | sed "s:/jre/bin/java::")" >> /etc/bash.bashrc

# Upgrade pip
RUN pip install --upgrade pip

# Install core python packages
RUN pip install pipenv==2018.5.18

Build and run:
docker build -t pipenvtest:latest .
docker run -it pipenvtest:latest bash

When connected to the container:
pipenv --python 2.7
pipenv install --dev airflow

Which fails with this error:

building '_yaml' extension
creating build/temp.linux-x86_64-2.7/ext
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-2.7.13=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c ext/_yaml.c -o build/temp.linux-x86_64-2.7/ext/_yaml.o
ext/_yaml.c:4:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

(the ^ actually appears at the end of the line preceding it but I don't know how to format the quoted text as such)

I admit to not having the faintest idea how to go about solving this so hoping someone can give me some pointers. I hope the repro that I've included here works for you.

Upvotes: 1

Views: 1285

Answers (2)

jamiet
jamiet

Reputation: 12264

OK, I was being really dumb. I was trying to setup a python2.7 virtualenv on an image built from python:3.6-stretch.

I changed

pipenv --python 2.7

to

pipenv --python 3.6

and it worked.

Upvotes: 1

SergiyKolesnikov
SergiyKolesnikov

Reputation: 7815

Is the --dev switch in pipenv install --dev airflow intended? It instructs pipenv to install development dependencies of Airflow too. One of these dependencies needs the Python.h header file (which is missing). To resolve the problem:

  • If you do not need the development dependencies then remove the --dev switch.
  • If you need the development dependencies then install the libpython2.7-dev package, which provides Pthon.h, before you install Aiflow: apt install libpython2.7-dev

Upvotes: 1

Related Questions