David Espinosa
Reputation: 879
Docker: Build Debian 11 image, with Python and Google Cloud SDK in it
I want ot build a Docker image with the following features:
- Must have a Debian 11 base image (Mandatory)
- On top of it, I also need installed:
- Python 3.
- spaCy package (There could be others of my choice).
- Google Cloud SDK (I will need this to run commands using
gcloudandgsutil).- Most likely, I will download some files stored in some Cloud Storage bucket.
- Python 3.
I decided to use gcr.io/cloud-marketplace/google/debian11:latest because as far as I understood, that base image should have both Python and Google Cloud SDK already installed on top of it, but it did not, so I started adding what I needed on my own.
My first Dockerfile attempt (defaults) looks as follows:
FROM gcr.io/cloud-marketplace/google/debian11:latest
RUN apt-get update -y
# Suggested by @JohnHanley, in the comments section
RUN apt-get install apt-transport-https ca-certificates gnupg -y
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && \
apt-get update -y && \
apt-get install google-cloud-cli -y
# Back to my original implementation
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN pip install spacy==3.2.1
CMD ["gsutil","--version"]
The error triggered:
------
> [4/6] RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-cli -y:
#6 0.403 deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main
#6 0.403 /bin/sh: 1: curl: not found
#6 0.442 Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
#6 0.453 gpg: no valid OpenPGP data found.
------
executor failed running [/bin/sh -c echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-cli -y]: exit code: 2
My second Dockerfile attempt ("If apt-key command is not supported") looks as follows:
FROM gcr.io/cloud-marketplace/google/debian11:latest
RUN apt-get update -y
# Suggested by @JohnHanley, in the comments section
RUN apt-get install apt-transport-https ca-certificates gnupg -y
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y && \
apt-get install google-cloud-sdk -y
# Back to my original implementation
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN pip install spacy==3.2.1
CMD ["gsutil","--version"]
The error triggered:
------
> [4/6] RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y && apt-get install google-cloud-sdk -y:
#7 0.423 deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main
#7 0.423 /bin/sh: 1: curl: not found
#7 0.974 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
#7 1.130 Get:2 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
#7 1.215 Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
#7 1.373 Get:4 http://packages.cloud.google.com/apt cloud-sdk InRelease [6751 B]
#7 1.420 Err:4 http://packages.cloud.google.com/apt cloud-sdk InRelease
#7 1.420 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05 NO_PUBKEY FEEA9169307EA071
#7 1.431 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [262 kB]
#7 1.590 Reading package lists...
#7 1.985 W: GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05 NO_PUBKEY FEEA9169307EA071
#7 1.985 E: The repository 'http://packages.cloud.google.com/apt cloud-sdk InRelease' is not signed.
------
executor failed running [/bin/sh -c echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y && apt-get install google-cloud-sdk -y]: exit code: 100
If the section where Google Cloud SDK is installed gets commented, the rest works OK. Another important info:
- The base OS:
Edition Windows 10 Pro
Version 22H2
Installed on 10/27/2022
OS build 19045.2251
Experience Windows Feature Experience Pack 120.2212.4180.0
- Docker version:
Docker version 20.10.21, build baeda1f
QUESTIONS:
- What am I doing wrong?
- Is there a better way to do what I want? (Image size can have up to 10GB in size, so optimization might be optional for this case).
UPDATES: This section will include lengthy answers to questions raised in the comment sections.
- 2022-12-02, @JohnHanley suggested in thew comments, to follow Google official documentation (section "Docker Tip") to write the Dockerfiles, which I was already doing, but did not include in the original post; I am including that now. OS and Dockerfile versions are included also.
Upvotes: 0
Views: 3216
Answers (1)
Mazlum Tosun
Reputation: 6572
You can install gcloud cli in your container with the following way :
FROM gcr.io/cloud-marketplace/google/debian11:latest
RUN apt-get update -y
RUN apt install curl -y
ENV PATH=/google-cloud-sdk/bin:$PATH
WORKDIR /
RUN export CLOUD_SDK_VERSION="410.0.0" && \
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
tar xzf google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
rm google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
ln -s /lib /lib64
RUN gcloud config set core/disable_usage_reporting true && \
gcloud config set component_manager/disable_update_check true && \
gcloud config set metrics/environment github_docker_images && \
gcloud -q components install beta kubectl
ENTRYPOINT [ "sh", "-c", "gcloud --version" ]
This part is not mandatory :
RUN gcloud config set core/disable_usage_reporting true && \
gcloud config set component_manager/disable_update_check true && \
gcloud config set metrics/environment github_docker_images && \
gcloud -q components install beta kubectl
Upvotes: 4
Related Questions
- How to install the Google Cloud SDK in a Docker Image?
- Using Docker Buildkit on Google Cloud Build
- Google Cloud Build using volumes on Dockerfile
- Cloud Build Docker not building
- Is this a google cloud build bug? or am I doing Docker wrong?
- How to build a docker image using cloud build with sdk, in local machine without dying trying it
- error using gcp cloud build to create dockerfile
- Google App Engine Standard Python Cloud Build
- Docker Build Using gcloud Configuration
- is it possible to make a docker image from a debian image