CODEWITHSUNDEEP
pythonpython-3.xdockerpachyderm
aviral sanjay
aviral sanjay

Reputation: 983

File not found even after adding the file inside docker

I have written a docker file which adds my python script inside the container: ADD test_pclean.py /test_pclean.py

My directory structure is:

.
├── Dockerfile
├── README.md
├── pipeline.json
└── test_pclean.py

My json file which acts as a configuration file for creating a pipeline in Pachyderm is as follows:

{
    "pipeline": {
        "name": "mopng-beneficiary-v2"
    },
    "transform": {
        "cmd": ["python3", "/test_pclean.py"],
        "image": "avisrivastava254084/mopng-beneficiary-v2-image-7"
    },
    "input": {
        "atom": {
            "repo": "mopng_beneficiary_v2",
            "glob": "/*"
        }
    }
}

Even though I have copied the official documentation's example, I am facing an error: python3: can't open file '/test_pclean.py': [Errno 2] No such file or directory

My dockerfile is:

FROM    debian:stretch

# Install opencv and matplotlib.
RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install -y unzip wget build-essential \
        cmake git pkg-config libswscale-dev \
    && apt-get clean \
    && rm -rf /var/lib/apt

RUN apt update
RUN apt-get -y install python3-pip
RUN pip3 install matplotlib
RUN pip3 install pandas

ADD test_pclean.py /test_pclean.py
ENTRYPOINT [ "/bin/bash/" ]

Upvotes: 0

Views: 744

Answers (2)

aviral sanjay
aviral sanjay

Reputation: 983

I was not changing the commits to my docker images on each build and hence, Kubernetes was using the local docker file that it had(w/o tags and commits, it doesn't acknowledge any change). Once I started using commit with each build, Kubernetes started downloading the intended docker image.

Upvotes: 0

niharvey
niharvey

Reputation: 2807

Like some of the comments above suggest. It looks like your test_pclean.py file isn't in the docker image. Here's what should fix it.

  1. Make sure your test_pclean.py file is in your docker image by having be included as part of the build process. Put this as the last step in your dockerfile:

COPY test_pclean.py .

  1. Ensure that your pachyderm pipeline spec has the following for the cmd portion: "cmd": ["python3", "./test_pclean.py"]

  2. And this is more of a suggestion than a requirement.... You'll make life easier for yourself if you use image tags as part of your docker build. If you default to latest tag, any future iterations/builds of this step in your pipeline could have negitave affects (new bugs in your code etc.). Therefore the best practice is to use a particular version in your pipeline: mopng-beneficiary-v2-image-7:v1 and mopng-beneficiary-v2-image-7:v2 and so on. That way you can iterate on say version 3 and it won't affect the already running pipeline.

docker build -t avisrivastava254084/mopng-beneficiary-v2-image-7:v1

Then just update your pipeline spec to use avisrivastava254084/mopng-beneficiary-v2-image-7:v1

Upvotes: 1

Related Questions