CODEWITHSUNDEEP
python-3.xwindowsdockersubprocessubuntu-18.04
Darcey BM
Darcey BM

Reputation: 301

Subprocess can't find file when executed from a Python file in Docker container

I have created a simple Flask app which I am trying to deploy to Docker.

The basic user interface will load on localhost, but when I execute a command which calls a specific function, it keeps showing:

"Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."

Looking at Docker logs I can see the problem is that the file cannot be found by the subprocess.popen command:

"FileNotFoundError: [Errno 2] No such file or directory: 'test_2.bat': 'test_2.bat' 172.17.0.1 - - [31/Oct/2019 17:01:55] "POST /login HTTP/1.1" 500"

The file certainly exists in the Docker environment, within the container I can see it listed in the root directory.

I have also tried changing:

item = subprocess.Popen(["test_2.bat", i], shell=False,stdout=subprocess.PIPE)

to:

item = subprocess.Popen(["./test_2.bat", i], shell=False,stdout=subprocess.PIPE)

which generated the alternative error:

"OSError: [Errno 8] Exec format error: './test_2.bat' 172.17.0.1 - - [31/Oct/2019 16:58:54] "POST /login HTTP/1.1" 500"

I have added a shebang to the top of both .py files involved in the Flask app (although I may have done this wrong):

#!/usr/bin/env python3

and this is the Dockerfile:

FROM python:3.6

RUN adduser lighthouse

WORKDIR /home/lighthouse

COPY requirements.txt requirements.txt
# RUN python -m venv venv
RUN pip install -r requirements.txt
RUN pip install gunicorn

COPY templates templates
COPY json_logs_nl json_logs_nl
COPY app.py full_script_manual_with_list.py schema_all.json ./
COPY bq_load_indv_jsons_v3.bat test_2.bat ./
RUN chmod 644 app.py
RUN pip install flask

ENV FLASK_APP app.py

RUN chown -R lighthouse:lighthouse ./
USER lighthouse

# EXPOSE 5000
CMD ["flask", "run", "--host=0.0.0.0"]

I am using Ubuntu and WSL2 to run Docker on a Windows machine without a virtual box. I have no trouble navigating my Windows file system or building Docker images so I think this configuration is not the problem - but just in case.

If anyone has any ideas to help subprocess locate test_2.bat I would be very grateful!

Edit: the app works exactly as expected when executed locally via the command line with "flask run"

Upvotes: 0

Views: 2077

Answers (1)

Darcey BM
Darcey BM

Reputation: 301

If anyone is facing a similar problem, the solution was to put the command directly into the Python script rather than calling it in a separate file. It is split into separate strings to allow the "url" variable to be iteratively updated, as this all occurs within a for loop:

url = str(i)
var_command = "lighthouse " + url + " --quiet --chrome-flags=\" --headless\" --output=json output-path=/home/lighthouse/result.json"
item = subprocess.Popen([var_command], stdout=subprocess.PIPE, shell=True)
item.communicate()

As a side note, if you would like to run Lighthouse within a container you need to install it just as you would to run it on the command line, in a Node container. This container can then communicate with my Python container if both deployed in the same pod via Kubernetes and share a namespace. Here is a Lighthouse container Dockerfile I've used: https://github.com/GoogleChromeLabs/lighthousebot/blob/master/builder/Dockerfile

Upvotes: 2

Related Questions