CODEWITHSUNDEEP
pythondockerasp.net-core
nv-2008
nv-2008

Reputation: 35

How to run a python script inside a .net core application in a docker container?

I have a .Net 5 Core application which I am converting to a docker container(linux base). This application however calls python script to do some computation. The python script needs stix2 library, so I do an initial setup first to create a virtualenv and install stix2 library like this:

  pip install virtualenv
  virtualenv virtual_env
  virtual_env\Scripts\activate
  pip install stix2
  deactivate

When the python script is called, it reads some json, processes with stix2 library and stores the result in another json file in the same folder. The python script, input, output files reside in the ContentRootPath of the .net application. Currently I am using SlavaGu console launcher to run the python script from within the .net application (DoSomePythonProcess.cs):

using SlavaGu.ConsoleAppLauncher;
.
.
.
string cmdLine = @"/c <contentrootpath>\Python\virtual_env\Scripts\activate "+
                         @"&& python " + <AbsolutePathToPythonScriptFile> + " " + <JSONInputFilePath> + " " + <JSONOutputFilePath> + @"&& <contentrootpath>\Python\virtual_env\Scripts\Scripts\deactivate";
ConsoleApp.Result result = ConsoleApp.Run("cmd", cmdLine);

Now, I am trying to convert this application to docker. Currently, this is my dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
RUN apt-get update -y && apt-get install python3-pip python3 -y && pip3 install stix2
WORKDIR /app
EXPOSE 5070

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["project.Web.csproj", "project.Web/"]
RUN dotnet restore "project.Web.csproj"
COPY . .
WORKDIR "/src/project.Web"
RUN dotnet build "project.Web.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "project.Web.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENV ASPNETCORE_URLS=http://*:5070
ENTRYPOINT ["dotnet", "project.Web.dll"]

Here are my questions:

How do I setup the virtualenv for the python script? Can it be done in dockerfile?

How can I call and execute the python script within this docker container?

Also it looks like the slavagu is not recognized when I am running the container. If that's the case, how can I call the python script within the .net application in a linux docker container?

I have searched online but I am not able to find anything relevant

Any help is much appreciated! Thanks in advance!

##UPDATE## Thanks to suggestions here, I was able to find a solution. Sharing it , in case someone needs it in future:

There is no change in dockerfile. The python will be installed in the container under /usr/bin/python3. You can try "which python3" in the container terminal to check this.

On how to call the python script within the .net file:

install cliwrap nuget package

using CliWrap;
using CliWrap.Buffered;

#eg, to check where python is installed
var pathToPy = await Cli.Wrap("which")
      .WithArguments(new[] { "python3" })
      .ExecuteBufferedAsync();
var resOutput = pathToPy.StandardOutput;
_logger.LogInformation(resOutput);

Upvotes: 2

Views: 5977

Answers (2)

abdusco
abdusco

Reputation: 11101

Create a virtualenv in a python container, install dependencies inside this venv and copy it to your app container.

# prepare python env
FROM python:slim as python
# create a venv and install dependencies, use pip/poetry etc.
RUN python -m venv /venv
COPY requirements.txt .
# make sure you use the pip inside the venv
RUN /venv/bin/python -m pip install -r requirements.txt


# build your dotnet app
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
# ...


# build app container
FROM mcr.microsoft.com/dotnet/aspnet:5.0

COPY --from=build /app/publish .

# copy the python environment we've prepared
COPY --from=python /venv /python

# other configurations
# ...  
ENTRYPOINT ["dotnet", "project.Web.dll"]

Then you can call /python/bin/python <args> from the app. Dependencies will work without problems, because they will be covered by python's default search paths.

Running python scripts from dotnet

Use Alexey Golub's CliWrap library when working with external processes.

var result = await Cli.Wrap("/python/bin/python")
    .WithArguments(new []{"my_python_script.py", "arg1", "arg2"})
    .WithWorkingDirectory("/app/python_scripts")
    .ExecuteBufferedAsync();

var output = result.StandardOutput;
// do something with the output

Upvotes: 6

askids
askids

Reputation: 1706

Though not a direct solution to your question, I would suggest that you reconsider the current design. A better approach may be deploy your python as a separate container with Flask or FastAPI to run as a API and then consume that API from your .Net Core container like an regular REST API. You wont need any custom application launcher or store intermediate result files.

For deployment, you can either chose to have it as a multi-container pod or deploy them as separate pods. So lets say, you add more computational functions into your Python API, you can scale it independently, if you are deploying them as separate pods.

Upvotes: 2

Related Questions