CODEWITHSUNDEEP
ironpython
Dan
Dan

Reputation: 5972

IronPython DLL compilation in Linux doesn't work with modules in sub-folders

I'm struggling with an issue where creating an IronPython DLL from Python files in Linux doesn't work when the source Python script has modules in sub-directories. The exact same build from Windows works fine.

I've created a cut-down example which demonstrates this, which can be found here.

That example uses Docker to quickly spin up a Linux container to build the IronPython DLL in Linux. Then I have a bit of C# that I'm running via LINQPad (see RunCompiledRoutine.linq) which tries to run the generated IronPython DLL. This is the bit that's failing (when the IronPython DLL is built in Linux).

To run the example, just run the go.ps1 script.

Sorry, I'm aware that to run that example, it requires both Docker and LINQPad to be installed, so I'll also explain it below...

The main source Python script I'm trying to build is the TestScript.py file, which just contains this ...

import sys
import os

from TestSubDirectory.TestSubModule import TestSubModule

def main():
    TestSubModule().PrintMessage()

So, nothing special - it's just importing a module in a directory called TestSubDirectory, and that module just looks like this...

import sys
import os

class TestSubModule():
    def PrintMessage(self):
        print "*** SUCCESS ***"

So literally just printing a message on the screen.

Then in the Linux container, I'm just doing this to build the compiled IronPython DLL...

mono ./IronPython/ipy.exe ./CompileIntoDll.py

Where CompileIntoDll.py is just doing clr.CompileModules("/app/CompiledPython.dll", mainModule = None, *files) with a list of files to include.

This successfully builds the IronPython DLL. However, when I try to run it from C# (see RunCompiledRoutine.linq), it fails with this ...

ImportException: cannot import TestSubModule from TestSubDirectory
    at IronPython.Runtime.Importer.ImportNestedModule(CodeContext context, PythonModule module, String[] parts, Int32 current, List path)
    at IronPython.Runtime.Importer.ImportModuleFrom(CodeContext context, Object from, String[] parts, Int32 current)
    at IronPython.Runtime.Importer.ImportModule(CodeContext context, Object globals, String modName, Boolean bottom, Int32 level)
    at IronPython.Modules.Builtin.__import__(CodeContext context, String name, Object globals, Object locals, Object fromlist, Int32 level)
    at Microsoft.Scripting.Interpreter.FuncCallInstruction`7.Run(InterpretedFrame frame)
    at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
    at Microsoft.Scripting.Interpreter.LightLambda.Run7[T0,T1,T2,T3,T4,T5,T6,TRet](T0 arg0, T1 arg1, T2 arg2, T3 arg3, T4 arg4, T5 arg5, T6 arg6)
    at IronPython.Runtime.Importer.ImportLightThrow(CodeContext context, String fullName, PythonTuple from, Int32 level)
    at DLRCachedCode.TestScript$1(CodeContext $globalContext, FunctionCode $functionCode)
    at IronPython.Compiler.OnDiskScriptCode.Run()
    at IronPython.Compiler.OnDiskScriptCode.Run(Scope scope)
    at IronPython.Runtime.PythonContext.InitializeModule(String fileName, ModuleContext moduleContext, ScriptCode scriptCode, ModuleOptions options)
    at IronPython.Runtime.ModuleLoader.load_module(CodeContext context, String fullName)
    at Microsoft.Scripting.Interpreter.FuncCallInstruction`4.Run(InterpretedFrame frame)
    at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
    at Microsoft.Scripting.Interpreter.LightLambda.Run4[T0,T1,T2,T3,TRet](T0 arg0, T1 arg1, T2 arg2, T3 arg3)
    at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
    at IronPython.Runtime.PythonContext.Call(CodeContext context, Object func, Object arg0)
    at IronPython.Runtime.Importer.FindAndLoadModuleFromImporter(CodeContext context, Object importer, String fullName, List path, Object& ret)
    at IronPython.Runtime.Importer.TryLoadMetaPathModule(CodeContext context, String fullName, List path, Object& ret)
    at IronPython.Runtime.Importer.ImportTopAbsolute(CodeContext context, String name)
    at IronPython.Runtime.Importer.ImportModule(CodeContext context, Object globals, String modName, Boolean bottom, Int32 level)
    at IronPython.Hosting.PythonService.ImportModule(ScriptEngine engine, String name)
    at IronPython.Hosting.Python.ImportModule(ScriptEngine engine, String moduleName)
    at UserQuery.Main(String[] args) in C:\Code\IronPythonLinuxIssue\RunCompiledRoutine.linq:line 12
    at LINQPad.ExecutionModel.ClrQueryRunner.Run()
    at LINQPad.ExecutionModel.Server.RunQuery(QueryRunner runner)

Note that building the DLL in Windows and running in the same LINQPad script works fine. I'm wondering if somewhere IronPython isn't handling the path separator (slash) properly when run from Linux.

It also works fine using that Linux build if I put the TestSubModule.py into the same folder as the base TestScript.py (ie. not using a sub-directory).

Note that this is using the .NET 4.5 version of IronPython 2.7.8. I tried using the .NET Core 2.0 version, but that fails saying that clr.CompileModules isn't supported.

Upvotes: 2

Views: 424

Answers (1)

Dan
Dan

Reputation: 5972

To answer my own question - I also posted this question as a GitHub issue on the IronPython GitHub, and it turns out that they weren't handling the different directory separators to work across different platforms - so a DLL build in Linux wouldn't work in Windows, and vice-versa.

However, literally the same day, slozier one of their main contributors, made a fix, and it was merged into master! I'll test this on Friday when I'm back in the office.

Upvotes: 0

Related Questions