CODEWITHSUNDEEP
pythonc++dllctypesloadlibrary
Righteous Mullet
Righteous Mullet

Reputation: 184

Load a DLL with dependencies in Python

I have proj1.dll that has a dependency to another DLL, proj2.dll. I compiled proj1.dll against the import library that was output by the compiler when compiling proj2.dll in VS2013. I also exported the public functions I am interested in using. So now I have two separate DLLs that all conform to the 'cdll' standard.

I want to use proj1.dll in Python but I am running into the following issue:

import ctypes

# Crashes saying no entry point for "some_func" in proj2.dll
ctypes.cdll.LoadLibrary("C:\myfolder\proj1.dll")

ctypes.cdll.LoadLibrary("C:\myfolder\proj2.dll") # Loads fine
ctypes.cdll.LoadLibrary("C:\myfolder\proj1.dll") # Loads fine if proj2 is loaded first

Calling into this DLL from Python previously worked when I built proj2 as a static library and linked against it in proj1. The two DLLs exist in the same folder. I even attempted adding the folder's path to my PATH environment variable to see if this was a pathing issue but nothing changed.

I was under the assumption that Windows would load proj1.dll and then load the dll's dependencies. Am I wrong? Does the caller (Python) have to load the dependency DLLs first? Anyone know why this is happening?

Upvotes: 8

Views: 10375

Answers (3)

CristiFati
CristiFati

Reputation: 41147

Listing [Python 3.Docs]: ctypes - A foreign function library for Python.

Given the time the question was asked, Python 3.8 can be ruled out (but [SO]: PyWin32 and Python 3.8.0 (@CristiFati's answer) might still be interesting).
Basically, it's the same as [SO]: Can't import dll module in Python (@CristiFati's answer).

For .dll loading generics, [SO]: Python Ctypes - loading dll throws OSError: [WinError 193] %1 is not a valid Win32 application (@CristiFati's answer) might contain useful info.

Now, the question isn't exactly a MCVE or REPREX ([SO]: How to create a Minimal, Reproducible Example (reprex (mcve))), but I assume that if it was, the answer would probably be obvious, and therefore no need for the question :)

Investigation

The error is ERROR_PROC_NOT_FOUND (127, 0x7F).

Note: A .dll is loaded only after all its dependencies have been successfully loaded (recursively). If one of the dependencies fails, the failure is propagated all the way to the top, and the (original) .dll load fails with that error.

What happens (automatically) in the 2 cases:

  • Success:

    1. proj2 is found and attempted to be loaded. It succeeds

    2. proj1 is found attempted to be loaded

      1. As it depends on proj2, that one is attempted to be loaded 1st

      2. proj2 is already in memory (from #1.)

      3. Functions from proj2 that are required by proj1 are searched in proj2. They are all found


      As a consequence, this step (#2.) also succeeds

  • Failure:

    1. proj1 is found attempted to be loaded

      1. As it depends on proj2, that one is attempted to be loaded 1st

      2. proj2 was found and loaded (if it wasn't, the error would have been ERROR_MOD_NOT_FOUND)

      3. Functions from proj2 that are required by proj1 are searched in proj2. Apparently, (at least) one, is not found and the program crashes

From the above, one can only conclude that there is a proj2.dll somewhere in the %PATH% and that one is being automatically loaded by the OS when loading proj1.dll.
It isn't C:\myfolder\proj2.dll. It might be an older version that doesn't export the required function, or another one totally unrelated.

I managed to reproduce the crash with 2 simple (dependent) .dlls and an .exe (if required, I'll also post the code):

img00

The (simplest) solution is to add "C:\myfolder" which is proj2.dll (the right)'s directory to %PATH%, before loading proj1.dll. Check [MS.Learn]: Dynamic-Link Library Search Order, and bear in mind the following aspects:

  • Appending it (at the end) will not do, as the wrong .dll dir is already in %PATH% (before the right's one), and the wrong .dll will be found (and loaded) 1st

  • You'll have to add it before the wrong one's dir (or even better: remove the wrong one's dir from %PATH% altogether). To check where it is, use (in the cmd) terminal:

    where proj2.dll

  • Since I don't know where the wrong .dll dir position is in the %PATH%, adding the right .dll dir at the beginning of %PATH% would be OK. Personally, I don't consider prepending directories before Win system ones a good practice

  • If however proj2.dll is located under "%SystemRoot%\System32", you'll have to remove (or rename) it (the file)

Anyway, %PATH% altering can be done either:

  • Before starting Python:

    set PATH=C:\myfolder;%PATH%

  • From Python itself:

    os.environ["PATH"] = "C:\myfolder;" + os.environ["PATH"]

    For Python 3.8, check the related URL at the beginning of the answer

Might also want to check:

Upvotes: 3

tyleax
tyleax

Reputation: 1788

You need to make sure your environment path contains the path to the dependencies. This will work.

import os
from ctypes import *

path_to_deps = "C:\\myfolder"
os.environ['PATH'] = path_to_deps + os.pathsep + os.environ['PATH']
d = CDLL("C:\myfolder\proj2.dll")

Update:

In python 3.8 update notes the call os.add_dll_directory(path) has been added and is used for specifying dll directories to search in.

Upvotes: 5

d0d0
d0d0

Reputation: 44

Maybe Build settings need to be changed for your project..

See if this helps: Copying a DLL's dependencies in Visual Studio

Upvotes: 0

Related Questions