3

I've been stressin for a few days trying to compile a modified version of libuvc on windows and now that I've finally done it, I can't seem to load it on Python. This lib that I've already compiled and successfully imported using the same version of Python on Linux machines, doesn't like w10 at all.

System

  • win 10 64 bit
  • python 3.8 64 bit
  • libusb 1.022
  • libuvc.dll compiled with MinGW64

Problem

When trying the 

import ctypes
import ctypes.util
name = ctypes.util.find_library('libuvc')
lib = ctypes.cdll.LoadLibrary(name)

I get the following error:

Could not find module 'C:\Program Files (x86)\libuvc\lib\libuvc.dll'.
Try using the full path with constructor syntax. 
Error: could not find libuvc!

The issue is that the file exists since it was found by util.find_library, but python doesn't think it is where it is, or maybe the output is just the default. What am I missing here? What could be failing to be unable to not just load the module, but find it? I'm sorry I don't have more output than this.

P.S: I've tried reformatting the string in different ways, but the message doesn't change.

Mad Physicist
  • 76,709
  • 19
  • 122
  • 186
tigonza
  • 59
  • 1
  • 1
  • 4
  • Perhaps something to do with permissions on that file? Give [process monitor](https://docs.microsoft.com/en-us/sysinternals/downloads/procmon) a try and see what happened while accessing that file. – Pedro Rodrigues Dec 13 '19 at 23:15
  • what would be an error message that would indicate permission error? cause there is a lot of feed in process monitor. What am i supposed to be looking for? there is a lot of successful readings of the dll. – tigonza Dec 13 '19 at 23:52
  • Filter by the path you're trying to access. Or try to count the Result column and look for some permission denied or similar – Pedro Rodrigues Dec 14 '19 at 00:40
  • See [this answer](https://stackoverflow.com/a/18650202/235698) for how to use Process Monitor to find a DLL issue. It may be a dependent DLL in a different path it can't find, so limiting by path could miss something. Limit by process and CreateFile and look at the end of the list after trying to load your DLL. – Mark Tolonen Dec 14 '19 at 01:36
  • I've taken the liberty of filing a minor bug with python to resolve the discrepancy between the docs and the actual behavior: https://bugs.python.org/issue42114 – Mad Physicist Oct 21 '20 at 22:14

4 Answers4

8

Starting with Python 3.8, the .dll search mechanism has changed.

According to [Python.Docs]: os.add_dll_directory(path) (emphasis is mine):

Add a path to the DLL search path.

This search path is used when resolving dependencies for imported extension modules (the module itself is resolved through sys.path), and also by ctypes.

So, you could do:

os.add_dll_directory("${path_to_working_dlls_directoy}")

You can check [SO]: PyWin32 and Python 3.8.0 (@CristiFati's answer) (which although it seems very different, has the same cause), for more details.

Mad Physicist
  • 76,709
  • 19
  • 122
  • 186
CristiFati
  • 28,721
  • 9
  • 41
  • 63
4

A year late, but I've figured out what is going on and how to fix it. If you look at the code for ctypes.CDLL on around line 340, you can see that the docs are actually incorrect. The code defines the constructor as

def __init__(self, name, mode=DEFAULT_MODE, handle=None,
             use_errno=False, use_last_error=False, winmode=None):

The docs, however, say winmode=0. If you look at line 358, you can see that it matters quite a bit. When winmode=None, the search mode used by _ctypes.LoadLibrary in line 374 (aliased as _dlopen in line 110) is set to nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS on line 363. This search mode does not appear to respond to changes to os.environ['PATH'], sys.path or os.add_dll_directory.

However, if you bypass that setting by using winmode=0 instead of None, the library appears to load fine. Zero is a valid mode for a full path (as would be nt._LOAD_WITH_ALTERED_SEARCH_PATH). A full list of modes is available here: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa, under the dwFlags parameter.

As @CristiFati indicates, the behavior changed in Python 3.8. This happened because prior to that, the winmode parameter did not exist. Instead, mode was used directly, with a default value of ctypes.DEFAULT_MODE, which happens to correspond to a zero and work on all platforms.

Python bug report to resolve discrepancy: https://bugs.python.org/issue42114

Mad Physicist
  • 76,709
  • 19
  • 122
  • 186
  • Yes, I remember that I went through the *CTypes* code a year ago. You are right, the doc does not reflect what's in the code (hmm, I wonder how come I missed that). However, you're not correct about *os.add\_dll\_directory.* With `winmode=None` things work fine because of *LOAD\_LIBRARY\_SEARCH\_USER\_DIRS* which is sensitive to *AddDllDirectory*. – CristiFati Oct 21 '20 at 22:59
  • With `LOAD_LIBRARY_SEARCH_DEFAULT_DIRS`, the current directory and `PATH` are ignored for the sake of security, but the default search path includes the application directory (of python.exe) and System32 directory, and any directory added via `os.add_dll_directory`. If all dependent DLLs are distributed in the same directory, then nothing special should be needed other than using a qualified path to ensure that it uses the flag `LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR`. Otherwise, for a secure load, as used by default, you have to manually include every required directory via `os.add_dll_directory`. – Eryk Sun Oct 21 '20 at 23:00
  • @CristiFati. It didn't work on win 10. I would have expected the flags to work as you described. Perhaps there's something different in the security that Eryk is referring to. – Mad Physicist Oct 21 '20 at 23:03
  • @MadPhysicist: I just created a small example (that I can add to my answer if you like): *Python* code that loads a *.dll* which depends on another *.dll* (located in a different *dir*). As expected, it doesn't work *OOTB*. In *Python 3.7* adding that *dir* to *%PATH%* fixes it, but not on *Python 3.8*. Adding that same *dir* with *os.add\_dll\_directory* **solves the problem in *Python 3.8***. I am on *Win 10*. Maybe on older *OS* versions it didn't work because the flag does not exist without installing *KB2533623*? – CristiFati Oct 21 '20 at 23:09
  • You have to analyze the dependent DLLs if you want a secure load. Try it first with `winmode=0`. Then in Process Explorer, switch the view in the bottom pane to show the full path of loaded DLLs. All directories except the application directory (of python.exe), the System32 directory, and the directory that contains the DLL that was directly loaded via `ctypes.CDLL` have to be added manually via `os.add_dll_directory` before calling `ctypes.CDLL`. If dependencies are in the DLL directory, make sure the path passed to CDLL is qualified in order to include its directory in the search. – Eryk Sun Oct 21 '20 at 23:14
  • @ErykSun: I was talking about *LOAD\_LIBRARY\_SEARCH\_USER\_DIRS*. – CristiFati Oct 21 '20 at 23:16
  • @ErykSun: I was trying to imagine a scenario that *os.add\_dll\_directory* wouldn't do the trick (as MadPhysicist suggests in the 3rd comment). Probably if running *Python 3.8* on an older *Win* version without that *KB*, the flag would not be present there (and most likely the functionality (search *AddDllDirectory* paths) associated with it would not work either). – CristiFati Oct 21 '20 at 23:20
  • @ErykSun. I think I see your point about dependency analysis. I'll check tomorrow. – Mad Physicist Oct 21 '20 at 23:22
  • @ErykSun. Thanks for the tip. I carefully added all the dependent folders and everything started working as expected. – Mad Physicist Oct 22 '20 at 19:34
  • Please change "does not appear to respond to changes to `os.environ['PATH']`, `sys.path` or `os.add_dll_directory`" to an accurate statement such as "does not include `os.environ['PATH']` and the current working directory". `sys.path` is for Python modules; there's no need to mention it here. Also, note that `mode` doesn't matter in Windows; it used to be passed to but not used by low-level `_ctypes.LoadLibrary`; hence the need for `winmode`. It would also be good to summarize some steps needed to support the secure DLL search path with the default `winmode=None` and `os.add_dll_directory`. – Eryk Sun Oct 22 '20 at 20:58
  • @ErykSun. I'll read through the comments you made, as well as some other stuff tonight and rewrite this answer. – Mad Physicist Oct 22 '20 at 21:08
2

OK so i fixed it, it was required that i changed the working directory to where the script was being executed before loading the dll from the same place.

os.chdir('path_to_working_dlls_directoy')

not entirely sure why this helped though.

tigonza
  • 59
  • 1
  • 1
  • 4
  • Note that this is just a workaround. If there are multiple folders with *.dll*s, it won't work. Try the suggestion from my answer. -1. – CristiFati Feb 14 '21 at 01:56
-1

You can specify the path to the library

import snap7
import struct
from snap7.common import Snap7Library
from snap7.util import *

# If you are using a different location for the library
Snap7Library(lib_location='C:/snap7/snap7.dll')
load_library() #Testing library is correctly <WinDLL 'C:\snap7\snap7.dll', handle 7ff9d5d90000 at 0x1a5a0417640>

plc = snap7.client.Client()
plc.connect("10.112.115.10",0,1)
Machavity
  • 28,730
  • 25
  • 78
  • 91
Hoat23
  • 11
  • 1