Skip to content

Conversation

Doekin
Copy link

@Doekin Doekin commented May 15, 2025

This PR addresses issues such as ModuleNotFoundError that could occur during stub generation when extension modules depend on external shared libraries (e.g., .dll on Windows, .so on Linux) located outside of the standard search paths.

Background

On Windows, when stubgen attempts to import an extension module to generate stubs, importlib.import_module() may fail if the Python interpreter cannot locate the required dependent DLLs—for example, if they are not in the same directory as the .pyd file.

Example error:

 Generating my_ext.pyi
 Traceback (most recent call last):
   File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-snyzxcow\overlay\Lib\site-packages\nanobind\stubgen.py", line 1440, in <module>
     main()
   File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-snyzxcow\overlay\Lib\site-packages\nanobind\stubgen.py", line 1367, in main
     mod_imported = importlib.import_module(mod)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ...
 ModuleNotFoundError: No module named 'my_ext'

In contrast, on Linux, the dynamic linker can use the extension module's embedded RPATH to find dependencies.

Proposed Change

A new LIB_PATH argument has been introduced to nanobind_add_stub. This option allows you to specify a list of directories containing dependent shared libraries. During stub generation:

  • On Windows, these directories are added to the DLL search path using os.add_dll_directory().
  • On Linux, the paths are prepended to the LD_LIBRARY_PATH environment variable (on macOS, to DYLD_LIBRARY_PATH).

Example Usage

# Suppose my_ext.pyd links against some_dynamic_lib.dll,
# and some_dynamic_lib.dll is in a separate output directory.
nanobind_add_module(my_ext my_ext.cpp)
target_link_libraries(my_ext PRIVATE some_dynamic_lib)

nanobind_add_stub(
    my_ext_stub
    MODULE my_ext
    OUTPUT my_ext.pyi
    PYTHON_PATH $<TARGET_FILE_DIR:my_ext>
    # Provide the directory containing some_dynamic_lib.dll or .so
    LIB_PATH $<TARGET_FILE_DIR:some_dynamic_lib>
    DEPENDS my_ext
)

@Doekin Doekin force-pushed the stubgen_win_dll branch from 4b84a7b to 0a3bc72 Compare May 15, 2025 11:58
@wjakob
Copy link
Owner

wjakob commented May 20, 2025

I am generally open to this feature, but what bothers me is the windows-specific nature. The two other supported platforms (Linux, macOS) have analogous ways of adding dynamic library search paths.

@Doekin Doekin force-pushed the stubgen_win_dll branch from 0a3bc72 to 1778524 Compare May 20, 2025 03:01
@Doekin
Copy link
Author

Doekin commented May 20, 2025

Thanks for raising that point! I updated the code to handle dependent shared libraries on Linux and macOS as well. The option has been renamed from DLL_PATH to LIB_PATH, and LIB_PATH now prepends the specified paths to LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

@Doekin Doekin changed the title stubgen: add DLL_PATH option to locate dependent DLLs on Windows stubgen: add LIB_PATH option to locate dependent shared libraries May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants