Precompilation plan

[henryiii]

Draft for maintainer discussion. Not started yet.

🤖 AI text below 🤖

Optional pre-compilation of pybind11 (nanobind-style static library mode)

Context

pybind11 is header-only; every consumer TU re-parses and re-generates ~1,800 lines of
genuinely non-template implementation code (cpp_function::dispatcher alone is 444 lines,
initialize_generic 214, error_fetch_and_normalize 222, essentially all of
detail/class.h). Goal: an opt-in mode where that code is compiled once into a
static library built inside the consumer's own project (nanobind's model), while
header-only stays the default and byte-for-byte unchanged. Easy from CMake; possible from
Meson/setuptools/any build system.

Prior art: PR #2445 (full fmtlib-style split; gem5 25→15 min; stalled on process, not
design), PR #4001 (minimal pybind11::static target, common.h only; abandoned;
rwgk's blockers — smart_holder in flight, pre-3.0 file org — are now moot), nanobind
(lazy EXCLUDE_FROM_ALL static lib, nb_combined.cpp amalgam for non-CMake builds).
henryiii on record in #4001: "break this up into incremental stand-alone chunks."

Why static-only is safe: per-DSO identity is load-bearing in 4 places
(get_local_internals_key() address-of-local-static key, module_local_load == &local_load self-check, libc++ per-DSO exception classes, loader_life_support
thread_local). A static lib linked into each module preserves all of them per-module; no
export/import macros, no versioned namespace, wheel stays py3-none-any (ships sources).

Decisions (user-confirmed)

1. Scope: high-payoff subset (~1,800 lines), not a full split.
2. Form: static only, built per-consumer-project. Never shared.
3. Non-CMake: combined single-TU .cpp + --srcdir CLI + pkg-config var +
   Pybind11Extension(precompile=True) + Meson docs snippet.

Design

Macros (include/pybind11/detail/common.h)

One user-facing switch: PYBIND11_PRECOMPILED ("definitions were compiled
elsewhere") — accurate for both library TUs and consumer TUs; single macro suffices
because there is no DLL import/export asymmetry.

// Decompose PYBIND11_NOINLINE (common.h:159-168), as PR #4001 did:
#if defined(PYBIND11_NOINLINE_DISABLED)
#    define PYBIND11_NOINLINE_ATTR
#elif defined(_MSC_VER)
#    define PYBIND11_NOINLINE_ATTR __declspec(noinline)
#else
#    define PYBIND11_NOINLINE_ATTR __attribute__((noinline))
#endif
#define PYBIND11_NOINLINE PYBIND11_NOINLINE_ATTR inline

#if defined(PYBIND11_PRECOMPILED)
#    define PYBIND11_INLINE
#else
#    define PYBIND11_INLINE inline
#endif

All three current PYBIND11_NOINLINE expansions are reproduced exactly; no new GCC
inline+noinline warning exposure. PYBIND11_INLINE is unused today (verified).

File layout (fmtlib style, per header)

• include/pybind11/<name>-inl.h next to its header: out-of-line definitions, every one
  tagged PYBIND11_INLINE (or PYBIND11_NOINLINE_ATTR PYBIND11_INLINE). Starts with
  #pragma once + #include "<name>.h".
• Header keeps class definitions with member declarations; at the very bottom:
  #ifndef PYBIND11_PRECOMPILED → #include "<name>-inl.h" → #endif.
  Included at the same TU point the definitions sit today ⇒ default mode unchanged.
• src/<name>.cpp one-liners (#include <pybind11/<name>-inl.h>), plus
  src/pybind11_combined.cpp including all -inl.h in pybind11.h order (non-CMake
  single-TU entry point).
• Rule: only non-template, non-constexpr functions move. Templates
  (load_impl<>, internals_pp_manager<T>, atomic_get_or_create_in_state_dict<T>)
  stay in headers — the latter two need no explicit instantiation: their only
  instantiation points move into the library TU.
• Review rule (grep-able): every function definition in a -inl.h begins with
  PYBIND11_INLINE. Watch decl/def drift: default args stay on the declaration;
  noexcept must match.

Link-time config guard

A function whose name encodes PYBIND11_INTERNALS_VERSION, Py_GIL_DISABLED,
PYBIND11_SIMPLE_GIL_MANAGEMENT, PYBIND11_DETAILED_ERROR_MESSAGES (Debug/Release
mismatch catcher — it defaults on !NDEBUG): declared in detail/internals.h, defined
in internals-inl.h, odr-used from the PYBIND11_MODULE expansion only when
PYBIND11_PRECOMPILED is set. Mismatch or forgot-to-link ⇒ one readable undefined
symbol instead of pages of link errors.

CMake (tools/pybind11Common.cmake — included in both config & subdirectory modes)

UX: pybind11_add_module(target PRECOMPILE ...) keyword + PYBIND11_PRECOMPILE cache
option that flips the default for all calls + public pybind11_precompile() for
raw-target users (target_link_libraries(t PRIVATE pybind11::precompiled)).

Lazy builder (nanobind model — works in installed-config mode, the gap in #4001):

function(pybind11_precompile)
  if(TARGET pybind11_precompiled)
    return()
  endif()
  # FATAL_ERROR under PYBIND11_NOPYTHON (needs Python headers)
  file(GLOB _srcs "${pybind11_SRC_DIR}/*.cpp")
  list(REMOVE_ITEM _srcs "${pybind11_SRC_DIR}/pybind11_combined.cpp")
  add_library(pybind11_precompiled STATIC EXCLUDE_FROM_ALL ${_srcs})
  add_library(pybind11::precompiled ALIAS pybind11_precompiled)
  target_compile_definitions(pybind11_precompiled PUBLIC PYBIND11_PRECOMPILED)
  target_link_libraries(pybind11_precompiled PUBLIC pybind11::headers
                                             PRIVATE pybind11::pybind11)
  set_target_properties(pybind11_precompiled PROPERTIES POSITION_INDEPENDENT_CODE ON
      CXX_VISIBILITY_PRESET hidden VISIBILITY_INLINES_HIDDEN ON)  # only if user unset
  # MSVC: pybind11::windows_extras (/bigobj)
endfunction()

Hook in both tools/pybind11NewTools.cmake (:259 options, after :277-283) and classic
tools/pybind11Tools.cmake (:142, after :161) — classic works free since
pybind11::pybind11 carries Python headers in both modes:

if(ARG_PRECOMPILE OR PYBIND11_PRECOMPILE)
  pybind11_precompile()
  target_link_libraries(${target_name} PRIVATE pybind11::precompiled)
endif()

(PRIVATE link still applies the lib's PUBLIC PYBIND11_PRECOMPILED to the module's TUs.)

Deliberate choices: no variant suffixing (one Python per build tree; multi-config
generators handle Debug/Release natively; must-match macros ride pybind11_headers
INTERFACE); no LTO/OPT_SIZE on the lib (defeats compile-once; per-module keywords
still affect module TUs); lib inherits consumer's directory flags and
CMAKE_CXX_STANDARD at first use (warn if a PRECOMPILE module's standard differs);
not installed/exported.

Packaging / non-CMake

• install(DIRECTORY src/ DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/pybind11/src) →
  wheel path pybind11/share/pybind11/src/*.cpp, sibling of share/cmake; wheel stays
  pure. pybind11_SRC_DIR cache var (subdir mode) + @pybind11_SRCDIR@ in
  tools/pybind11Config.cmake.in.
• commands.py: get_source_dir() (installed-then-repo fallback like get_include);
  __main__.py: --srcdir; tools/pybind11.pc.in: srcdir= variable.
• setup_helpers.py: Pybind11Extension(..., precompile=True) appends
  <srcdir>/pybind11_combined.cpp + defines the macro; hard error if sources absent
  (no silent header-only fallback); guarded so the standalone-copied file still works
  without the kwarg. Per-extension copy; no cross-extension sharing (documented).
• Meson: docs-only snippet — dependency('pybind11').get_variable('srcdir') /
  pybind11_combined.cpp + -DPYBIND11_PRECOMPILED.

PR sequence (each independently mergeable)

0. Cleanup: move PYBIND11_DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION /
   PYBIND11_SIMPLE_GIL_MANAGEMENT from global add_compile_definitions
   (CMakeLists.txt:112-117) onto pybind11_headers INTERFACE (pattern of
   PYBIND11_INTERNALS_VERSION at :306-309).
1. Macro infra (~20 lines, pure refactor): PYBIND11_NOINLINE_ATTR +
   PYBIND11_INLINE in detail/common.h.
2. First split + full build/packaging infra: pytypes.h
   (error_fetch_and_normalize, error_string, raise_from,
   memoryview::from_buffer) + guard symbol + pybind11_precompile() + keyword/option
   hooks + install rules + tests/extra_python_package/test_files.py updates (new
   -inl.h and src/ sets for wheel/sdist/global-wheel + pkg-config template) + two
   tests/test_cmake_build/ cases (subdirectory_precompile, installed_precompile,
   cloned from *_target; the installed one catches the @PACKAGE_INIT@ srcdir path
   computation, the main config-mode risk) + PYBIND11_TEST_PRECOMPILE option in
   tests/CMakeLists.txt (when ON: set(PYBIND11_PRECOMPILE ON) before the module
   loops — link errors give full symbol coverage of the split) + 3 CI jobs (Linux GCC,
   Windows MSVC, macOS; not a matrix dimension) + PYBIND11_TEST_PRECOMPILE: ON in
   the tidy preset so clang-tidy sees the -inl.h TUs. Doxygen: add *-inl.h to
   EXCLUDE_PATTERNS.
3. detail/class.h — cleanest large win (~850 lines, whole file is non-template
   plumbing; slot functions used by address, transparent to move).
4. detail/type_caster_base.h non-template subset — type_caster_generic::cast,
   all_type_info_populate, get_type_info, instance::{de,}allocate_layout,
   loader_life_support, cpp_conduit_method. load_impl<> and all type_caster<T>
   members stay.
5. detail/internals.h + detail/exception_translation.h + common.h
pybind11_fail — translate_exception, get_internals and non-template
   accessors. Document per-DSO semantics: linking the lib into a shared "core" lib
   shared by modules shares local internals — identical to today's behavior with
   inline code in a core lib, not a regression.
6. pybind11.h — the headline: dispatcher, initialize_generic, destruct,
   generate_function_signature, generic_type::initialize, enum_base,
   get_type_override, keep_alive_impl, module-cache helpers. descr.h-constexpr
   machinery stays.
7. CLI/pkg-config/setup_helpers/Meson (any time after PR 2) + extend
   tests/extra_setuptools/test_setuphelper.py with a precompile=True build.
8. Docs + benchmark numbers: docs/compiling.rst new subsection (keyword, option,
   function, caveats: must-match macros, static-only, set standard/defines before first
   use, NOPYTHON unsupported), docs/faq.rst "reduce build time" first recommendation,
   pybind11Config.cmake.in doc header, changelog entry.

Out of scope: embed.h (PyPy), gil.h/attr.h (low payoff), numpy.h/eigen
(template-dominated), shared libpybind11, Meson WrapDB (community, like nanobind).

Verification

• Every PR: cmake --workflow venv (or default preset) both with and without
  -DPYBIND11_TEST_PRECOMPILE=ON; full pytest must pass in both modes. The
  cross-module tests (cross_module_gil_utils,
  cross_module_interleaved_error_already_set, subinterpreter modules) exercise
  exactly the per-DSO claims.
• test_cmake_build covers keyword + config-mode lazy creation; test_files.py
  (nox -s tests_packaging) covers wheel/sdist file sets.
• ODR safety: grep rule (all -inl.h definitions start with PYBIND11_INLINE);
  precompiled CI mode turns any missed definition into a link error across ~50 test
  modules.
• Benchmark (produce the numbers reviewers demanded in Create a static library mode for pybind11 #4001): A/B clean build of
  pybind11_tests (~50 TUs) in Debug (MinSizeRel+LTO understates the win — link
  dominates) and MinSizeRel; incremental touch one test .cpp rebuild;
  -ftime-trace on one TU; existing sosize-*.txt hook watches binary size (expect
  slight shrink). Small driver script or nox session, run on all three platforms;
  optional manual workflow_dispatch CI job, non-blocking.
• prek -a --quiet before each commit.