Skip to content

Filter std:: instantiations out of Halide.dll's export list - #9489

Merged
alexreinking merged 1 commit into
mainfrom
alexreinking/filter-dll-exports
Sep 30, 2026
Merged

alexreinking merged 1 commit into
mainfrom
alexreinking/filter-dll-exports

Conversation

@alexreinking

@alexreinking alexreinking commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS exports every external symbol in Halide's object files, including COMDAT instantiations of std:: templates (e.g. std::vector<Halide::Func>::~vector). The windows-32 (MSVC x86) job is just under link.exe's 65535-export limit on main; #9486 adds one source file and trips LNK1189: library limit of 65535 objects exceeded.

For MSVC shared builds only, Halide now turns off WINDOWS_EXPORT_ALL_SYMBOLS and instead:

  1. writes its object list (including Halide_initmod) with file(GENERATE) from $<TARGET_OBJECTS>,
  2. runs cmake -E __create_def as a PRE_LINK step — the same internal command CMake uses for export-all — to produce the full .def,
  3. runs a CMake script, cmake/FilterDllExports.cmake, which drops symbols whose outermost enclosing scope is std, stdext, Concurrency, llvm or clang, and
  4. links with /DEF:<filtered.def>.

The filter reads the enclosing scope from the MSVC mangling rather than matching substrings, since Halide functions taking std::string also contain @std@@. Anything it can't parse confidently is kept, as are C symbols (halide_*) and everything in Halide::. Consumers instantiate their own copies of std templates, so they don't need them from the DLL. Template argument lists without nested templates are a regular language, so the script repeatedly collapses innermost ?$name@args@ template-ids with a regex until the qualified name is flat, then reads its outermost scope — no character loops (~4 s on 65k synthetic symbols). Export counts are reported only at --log-level=VERBOSE; it warns above 95% of the limit and fails with a clear error (instead of LNK1189) above it. clang-cl, non-Windows and static builds are unchanged.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

Validation: the new filter_dll_exports ctest (runs the script on 41 hand-written mangled names vs. expected output) passes on macOS; the Halide target configures and builds on macOS with the non-Windows path unchanged; forcing the MSVC branch on and inspecting build.ninja showed the PRE_LINK command and /DEF: flag on the link step.

CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS exports every external symbol in
Halide's object files, including COMDAT instantiations of std:: templates
(e.g. std::vector<Halide::Func>::~vector). The windows-32 (MSVC x86) job
is just under link.exe's limit of 65535 exports, so adding a modest
source file trips LNK1189.

For MSVC shared builds only, Halide now turns off
WINDOWS_EXPORT_ALL_SYMBOLS and instead:

  1. writes the object list with file(GENERATE) from $<TARGET_OBJECTS>,
  2. runs `cmake -E __create_def` (what CMake itself runs) as a PRE_LINK
     step to produce the full .def,
  3. runs cmake/FilterDllExports.cmake (`cmake -P`) to drop symbols whose
     enclosing scope is std, stdext, Concurrency, llvm or clang, and
  4. links with /DEF:<filtered.def>.

The filter parses the qualified-name part of the MSVC mangling rather
than matching substrings, since Halide functions taking std::string also
contain "@std@@". Innermost template-ids (whose argument lists are a
regular language) are collapsed with string(REGEX REPLACE) until the
qualified name is flat, then the outermost scope is read with one regex;
this takes ~4s for 65k symbols, where a character loop in CMake takes
over 13s. Anything that cannot be confidently reduced is kept. C symbols
(halide_*) and everything in Halide:: are kept. Consumers instantiate
their own copies of std templates, so they don't need them from the DLL.

Export counts are reported with message(VERBOSE); the script warns above
95% of the limit and fails with a clear error above it.

Note that `__create_def` is an internal CMake command. A ctest
(filter_dll_exports) checks the filter against hand-written mangled names
on all platforms. Non-MSVC and static builds are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alexreinking
alexreinking force-pushed the alexreinking/filter-dll-exports branch from 671054a to a505414 Compare September 30, 2026 12:03
@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.87%. Comparing base (95127ea) to head (a505414).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9489      +/-   ##
==========================================
+ Coverage   70.77%   70.87%   +0.10%     
==========================================
  Files         261      261              
  Lines       79989    79989              
  Branches    19494    19494              
==========================================
+ Hits        56612    56696      +84     
+ Misses      17487    17451      -36     
+ Partials     5890     5842      -48     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking marked this pull request as ready for review September 30, 2026 15:00

@derek-gerstmann derek-gerstmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh man ... good find! We definitely don't want all of std:: !!

@alexreinking
alexreinking merged commit 3fcc674 into main Sep 30, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants