Repository navigation
[BUG]: Python atexit broken with pybind11 version >= 2.9.2 #4459
Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
2.10.2
Problem description
We want to upgrade from pybind11 release 2.9.0 to 2.10.2, but we encountered an issue in the Python atexit (https://docs.python.org/3/library/atexit.html) functions, that our custom types are not registered anymore. With delta debugging we found that this PR caused the issue: #3744
In my opinion the
detail::get_local_internals().registered_types_cpp.clear();
detail::get_local_internals().registered_exception_translators.clear();
calls should be after the Py_Finalize function call.
When I change the orders of those function calls, the problem is gone.
Reproducible example code
No response
Is this a regression? Put the last known working version here if it is.
2.9.1
Activity
Skylion007 commentedon Jan 18, 2023
FYI @rwgk
rwgk commentedon Jan 19, 2023
@chrreisinger could you help by
.clear()calls after thePy_Finalize()call,The change makes sense to me. My understanding is: the
Py_Finalize()invalidates allPyObject*pointers in thelocal_internals, but looking at thestruct local_internalscode, it seems to me the.clear()calls don't ever use those pointers. There are a bunch of non-trivial destructors that will run, but none of them will call into the Python C API.chrreisinger commentedon Jan 19, 2023
I will give it a try on the weekend.
rwgk commentedon Jan 19, 2023
Awesome.
If you create a PR with just the lines moved, that'll be a great start. It will be good to confirm that valgrind does not find any issues. We can work on the test together from there.
Skylion007 commentedon Jan 29, 2023
@chrreisinger Any updates on this?
Keep registered types until after Py_Finalize(). Fix pybind#4459
bugfix: Keep registered types until after Py_Finalize(). Fix #4459 (#…
Revert "bugfix: Keep registered types until after Py_Finalize(). Fix p…
Skylion007 commentedon Feb 8, 2023
Oh boy, this broke some more things.
Fix pybind#4459 (cpp atexit callbacks) without segfault
henryiii commentedon Jul 25, 2026
🤖 AI text below 🤖
The finalization order was rewritten by #5564.
embed.hnow callsPy_Finalize()before it destroys the internals, which is what this issue asked for. Pythonatexithandlers therefore run while the bindings are still alive.If you can still reproduce this with current master (v3), please comment and we can reopen.