-
-
Notifications
You must be signed in to change notification settings - Fork 32.3k
gh-136421: Fix crash when _datetime is been initialized in multiple sub-interpreters #136422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need a more fundamental fix that prevents static types from being initialized concurrently. This is sort of a known issue, see #129817.
@@ -7344,9 +7344,13 @@ init_static_types(PyInterpreterState *interp, int reloading) | |||
|
|||
/* Bases classes must be initialized before subclasses, | |||
* so capi_types must have the types in the appropriate order. */ | |||
static PyMutex mutex = {0}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the assumption is that this can run concurrently, then we need to also protect the assignments to tp_base
above.
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
PyMutex_Lock(&mutex); | ||
int ret = _PyStaticType_InitForExtension(interp, type); | ||
PyMutex_Unlock(&mutex); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lock needs to be applied to whole the loop or function rather than to a type to avoid a crash here (for me), because all managed static extension types are currently supposed to be indexed by using the same interpreter:
Lines 322 to 329 in db47f4d
else { | |
PyMutex_Lock(&interp->types.mutex); | |
index = interp->types.for_extensions.next_index; | |
interp->types.for_extensions.next_index++; | |
PyMutex_Unlock(&interp->types.mutex); | |
assert(index < _Py_MAX_MANAGED_STATIC_EXT_TYPES); | |
} | |
managed_static_type_index_set(self, index); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lock needs to be applied to whole the loop or function
Hmm, still crashes with many workers...
_datetime
in sub-interpreters in the same time may crash the process #136421