Skip to content

gh-136421: Fix crash when _datetime is been initialized in multiple sub-interpreters #136422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

aisk
Copy link
Contributor

@aisk aisk commented Jul 8, 2025

@aisk aisk requested review from pganssle and abalkin as code owners July 8, 2025 14:01
@aisk aisk changed the title gh-136421: Fix crash when _datetime is been initialized in multiple sub-interpre… gh-136421: Fix crash when _datetime is been initialized in multiple sub-interpreters Jul 8, 2025
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a more fundamental fix that prevents static types from being initialized concurrently. This is sort of a known issue, see #129817.

@@ -7344,9 +7344,13 @@ init_static_types(PyInterpreterState *interp, int reloading)

/* Bases classes must be initialized before subclasses,
* so capi_types must have the types in the appropriate order. */
static PyMutex mutex = {0};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the assumption is that this can run concurrently, then we need to also protect the assignments to tp_base above.

@bedevere-app
Copy link

bedevere-app bot commented Jul 8, 2025

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Comment on lines +7350 to +7352
PyMutex_Lock(&mutex);
int ret = _PyStaticType_InitForExtension(interp, type);
PyMutex_Unlock(&mutex);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lock needs to be applied to whole the loop or function rather than to a type to avoid a crash here (for me), because all managed static extension types are currently supposed to be indexed by using the same interpreter:

else {
PyMutex_Lock(&interp->types.mutex);
index = interp->types.for_extensions.next_index;
interp->types.for_extensions.next_index++;
PyMutex_Unlock(&interp->types.mutex);
assert(index < _Py_MAX_MANAGED_STATIC_EXT_TYPES);
}
managed_static_type_index_set(self, index);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lock needs to be applied to whole the loop or function

Hmm, still crashes with many workers...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants