gh-128679: Redesign tracemalloc locking

[vstinner]

• Use TABLES_LOCK() to protect 'tracemalloc_config.tracing'.
• Hold TABLES_LOCK() longer while accessing tables.
• tracemalloc_realloc_gil() and tracemalloc_raw_realloc() no longer remove the trace on reentrant call.
• _PyTraceMalloc_Stop() unregisters _PyTraceMalloc_TraceRef().
• _PyTraceMalloc_GetTraces() sets the reentrant flag.
• tracemalloc_clear_traces_unlocked() sets the reentrant flag.

• Issue: Race condition in tracemalloc causes segfaults #128679

[vstinner]

_PyTraceMalloc_Stop() unregisters _PyTraceMalloc_TraceRef().

This change cannot be backported to Python 3.12 which doesn't have the PyRefTracer_SetTracer() function.

Anyway, I don't plan to backport this major redesign to Python 3.13 neither.

[ZeroIntensity]

This looks like a much better solution. A few notes:

• I think it's worth trying to migrate away from PyThread, because it's a lot more difficult to work with than the modern APIs. We can go with PyMutex and _Py_thread_local instead.
• We need to be careful with holding locks while acquiring the GIL, because that can lead to lock-ordering deadlocks.
• I'll need to look into what kind of subinterpreter problems might come up because of PyGILState_Ensure, as it doesn't work very well. What would happen if e.g. a thread calls PyTraceMalloc_Track, which sets the GILstate for the main interpreter, then enters a different interpreter?

[vstinner]

I think it's worth trying to migrate away from PyThread, because it's a lot more difficult to work with than the modern APIs. We can go with PyMutex and _Py_thread_local instead.

Why not. It can be done afterwards.

We need to be careful with holding locks while acquiring the GIL, because that can lead to lock-ordering deadlocks.

Right. The order is: first get the GIL, and then call TABLES_LOCK().

I'll need to look into what kind of subinterpreter problems might come up because of PyGILState_Ensure, as #123728. What would happen if e.g. a thread calls PyTraceMalloc_Track, which sets the GILstate for the main interpreter, then enters a different interpreter?

This PR doesn't change that. But right, there should be corner cases when sub-interpreters are involved.