rare ijl_bounds_error_int with Julia 1.11 & python 3.13

[dpinol]

Affects: JuliaCall

Describe the bug
Very rarely, my python application crashes while juliacall is invoking a julia function.

This is the error I get:

error in running finalizer: BoundsError(a=MemoryRef(offset=0, ptr_or_offset=0x5a44f85dd300, mem=Memory{Int64}(1819, 0x5a44f85dd300)[....]

ijl_bounds_error_int at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/rtutils.c:207
memoryref at ./boot.jl:524 [inlined]
unsafe_copyto! at ./genericmemory.jl:115 [inlined]
#133 at ./array.jl:1133
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
_pyjl_dealloc at /home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/C.jl:35
jfptr__pyjl_dealloc_9930 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
jlcapi__pyjl_dealloc_10065 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
subtype_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2388
Py_DecRef at /home/avatar/.julia/packages/PythonCall/hXycg/src/C/pointers.jl:300 [inlined]
enqueue at /home/avatar/.julia/packages/PythonCall/hXycg/src/GC/GC.jl:99
py_finalizer at /home/avatar/.julia/packages/PythonCall/hXycg/src/Core/Py.jl:27
unknown function (ip: 0x7c5556d13607)
run_finalizer at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:303
jl_gc_run_finalizers_in_list at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:393
run_finalizers at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:439
run_finalizers at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:420 [inlined]
ijl_gc_collect at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:3915
maybe_collect at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:926 [inlined]
jl_gc_pool_alloc_inner at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:1319
jl_gc_alloc_ at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/julia_internal.h:523 [inlined]
_new_genericmemory_ at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/genericmemory.c:56 [inlined]
jl_alloc_genericmemory at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/genericmemory.c:99
GenericMemory at ./boot.jl:516 [inlined]
array_new_memory at ./array.jl:1058 [inlined]
#133 at ./array.jl:1129
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
_pyjl_dealloc at /home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/C.jl:35
jfptr__pyjl_dealloc_9930 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
jlcapi__pyjl_dealloc_10065 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
subtype_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2388
Py_DECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:949 [inlined]
Py_XDECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:1042 [inlined]
tupledealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/tupleobject.c:208
Py_DECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:949 [inlined]
Py_XDECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:1042 [inlined]
list_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/listobject.c:566
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:2885
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:361 [inlined]
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:333 [inlined]
PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:373
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1355
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9570
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:146
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:361 [inlined]
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:333 [inlined]
PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:373
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1355
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyEval_EvalFrame at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/internal/pycore_ceval.h:119 [inlined]
_PyEval_Vector at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/ceval.c:1816 [inlined]
PyEval_EvalCode at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/ceval.c:604
run_eval_code_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1381 [inlined]
run_eval_code_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1348
run_mod at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1466
pyrun_file at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1295 [inlined]
_PyRun_SimpleFileObject at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:517
_PyRun_AnyFileObject at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:77
pymain_run_file_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:410 [inlined]
pymain_run_file at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:429 [inlined]
pymain_run_python at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:696 [inlined]
Py_RunMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:775
pymain_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:805 [inlined]
Py_BytesMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:829
unknown function (ip: 0x7c555cc2a1c9)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)

...My python code...
  File "/home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/any.jl", line 262, in __call__
    return self._jl_callmethod($(pyjl_methodnum(pyjlany_call)), args, kwargs)
juliacall.JuliaError: <exception str() failed>

[1695082] signal 11 (1): Segmentation fault
in expression starting at none:0
delta_new at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Modules/_datetimemodule.c:2745
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1985
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1502
_PyObject_VectorcallTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/internal/pycore_call.h:168 [inlined]
PyObject_CallOneArg at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:395
call_unbound_noarg at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2577 [inlined]
slot_tp_finalize at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9863
finalize_garbage at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/gc.c:980 [inlined]
gc_collect_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/gc.c:1408
finalize_modules at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pylifecycle.c:1757
_Py_Finalize at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pylifecycle.c:2134
Py_RunMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:777
pymain_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:805 [inlined]
Py_BytesMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:829
unknown function (ip: 0x7c555cc2a1c9)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)
Allocations: 30653030 (Pool: 30651924; Big: 1106); GC: 26

Your system
Please provide detailed information about your system:

• The operating system: Ubuntu 25.04
• Julia 1.11.6, python 3.13.5, PythonCall 0.9.27 , JuliaCall 0.9.27
• If an issue with JuliaCall, the output of pip list or conda list and juliapkg.status().

juliacall               0.9.27
juliapkg                0.1.17

Julia 1.11.6 (=1.11.6) @ /home/dani/.julia/juliaup/julia-1.11.6+0.x64.linux.gnu/bin/julia
Packages:
  PythonCall: {'uuid': '6099a3de-0909-46bc-b1f4-468b9a2dfc0d'}
  SCEngine: {'uuid': 'bfeac117-3e8e-4b47-838f-ff82b809d5a4', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-engine'}
  SCCli: {'uuid': '274a597d-43d1-4cc6-8622-452e712fc982', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-cli'}
  SCRunner: {'uuid': 'ea12bd49-5513-406d-806f-9aeee9d9fc30', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-runner'}
  SCTestHelpers: {'uuid': 'd3adfe28-cc0d-41fe-9d43-3a9a094f5274', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-engine', 'subdir': 'sc-testhelpers'}

[cjdoris]

Hmm. When we wrap Julia values into Python objects, the Julia object is stored in a global vector and referred to by index from the Python object. It seems that this vector has got corrupted somehow.

Do you have any multithreading? We don't use any thread-safe data structures for this, but then I thought the GIL should protect us.

[dpinol]

I have multithreading, but only on the julia side. So it shouldn't affect, right?
I'll try to create MWE which eventually crashes, but not sure it will do it.

Could this be related to #637? Does it make sense that I try this branch?

[cjdoris]

Honestly I don't know - just the nature of the error suggests the state is corrupted, which suggests some race condition. You could try that branch and see.

The state in question is these two vectors:

PythonCall.jl/src/JlWrap/C.jl

Lines 17 to 21 in 3633da0

	# we store the actual julia values here
	# the `value` field of `PyJuliaValueObject` indexes into here
	const PYJLVALUES = []
	# unused indices in PYJLVALUES
	const PYJLFREEVALUES = Int[]

Similar to that #637, you could add a lock to protect any accesses to these two vectors. If the problem goes away then there is a race condition.

Then if so, you can then migrate to using trylock and print a debug message whenever the lock cannot be immediately acquired. This will find where the race condition would be hit if we didn't have locks.