Skip to content

double free or corruption when enable python.IsolateNativeModules #563

@suiwenfeng

Description

@suiwenfeng

How to reproduce?

> environment

container image: ghcr.io/graalvm/graalpy-community:25
jdk: graalvm-jdk-25_linux-x64_bin.tar.gz

> python venv deps installed

pip install torch transformers patchelf

> multi context in maven repo(not packaged as native-image)

// Create a new GraalVM Python context and managed by org.apache.commons.pool2.impl.GenericObjectPool
Context context = GraalPyResources
        .contextBuilder(Path.of("python-resources"))
        .option("python.IsolateNativeModules", "true")
        .allowExperimentalOptions(true)
        .allowIO(IOAccess.ALL)
        .allowCreateThread(true)
        .allowCreateProcess(true)
        .allowHostAccess(HostAccess.ALL)
        .allowAllAccess(true)
        .build();
// use contexts in parallel, scriptPool has 3 context above
String scriptText = ""
        + "import torch\n"
        + "import transformers\n"
        + "\n"
        + "def main():\n"
        + "    print(f'Torch version: {torch.__version__}, Transformers version: {transformers.__version__}')\n";
PythonScriptPool scriptPool = new PythonScriptPool("testScript", scriptText, 3);
java.util.stream.Stream.iterate(0, n -> n + 1).limit(10).parallel().forEach(n -> {
    try {
        scriptPool.executeMethod("main", new Object[] {});
    } catch (Throwable e) {
        e.printStackTrace();
        System.exit(1);
    }
});

> error log

18:18:13  #14 0.761 WARNING: A restricted method in java.lang.System has been called
18:18:13  #14 0.761 WARNING: java.lang.System::load has been called by com.oracle.truffle.polyglot.JDKSupport in an unnamed module (file:/app/repo/target/bb-1.0-SNAPSHOT.jar)
18:18:13  #14 0.761 WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
18:18:13  #14 0.761 WARNING: Restricted methods will be blocked in a future release unless native access is enabled
18:18:13  #14 0.761 
18:18:13  #14 1.081 WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
18:18:13  #14 1.081 WARNING: sun.misc.Unsafe::objectFieldOffset has been called by com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntime (file:/app/repo/target/bb-1.0-SNAPSHOT.jar)
18:18:13  #14 1.081 WARNING: Please consider reporting this to the maintainers of class com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntime
18:18:13  #14 1.081 WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.806 double free or corruption (out)
18:18:21  #14 8.806 double free or corruption (!prev)
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # A fatal error has been detected by the Java Runtime Environment:
18:18:21  #14 8.806 #
18:18:21  #14 8.806 #  SIGSEGV (0xb) at pc=0x00007f24b9d9a997, pid=1, tid=52
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # JRE version: Java(TM) SE Runtime Environment Oracle GraalVM 25.0.1+8.1 (25.0.1+8) (build 25.0.1+8-LTS-jvmci-b01)
18:18:21  #14 8.806 # Java VM: Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 25.0.1+8.1 (25.0.1+8-LTS-jvmci-b01, mixed mode, sharing, tiered, jvmci, jvmci compiler, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
18:18:21  #14 8.806 # Problematic frame:
18:18:21  #14 8.806 # C  [libc.so.6+0x28997]  abort+0x174
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # Core dump will be written. Default location: /app/repo/core.1
18:18:21  #14 8.806 #
18:18:21  #14 8.807 # An error report file with more information is saved as:
18:18:21  #14 8.807 # /app/repo/hs_err_pid1.log
18:18:21  #14 8.835 [8.718s][warning][os] Loading hsdis library failed
18:18:53  #14 39.29 #
18:18:53  #14 39.29 # If you would like to submit a bug report, please visit:
18:18:53  #14 39.29 #   https://bugreport.java.com/bugreport/crash.jsp
18:18:53  #14 39.29 # The crash happened outside the Java Virtual Machine in native code.
18:18:53  #14 39.29 # See problematic frame for where to report the bug.
18:18:53  #14 39.29 #

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions