panic in other rust's crates

[daniellga]

What can I do to avoid crashing my R session when other dependencies panic? For example I am using ndarray and they have this method that panics.

[yutannihilation]

There's no other way than being careful to satisfy the requirements of the dependency. Savvy might provide some "Debug mode" build, which wraps everything with catch_unwind(), to diagnose what's the problem, but, in the end, the framework doesn't provide any workaround.

In the example case, you can just check the dimension, can't you?

[daniellga]

Sure I thought about that, but that would be a double check to whatever checks are on dependencies... And also a huge work to check all these panics. Here we are talking about simple bounds or dimension checks but there can be cases where the dependency does some huge calculation and panics based on the results...

[yutannihilation]

So, your struggle is mainly about diagnosing the problem, right? Then, I think the debug mode will save you.

[daniellga]

It's actually about making the user not lose his R session when a panic occurs because of a mistake in one of the arguments.

[daniellga]

could you please briefly ELI5 me why R crashes when a panic occurs? Do you know if that's done the same way in PyO3?

[yutannihilation]

You know, panic means unrecoverable, catastrophic error. I'm strongly against providing the false sense of safety to your users by recovering the unrecoverables.

https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-with-panic.html

[daniellga]

Is there any way to make it opt in? Like in a feature? Rust is full of hidden panics like in bounds checking, division by zero and others...

[yutannihilation]

Do you know if that's done the same way in PyO3?

I don't use PyO3 by myself, but it seems PyO3 treats panic as a type of Exception.

https://stackoverflow.com/questions/76377350/pyo3-and-panics
https://pyo3.rs/main/doc/pyo3/panic/struct.panicexception

Please be aware that the difference comes from the fact the error-handling mechanism of R is actually different from Python in that R uses longjmp, which is considered UB when it crosses over FFI.

Is there any way to make it opt in? Like in a feature?

I'm sorry, no plan at the moment. Again, I'm strongly against providing the false sense of safety. While I understand your struggle, since this is the very main reason I needed to create savvy, I don't get easily convinced. Extendr keeps relying on (possible) undefined behavior and there seems no one who can fix it.

c.f. extendr/extendr#278 (comment)

If you think some operation of your third-party crate is fallible in nature, you should ask the author to return Result. Or, if you are sure it's safe to catch the panic, I don't stop you using catch_unwind().

[yutannihilation]

Let me be a bit more clear about catch_unwind(). As long as unwinding the stack of Rust's side, it's not undefined behavior. So, while it's not recommended to use this as a general try/catch mechanism, it is probably safe. However, as the framework calls R API, it's inappropriate to wrap all the Rust code by the framework's side.

[daniellga]

I think catch_unwind will be the way to go. ndarray is too big of a crate for me alone to improve their error handling. Or let it crash R and warn in documentation the reasons for the crash.
If I understood correctly, PyO3 seems to behave the same way as savvy, causing the interpreter to exit.

[yutannihilation]

If I understood correctly, PyO3 seems to behave the same way as savvy, causing the interpreter to exit.

Oh, good to know. Thanks.

[yutannihilation]

(Sorry for a drifted topic)

Reading the discussion on PyO3, it seems I understand the problem oppositely (?). It needs to be abort the session anyway in the case of panic, but, at the same time, the unwinding should be stopped before the FFI boundary.

• It is UB for a panic! to unwind into non-Rust stack frames. This is why we need catch_unwind to avoid this UB.

• It is however unfortunately also UB for non-Rust exceptions to unwind into Rust code, and for catch_unwind to catch these unwinds.
  (Attempts to catch_unwind (on Linux) result in SIGABRT PyO3/pyo3#2102 (comment))

• https://github.com/PyO3/pyo3/blob/main/src/impl_/panic.rs
• https://github.com/PyO3/pyo3/blob/2f0869a6d643e790e71dd5a676a30a7e539e5138/src/impl_/trampoline.rs#L170-L191

[yutannihilation]

If I understood correctly, PyO3 seems to behave the same way as savvy, causing the interpreter to exit.

I tried some polars Python code that are reported to cause panics in their repository, but it seems it doesn't exit the interpreter.

[daniellga]

could you link here the functions you tried?

[yutannihilation]

You can search.

https://github.com/search?q=repo%3Apola-rs%2Fpolars+panic&type=issues