is there a way to handle out-of-memory errors more cleanly?

[ctb]

it may be possible to catch and ignore or flag out of memory errors within Rust; this would be a nice UI improvement since right now it's not clear that's what's going on when it happens.

[bluegenes]

Do you mean track the total memory we have an try to avoid OOM?

Looks like we could use sysinfo to check available memory, but I'm not sure how to do it mid-download and stop a download if no memory is available.

Maybe we could prevent downloads from starting if available memory exceeds the maximum size of a download thus far? Or we could maybe delay processing a file into signatures if there isn't sufficient memory available?

[bluegenes]

also ref #47 - is there a way to chunk downloads and still check the md5sum?

[ctb]

Re OOM errors, I was wondering if we could actually catch the error itself - it is not necessarily straightforward generally, but I was curious. Some googling found some links - this makes for interesting reading:

https://doc.rust-lang.org/stable/nomicon/vec/vec-alloc.html

as does this:

https://users.rust-lang.org/t/handling-memory-exhaustion-state-of-the-art/87375

and maybe this :)

https://stackoverflow.com/questions/44923306/why-doesnt-boxnew-return-an-option-or-result

and this

https://www.reddit.com/r/rust/comments/qinvlr/does_rust_really_abort_on_allocation_failure/

but I don't see a super clear answer except maybe in the first link.

The best plan would be to go look at the library you're using for downloads & unpacking and see if there are any handles to track failures to allocate.

Alternatively, we could think about downloading the files to a temporary disk space and processing them from there.