env: make the new mount options opt-in

[davvid]

Rocky 9.4 does not support the lowerdir+= option.
Make the new mount options an opt-in feature.

Related-to: #968
Related-to: #1164

[jrray]

Another consideration is if to make the way we determine the length limit configurable:

spk/crates/spfs/src/env.rs

Line 1151 in 799e700

match nix::unistd::sysconf(nix::unistd::SysconfVar::PAGE_SIZE) {

256 should still be long enough in most cases, but will require more layer flattening. Can we determine the correct limit at runtime?

[jrray]

Consider changing this name to mount_options_allow_append since we all know "new" things don't stay that way.

[rydrman]

Thoughts from the meeting today:

1. We already have infrastructure setup to try and detect the supported overlayfs options, can we piggyback on that in any way to move this from a compile-time flag to a low-cost runtime check?
2. re-iterating the same idea for how we determine the length of the supported args to ensure that our existing logic still works on newer kernels with different limits

(these could both be separate tickets/ follow ups)

[davvid]

Sounds good. I renamed the feature flag for now.

I can follow up in a separate PR with (1) and (2).

Since lowerdir+ is dependent on the kernel version, I can look at extending our parsing of modinfo overlay to read the version from there. That'll avoid an extra call to uname -r or similar since we're already calling that.

Once I do that, then we can reorient this from a feature flag to runtime behavior.

My other thought is that this approach can probably be then extended to #1181 to eliminate the need for (2) at all, since doing the direct syscalls should avoid any length limits whatsoever.

At that point, I kinda wonder whether the version check will even be needed. In theory, I may be able to get this working without any limits since we'd be using direct syscalls, and it's possible that it might work using the current lowerdir options supported by older kernels, so there might not be much benefit in using lowerdir+, even though the kernel supports it.

I'll have to test that. If using syscalls eliminates the errors w/out lowerdir+ then I think we might still want an option to opt-in to using syscalls, just because it's a fairly low-level thing, but I'm curious about your opinion on that since it does explode the combinatorics. I think the ultimate goal would be to squash the things down into a single way, so hopefully any feature flags or config options would be relatively short-lived.

So.. initially:

• Feature flag for lowerdir+

Then, if syscalls works with just lowerdir, then we'd have:

• An option to enable syscalls, and those syscalls only use lowerdir
• A feature flag for the mount code paths for enabling lowerdir+

So that's three different configurations. Pseudocode:

if syscalls_option_is_enabled:
    do_syscalls
elif use_lowerdir_plus_feature_flag:
    do_mount_with_lowerdirplus
else:
    do_mount_with_lowerdir (current behavior)

I'm 🤞 that the syscalls can work out without having to have an extra branch in there for whether or not to use lowerdir+.

Let me know if that sounds good. For now, this PR has been updated with the review notes so it might be okay to merge as-is.

I'll plan to re-appropriate #1181 to implement the plan laid out above if we y'all agree. I'd do that after this is merged so I can rebase against this as its base.

[davvid]

Superseded by #1192 where this has gone from a feature flag to a runtime probe.