feat(ses): export Permits

[boneskull]

Description

This exports the entirety of src/permits.js as the Permits object via the tools export.

We are using this information in lavamoat-tofu for SES-compat detection, and we would like to keep it up-to-date with SES (LavaMoat/LavaMoat#814).

There are other ways we could accomplish this, but exporting it is the easiest way for lavamoat-tofu to consume it.

Security Considerations

Unknown

Scaling Considerations

No

Documentation Considerations

The tools export is not currently documented. Whether this is considered a private API is unknown to me. Assuming it's not considered a private API, this change adds more stuff to the public API which will need to be taken into consideration during versioning.

Testing Considerations

I'm not sure it's valuable to test that an export simply exists, but I can add such a test if desired.

Upgrade Considerations

n/a

[boneskull]

• feat(ses): export Permits #1981 👈
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @boneskull and the rest of your teammates on Graphite

[boneskull]

@kumavis This is the "whitelist" that gets pulled into tofu.

[kriskowal]

I would prefer to avoid making the permits structure part of the SES public API so we can remain at liberty to change its structure and type as need evolves. It exists in its current form because of a host of simplifying assumptions that are not permanently reliable. We recently added support for symbols. We could conceivably need to allow [[name]] or @name to be permitted properties and they cannot today.

(If we did export it as-is, I would want to export it from a separate module.)

This is a case where I would prefer that you copy and paste.

If we ejected the permits into another package, so that ses could ramp the major version of its internal dependency without having to ramp its own major version, that might be acceptable too.

I would like @erights to weigh in on these considerations.

[kumavis]

I find it useful, moving it into a lower package seems appropriate

[erights]

First, a nit.

We recently added support for symbols.

The permits.js file already supported symbols via a textual encoding. What we recently added was symbol support to enablements.js. That one uses the symbols themselves, as there was no need to encode them.

@kriskowal 's main point

It exists in its current form because of a host of simplifying assumptions that are not permanently reliable.

is true. That is not necessarily incompat with a tools-only export, depending on what we say that means.

Some other considerations

• I am uncomfortable about exporting these unless we harden them first. They are not meant to be mutated.
• Vetted shims run after permits-intrinsics.js. We want to define the obligations of a "vetted shim" such that many well-behaved shims not aware of the ses-shim can be approved by this vetting process. The permits-intrinsics.js processing happens before the vetted shims run. The vetted shims are supposed to be able to mutate the primordials in ways that we vet as safe, but which can deviate from permits.js, rendering that tools export stale by the end of that phase. We have not decided that the vetting process must include meta data describing these changes to the primordials.
• If you export this, perhaps you should also export enablements.js. These are processed after vetted shims run, in which case we have the opposite problem: we might need meta data for each vetted shim saying how to update enablements.js before that phase runs.
• The info provided by these data structures has much overlap with our vague intentions for the "get intrinsics" proposal, which is what we hope to eventually standardize. This proposal has similar questions about being updated in light of mutations before hardening, but the answers to these questions are likely to be different in detail.

Nevertheless, I remain open to exporting via tools given an adequate shared understanding of what such an export means re future coordination.

Seems like a good discussion for an upcoming Endo meeting!

[kriskowal]

Seems like a good discussion for an upcoming Endo meeting!

Added to the agenda, though we have demos with Spritely scheduled.

I feel strongly that if we export permits, it should be from another package so that we don’t have to ramp the SES major version if its schema changes. That is tricky, though. Harden would not be available in that package. Pervasively shallow- freezing the permits would be a lot of ceremony.

[boneskull]

Did this get discussed already or no? If not, I'll be on the next call

[kriskowal]

Did this get discussed already or no? If not, I'll be on the next call

This did not get discussed. Consider the topic commuted!

[kriskowal]

Summary:

1. My position is that the permits should have a separate version number so breaking changes to the permits do not force a major version bump in SES. Major version bumps in SES increase the number of versions we have to back-port patches to.
2. Mark’s position is that the permits should be hardened.
3. There’s a reality that harden can’t be called until after lockdown, which ties 1 and 2 into a chicken and egg cycle.

However, the permits are acyclic. If we ejected the permits to @endo/permits and did a recursive freeze over transitive properties at the export site, they would be emergently hardened after lockdown, lockdown could still refer to them internally, and we wouldn’t have to do any special treatment in anticipation of cycles.

I’m curious whether @erights would find that safe enough.

[erights]

... ejected ...

"ejected"?

[boneskull]

fwiw: I have no opinion on how they get exported, but it'd be nice if we could find some mutually-agreed-upon way to do so. 😄

[kriskowal]

... ejected ...

"ejected"?

Move code currently in ses into a package ses depends upon.

[erights]

... ejected ...

"ejected"?

Move code currently in ses into a package ses depends upon.

Rereading with that in mind...

I’m curious whether @erights would find that safe enough.

yes.

[kriskowal]

Plan of record:

1. Create a new package, @endo/permits using scripts/create-package.sh permits
2. Move permits.js into @endo/permits under src.
3. Add a transitive property tree freeze and freeze permits.
4. export permits, import them in ses

[erights]

What about enablements? What other intrinsic enumerations should we consider?

[boneskull]

working some necromancy here

[boneskull]

@erights

What about enablements? What other intrinsic enumerations should we consider?

IMO, we can add them later if somebody needs them (I do not).

[boneskull]

@kriskowal @erights

1. I don't understand why (it's not explained) the new package must be hardened.
2. I don't understand how I'm supposed to harden the new package. Some sort of cycle seems unavoidable (maybe at the package-dep level). Is there prior art?
3. In theory, I like the idea of creating a new package for this. In practice, if such a module incurs more complexity due to workarounds for the above, maybe it's better to leave it in ses?
4. What if we create a subpath export (i.e. import permits from 'ses/permits') which exposes a function (e.g., getPermits()) that returns a copy of the data in permits module? The underlying data structure would continue to be used directly by ses, but consumers would just get a deep clone. This is acceptable for the purposes of lavamoat-tofu.

[kriskowal]

@kriskowal @erights

1. I don't understand why (it's not explained) the new package must be hardened.
2. I don't understand how I'm supposed to harden the new package. Some sort of cycle seems unavoidable (maybe at the package-dep level). Is there prior art?
3. In theory, I like the idea of creating a new package for this. In practice, if such a module incurs more complexity due to workarounds for the above, maybe it's better to leave it in ses?
4. What if we create a subpath export (i.e. import permits from 'ses/permits') which exposes a function (e.g., getPermits()) that returns a copy of the data in permits module? The underlying data structure would continue to be used directly by ses, but consumers would just get a deep clone. This is acceptable for the purposes of lavamoat-tofu.

The reason for a separate package is to decouple what constitutes a breaking change for SES from what constitutes a breaking change for the permits. Exposing the permits in-place from SES would greatly expand the occasions that a change to SES would constitute a breaking change, since the shape of the permits itself would become part of the public API. By putting the permits in a separate package, the permits can make breaking-changes at a more frequent and separate cadence from SES proper.

If the permits are exported by another package, SES needs to be able to rely on the immutability of those permits. A separate package would not have a special relationship with SES, so as you point out, the exports must either be made immutable or the public API for accessing them must make defensive copies for each consumer. For reasons of not burning RAM and CPU needlessly, I would favor a transitive freeze of the permits.

Harden would not be available in the hypothetical “@endo/ses-permits” package. We wouldn’t entertain a dependency cycle. I believe it would be sufficient to embed a miniature of harden that transitively freezes properties and doesn’t walk up prototypes. But, I could be wrong about this because SES would have to be even more careful of how it enumerates the permits, so as not to be vulnerable to Object.prototype pollution. It is probably already sufficiently careful, but in light of changing invariants, would need an audit.

It might be that recreating @endo/harden or @endo/make-hardener for this property-walk-only variant of harden would be incidentally useful for both this change and further other efforts to decouple harden and lockdown so that we can use more of Endo in realms that are not locked-down, #1686

[erights]

In the case where ses uses it, ses will freeze all the primordials anyway. ses can therefore just harden the import as soon as harden exists, to ensure nothing got lost in coordinating what's frozen.