"Instant" command processing

[alice-i-cecile]

What problem does this solve or what need does it fill?

Having to wait until a hard sync point (currently the end of the Stage) for basic commands to process slows down prototyping badly. This is particularly frustrating with

• spawning entities
• despawning entities
• adding components
• removing components

Without this feature, you have to be very thoughtful about how and when you're using commands, even when prototyping, and where you place systems that rely on other things being mutated via commands. Failing to account for command delays properly can result in frame lags or buggy behavior, and moving systems between stages to accommodate the required hard syncs takes a lot of work, which must be refactored whenever your prototype changes significantly.

Possible Solutions

This is a very technically complex feature request, and as such there are several potential solutions that deserve more exploration. The gist of the challenge is that commands can, in general, touch arbitrary data, making them very hard to schedule. To get around that, regular hard sync points exist, which have exclusive access to the entire world and can perform arbitrary operations to process any commands that may have built up.

1. Bit-flipping: see Fast add/remove components via bit flipping. #356. Iterators have to be aware that they can skip components in tables due to change detection, so we can use this property to toggle a bit to "soft-remove" an entity from an archetype without mutable access to the entire World (and possibly do something similar to add it to a new one). We can use hard syncs to clean up the archetypes, and recover data locality / future performance. From @BoxyUwU.
2. Commands subworlds: Copy the data into a subworld when using commands, queries must access the main world and all subworlds. From Kae on Discord.
3. Dynamically scheduled commands: fragment commands based on the data that they need to access. Dynamically schedule them at a point between they are created and the first time that data would be used. Remove most hard syncs in the process. See this comment on System organization overhaul and the road to a stageless schedule #1375.

[Davier]

I like the idea of having a "staging" world that commands insert into.
As discussed in #1446, it would be useful to spawn loaded assets (especially scenes) and modify them on the fly.

[Ratysz]

"It makes prototyping easier by letting you think about X less" does not sound like compelling motivation to me for this case. Maybe I'm picking up more from phrasing than there is.

I fairly sure we already had this conversation on why multiple worlds is not a good general solution for this. Some form of slowboat staging world could be a thing, but it doesn't have to be powered by commands; copying out parts of the world for things like rendering is a different matter entirely.

Fragmenting commands could work, but it is an entire new layer of constraints on the systems graph - automatically supporting that for all systems is bound to harm parallelization; it should be opt-in, and it should be immediately obvious which commands can be "instant", perhaps to the point of having separate dedicated queues. This also doesn't have to be dynamic: the system must declare what commands it may issue, exactly the same way it must declare what data it accesses or if it cares about execution order, so "scheduling commands" of a system can be done as part of scheduling the system itself.

It bears mentioning that no solution will be able to make all commands instant, only a subset.

[cart]

Something to consider: we can allow "immediate" insertions and removals if we can guarantee that no other system is reading or writing a given storage (Table, SparseSet, Archetypes, Entities).

We could add a new api that enables users to claim these resources uniquely.

Spawning/despawning in a specific Archetype requires unique write access to the following:

• Archetype value
  • Cannot be run concurrently with &Archetypes queries (requires new logic) or other operations that need this Archetype value (which can be guaranteed if all ArchetypeComponents are grabbed uniquely and that is globally treated as a unique Archetype borrow)
• Table of archetype
  • Claiming Write access to each ArchetypeComponent in the archetype prevents concurrent reads/writes to components
• Sparse Sets in archetype
  • Claiming Write access to each sparse Component in the archetype prevents concurrent reads/writes (this could be translated to all current ArchetypeComponents for a given Component to avoid complicating the scheduler)
• Entities
  • This is the "hard" part. Grabbing this uniquely prevents all other spawn/despawn actions, in addition to metadata lookups for things like Query::get_component::<T>(entity) and Query::get(entity). This wouldn't block Query iteration as it doesn't access the metadata collection. This would almost certainly require some kind of synchronization.

Meeting the constraints above should enable things like:

// Claims an _exact_ archetype (using the logic above) and nothing else
// (and initializes the archetype if it doesn't already exist)
fn sys(mut writer: ArchetypeWriter<(A, B, C)>) { 
  writer.spawn((A(0), B(0), C(0)));
  writer.despawn(some_entity); // fails if entity is not in archetype
}

// Claims _all_ archetypes that currently match the query  (using the logic above)
// no new archetypes are allowed (to prevent the need to lock Archetypes uniquely and
// because we can't anticipate what component types are needed ahead of time)
fn sys(mut writer: QueryWriter<(A, B, C)>) {
  writer.spawn((A(0), B(0), C(0)));
  writer.spawn((A(1), B(1), C(1), D(1)); // note that D's storage was implicitly claimed by QueryWriter
  writer.despawn(some_entity);
}

// Claims the before/after archetype (using the logic above) and nothing else
// (and initializes the archetypes if they doesn't already exist)
fn sys(mut remover: Remover<(A, B, C),  B>) { 
  remover.remove(entity);
}

We should also consider just making it possible to add exclusive systems (aka hard sync points) before/after any system in a stage. Solving that problem allows for simpler, less restrictive world access.

[alice-i-cecile]

We should also consider just making it possible to add exclusive systems (aka hard sync points) before/after any system in a stage. Solving that problem allows for simpler, less restrictive world access.

This should be vastly improved by a stageless design. Following up on it though, I would much rather see users able to add parallel systems as exclusive systems that induce a hard sync point, whose commands are immediately processed. This allows them to stick to the standard type signatures, and then convert them back into ordinary parallel systems quickly once they're done prototyping.

e.g. add_system(my_system) becomes add_exclusive_system(my_system) and vice versa, quickly letting you replace them without needing to completely rewrite things in terms of World like you do now. Using the .system() and .exclusive_system() methods would also work for now, but ideally that boilerplate will be able to be removed Soon:tm:.

[alice-i-cecile]

Something to consider: we can allow "immediate" insertions and removals if we can guarantee that no other system is reading or writing a given storage (Table, SparseSet, Archetypes, Entities).

This idea feels like... partially exclusive systems to me? I like them a lot, but for the same reasons as my last comment, I feel like they should be defined upon system insertion, rather than forcing a modification of the parameter types in your system.

[alice-i-cecile]

Overall, I think that the "run this parallel system as an exclusive system", inserted into arbitrary points in the schedule is the most promising approach to try out first because:

• It dovetails very nicely with the stageless work we want to do already
• As far as I can see, it introduces no new serious technical challenges
• It enables people to write exclusive systems with more comfortable syntax, rather than learning to need how to grab the data they need from the world directly
• It's very easy to convert systems to-and-from this form, allowing for simple prototyping
• It won't impose any overhead on optimized games that only use carefully scheduled parallel systems

[Ratysz]

Any system can already be used as an exclusive system, you just need to insert it with .exclusive_system(). Naturally, that comes with all the restrictions, main one being that it's inserted into the spots where exclusive systems go. It's not possible to insert such a coerced exclusive system into arbitrary points in the schedule for the same reasons as with regular exclusive systems.

Stageless will indeed let us make nicer APIs for this: specifying hard sync points without the additional boilerplate of stages is what stageless is all about, in the end. I'm working on it, but it'll need more thinking time; multiple issues seemingly need to be solved at the same time and I would like to avoid doing that with a ground-up rewrite.

None of that is relevant to the issue at hand, though. Commands are intended to enable parallel systems to perform generally non-parallelizable operations while remaining parallel; making some of the systems exclusive defeats the purpose. Calling systems issuing disjoint commands "partially exclusive" is confusing.

[alice-i-cecile]

Here's a really clear use case for this functionality, with no great workarounds.