Add back ModelInstance class

[lukemckinstry]

Description

Adds back a ModelInstance Class for runtime instancing.

Supply transformation matrices (Matrix4) to the Model for each instance by calling model.instance.add(transform). The transform should be in world space and the modelMatrix of the model should be the default identify Matrix4, which corresponds to the earth center.

In this way, the transformation matrix for each instance can be computed from a cartographic coordinate as shown below.

Example:

const viewer = new Cesium.Viewer("cesiumContainer");

const model = await Cesium.Model.fromGltfAsync({
    url: "../../SampleData/models/GroundVehicle/GroundVehicle.glb",
  });
const lng = -75.1652;
const lat = 39.9526;
const height = 30.0;

let position;
const headingPositionRoll = new Cesium.HeadingPitchRoll();
const fixedFrameTransform = Cesium.Transforms.localFrameToFixedFrameGenerator(
  "north",
  "west",
);

let instanceModelMatrix;
function genInstances(count) {
  for (let i = 0; i < count; i++) {
    position = Cesium.Cartesian3.fromDegrees(
      lng + i * Math.random() * 0.0001,
      lat + i * 0.0001,
      height + i,
    );
    instanceModelMatrix = new Cesium.Transforms.headingPitchRollToFixedFrame(
      position,
      headingPositionRoll,
      Cesium.Ellipsoid.WGS84,
      fixedFrameTransform,
    );
    model.instances.add(instanceModelMatrix);
  }
}
viewer.scene.camera.setView({
  destination: Cesium.Cartesian3.fromDegrees(lng, lat, height),
  orientation: new Cesium.HeadingPitchRoll(
    2.205737333179613,
    -0.7255022564055849,
    6.283181225638178,
  ),
});

Still TODO:

• broken tests (mostly classes impacted by the scene graph transforms refactor)
  • ModelSceneGraph
  • ModelDrawCommands
• Add tests
  • ModelInstance Class
  • RuntimeModelInstancingPipelineStage
  • ModelInstancesUpdateStage
• document edge case restrictions, have not yet restricted these in code
  • When instances are supplied to Model, modelMatrix must be the identify matrix.
  • Cannot supply instances to a model with glTF with ext_mesh_gpu_instance extension, throw developer error
  • 2D and columbus view
• Clean up sandcastle example
• document important technical details on issue - including scene graph and bounding sphere calculation refactors

Issue number and link

#10846

Testing plan

• Test using sandcastle example
• Review regression test sandcastles

Author checklist

• I have submitted a Contributor License Agreement
• I have added my name to CONTRIBUTORS.md
• I have updated CHANGES.md with a short summary of my change
• I have added or updated unit tests to ensure consistent code coverage
• I have updated the inline documentation, and included code examples where relevant
• I have performed a self-review of my code

[github-actions]

Thank you for the pull request, @lukemckinstry!

✅ We can confirm we have a CLA on file for you.

[ggetz]

This was copied over from the previous instancing stage, and will need to be updated to reflect the actual workflow.

[ggetz]

Thanks for getting this updated and itemized @lukemckinstry! Just a few notes on what we should make sure to test.

Since the workflow was touched by some refactoring, we should confirm the following are still working for non-instances models:

• Clamp to ground: Viz, zooming to, picking
• 2D Mode: Viz, zooming to, projectTo2D, picking
• Animations

We also need to check scaling with minimumPixelSize and maximumScale for both regular models and model instances.

[lukemckinstry]

Is this a terminology change, is there a difference between model coordinates and local glTF scene coordinates?

[lukemckinstry]

Since the workflow was touched by some refactoring, we should confirm the following are still working for non-instances models:

• Clamp to ground: Viz, zooming to, picking
• 2D Mode: Viz, zooming to, projectTo2D, picking
• Animations

All these are resolved.

We also need to check scaling with minimumPixelSize and maximumScale for both regular models and model instances.

Scaling is fixed for regular models, but not yet implemented for runtime instanced models. and implemented for runtime instanced models.

[lukemckinstry]

@ggetz

1. Confirming two restrictions in the runtime model instancing feature

• When instances are supplied to Model, modelMatrix must be the identify matrix.
• Cannot supply instances to a model with glTF with ext_mesh_gpu_instance extension, throw developer error

I did not add these to the code yet

2. Also confirming a detail related to making model.instances depend on a collection class so we have build in add and remove functions. Right now, model.instances is just a pass through to ModelSceneGraph._modelInstances, so is my understanding correct that the new collection should be in ModelSceneGraph._modelInstances and not model.instances?

[ggetz]

When instances are supplied to Model, modelMatrix must be the identify matrix.

Yes. Let's (a) ignore any other value that it might be set to, and (b) document this behavior in model instances property.

Cannot supply instances to a model with glTF with ext_mesh_gpu_instance extension, throw developer error

Yes, although a RuntimeError may make more sense then a developer error– The developer may not know all the models which will be loaded into their application until runtime. Again, this restriction should be documented.

[ggetz]

Also confirming a detail related to making model.instances depend on a collection class so we have build in add and remove functions. Right now, model.instances is just a pass through to ModelSceneGraph._modelInstances, so is my understanding correct that the new collection should be in ModelSceneGraph._modelInstances and not model.instances?

I think that would be fine. ModelSceneGraph is an implementation detail that should not be accessible by the user. The import things are that (a) the .add function and similar functionality of the collection is accessible from the public Model interface and (b) the needed information is passed through to ModelSceneGraph.

[ggetz]

@lukemckinstry Once the public API is where you'd like it, a reminder to clean up the Sandcastle example.

[ggetz]

@lukemckinstry Thanks for resolving the last few features!

I looked over the unit tests so far and left a few comments. Let me know if there are any concerns around those.

[ggetz]

Should this be scene.destroyForSpecs()?

[ggetz]

Let's prefer async/await syntax to promise chains. It flattens the test code and helps makes things more readable.

[ggetz]

Let's have this function only set up the mocks unrelated to the functionality we're testing. For instance, would probably expected to see something like the following line in the actual test block.

renderResources.sceneGraph.modelInstances = [sampleInstance1, sampleInstance2];

1. It helps make the test more readable for those who are working with them later.
2. It allows the mock and the test code to change independently.

[ggetz]

It looks like this test is jumping through a few hoops to validate the buffer contents.

Instead of creating a new buffer, perhaps consider validating the output of RuntimeModelInstancingPipelineStage._getTransformsTypedArray against expectedTransformsTypedArray, and then checking runtimeNode.instancingTransformsBuffer's usage and byteLength properties directly.

[ggetz]

Are we able to use an instance of ModelSceneGraph here rather than just a mock?

[ggetz]

Do either of these test scale transforms?

[ggetz]

Make sure to remove all console.log calls.

[ggetz]

So checking whether a specific function is called is always going to be more fragile than confirm the output is what we'd expect. In this case, instead of validating that getPrimitiveBoundingSphere is called, maybe a less fragile test would be to confirm the camera has zoomed to the expected position and orientation?

[ggetz]

Do we need this expect case? I don't think the bounding sphere values will be correct without it, so we can assume it was called correctly.

[lukemckinstry]

Another limitation that I should have included in the above comment but was not top of mind was that the current runtime model instancing implementation does not support changing the size of the instances buffer ie. number of instances arbitrarily over time. We support changing the instances themselves, but not the number available.

I believe we talked about this being OK for initial release earlier in development , but implementing the collection (ModelInstanceCollection with add and remove functions) this afternoon made this limitation a little more apparent. I do see how we can provide clear instructions so developers understand (initial) limitation.

[ggetz]

I believe we talked about this being OK for initial release earlier in development , but implementing the collection (ModelInstanceCollection with add and remove functions) this afternoon made this limitation a little more apparent. I do see how we can provide clear instructions so developers understand (initial) limitation.

If this is the case, I believe we should re-run the model pipeline with model.resetDrawCommands();. For an example, see updateClippingPlanes in Model.js. resetDrawCommands is called when the collection changes state.

[lukemckinstry]

If this is the case, I believe we should re-run the model pipeline with model.resetDrawCommands();. For an example, see updateClippingPlanes in Model.js. resetDrawCommands is called when the collection changes state.

That works. But it raises some question about API design

• Do we still need ModelInstanceUpdateStage?
• Do we call model.resetDrawCommands every time the user calls ModelInstanceCollection.add or remove. Or do we expose a function like ModelInstanceCollection.update(model) which runs model.resetDrawCommands on the supplied model?

[ggetz]

That works. But it raises some question about API design

@lukemckinstry and I discussed this offline—

We'll still need ModelInstanceUpdateStage to avoid rerunning the entire pipeline when one instance is updated. We'll re-run the pipeline when an instance is added or removed, as we'll need to reallocate the buffers containing the instance data.

Updates can be managed on the ModelInstance instances themselves, setting dirty flags as needed. The update stage should be able to handle those accordingly.

[javagl]

When instances are supplied to Model, modelMatrix must be the identify matrix.

Is it possible to summarize why this constraint exists?
(Beyond: "Well, all these matrices... it's complicated...")

One could argue that it's possible to "emulate" any modelMatrix, by baking this modelMatrix into the instances matrices. But that's also why that constraint could be hard to justify.

[ggetz]

Is it possible to summarize why this constraint exists?

@javagl I think mostly for API simplicity.

We're accounting for double precision for any instance locations on the globe. So a localized modelMatrix with instances relative to that shouldn't be needed for precision. Do you have another use cases in mind for providing a non-identity model matrix?

[javagl]

The question was unrelated to precision. (Precision is a tricky issue here, but ... unrelated for now).

I rather thought about cases where people want to create instances in a known, local coordinate space. Think about an airport runway where 200 lights are left and right of the runway, along a straight line, 10 meters apart. And then, users want to put these instanced models at a certain position on the globe, and use the modelMatrix for that.

In terms of convenience, the "best" API certainly depends on the use case. For example, in this screenshot, I used lat/lon/height as the input. In other cases, people might only have the local transforms. There may also be cases where the instancing information is given as TRS properties. And people will have to write quite a bit of boilerplate code to assemble these into Matrix4 objects, put them into the array, and maybe squeeze in the ENU-to-FF-matrix for the desired geolocation here.

(API design is often sort of a trade-off ... about shifting the responsibility for doing things (correctly!) between the implementor and the user. I think that the API should be easy to use correctly, and hard to use incorrectly. A seemingly(!) very specific aspect here is: People will create instances. They will set the modelMatrix. They will open an issue because the "model matrix does not work"...)