Skip to content

Commit a0c99a5

Browse files
committed
Add morph targets to bevy_pbr
Objective --------- - Add morph targets to `bevy_pbr` (closes bevyengine#5756) & load them from glTF - Supersedes bevyengine#3722 - Fixes bevyengine#6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. Solution -------- This implementation of morph targets uses a 3d storage texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 Acknowledgements --------------- * Thanks to @storytold for sponsoring the feature * Thanks to @superdump and @james7132 for guidance and help figuring out stuff Future work ----------- - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see bevyengine#8357 ---- Changelog --------- - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `GltfMeshExtras` component to query gltf extras specific to meshes of a given node - This allows reading morph weight names, see the new `scene_viewer` `morph_viewer_plugin.rs` module for details. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `multiple_morph_target_meshes.gltf` asset for morph targets testing. Load it with the scene viewer. Migration Guide --------------- - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation. - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `set_mesh_binding_defs` . - The `MeshBindGroup` resource doesn't exist anymore. Mesh bind groups are computed in `queue_mesh_bind_group` system and are available as the field `GpuMesh::bind_group` . [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
1 parent c488b70 commit a0c99a5

38 files changed

+3663
-252
lines changed

CREDITS.md

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,21 @@
2121
* Ground tile from [Kenney's Tower Defense Kit](https://www.kenney.nl/assets/tower-defense-kit) (CC0 1.0 Universal)
2222
* Game icons from [Kenney's Game Icons](https://www.kenney.nl/assets/game-icons) (CC0 1.0 Universal)
2323
* Space ships from [Kenny's Simple Space Kit](https://www.kenney.nl/assets/simple-space) (CC0 1.0 Universal)
24-
* glTF animated fox from [glTF Sample Models](https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/Fox)
25-
* Low poly fox [by PixelMannen](https://opengameart.org/content/fox-and-shiba) (CC0 1.0 Universal)
26-
* Rigging and animation [by @tomkranis on Sketchfab](https://sketchfab.com/models/371dea88d7e04a76af5763f2a36866bc) ([CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/))
24+
* glTF animated fox from [glTF Sample Models][fox]
25+
* Low poly fox [by PixelMannen] (CC0 1.0 Universal)
26+
* Rigging and animation [by @tomkranis on Sketchfab] ([CC-BY 4.0])
27+
* `multiple_morph_target_meshes.gltf`, glTF morph targets from glTF Sample Models, a combination of 4 different models:
28+
* [Animated Morph Cube] (CC0 1.0 Universal)
29+
* [Animated Morph Sphere] (CC0 1.0 Universal)
30+
* [MorphStressTest] ([CC-BY 4.0] by Analytical Graphics, Inc, Model and textures by Ed Mackey)
31+
* [Morph-Primitives Test] ([CC-BY 4.0] by [ft-lab](https://github.com/ft-lab))
2732
* FiraMono by The Mozilla Foundation and Telefonica S.A (SIL Open Font License, Version 1.1: assets/fonts/FiraMono-LICENSE)
33+
34+
[Animated Morph Cube]: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/AnimatedMorphCube
35+
[Animated Morph Sphere]: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/AnimatedMorphSphere
36+
[MorphStressTest]: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/MorphStressTest
37+
[Morph-Primitives Test]: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/MorphPrimitivesTest
38+
[fox]: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/Fox
39+
[by PixelMannen]: https://opengameart.org/content/fox-and-shiba
40+
[by @tomkranis on Sketchfab]: https://sketchfab.com/models/371dea88d7e04a76af5763f2a36866bc
41+
[CC-BY 4.0]: https://creativecommons.org/licenses/by/4.0/

Cargo.toml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@ anyhow = "1.0.4"
236236
rand = "0.8.0"
237237
ron = "0.8.0"
238238
serde = { version = "1", features = ["derive"] }
239+
serde_json = "1"
239240
bytemuck = "1.7"
240241
# Needed to poll Task examples
241242
futures-lite = "1.11.3"
@@ -709,6 +710,16 @@ description = "Plays an animation from a skinned glTF"
709710
category = "Animation"
710711
wasm = true
711712

713+
[[example]]
714+
name = "morph_targets"
715+
path = "examples/animation/morph_targets.rs"
716+
717+
[package.metadata.example.morph_targets]
718+
name = "Morph Targets"
719+
description = "Plays an animation from a glTF file with meshes with morph targets"
720+
category = "Animation"
721+
wasm = true
722+
712723
[[example]]
713724
name = "animated_transform"
714725
path = "examples/animation/animated_transform.rs"

0 commit comments

Comments
 (0)