[jak1] Actor Collision Mesh Issues

[gratefulforest]

Acknowledgements

• I have searched for duplicate or closed bug reports
• I understand that I am supposed to provide my own legitimately obtained copy of the game

Describe the Bug

There are currently four distinct issues related to the visuals of collision meshes:

1. If a collision mesh is assigned to an armature that only has one bone, the visuals of the collision mesh will attach and animated to Daxter/Jak's shoulder.

2. If a collision mesh uses the same material as another collision mesh, any subsequent collision meshes appear invisible. If multiple objects have no material (and get the black and white checker pattern in OpenGOAL), only the first object with the checker pattern gets the checker pattern (and others appear invisible).

3. All collision mesh's visuals (whilst receiving the animation/movements of the bone) get their location placed at 0,0,0 worldspace in Blender (or the entity's location). This is an issue since the collision of the collision mesh gets the bone's location, whilst the visuals of that collision mesh get 0,0,0.

4. Even if you put your bone at 0,0,0 (to align the collision with the visuals), there is still another issue, which is, the visuals receive the transformation/rotations of the animation, from a different bone than the collision:
   
   

How To Reproduce

I've provided two example meshes, one uses the trick of placing the bone at 0,0,0 to align the visuals, in an attempt to isolate one of the issues:

Wrong Bone + Visuals Aligned at 0,0,0

Wrong Bone + Visuals Misaligned

Both of these meshes has a cube called "DummyMeshIgnore" which is entirely non-existent and is just there so Blender exports an animation.
To use these examples, just assign the primitive to bone index 4, (set! (-> s2-0 transform-index) 4)

Example Mesh for Multiple Materials for this example you need to write two collision primitives and assign them both to bone index 2, (set! (-> s2-0 transform-index) 2)

Example Mesh for Attach to Daxter for this example a single collision primitive should be assigned to bone index 2, (set! (-> s2-0 transform-index) 2). This example also has a dummy mesh.

Here's another two really clear examples, mainly for the fact that they "prove" the collision mesh's visuals are being assigned to a different bone than the collision mesh's collision:

Static Visuals Moving Collision

Spinning Visuals Static Collision

Both of these just go on bone index 4 again. (There's only one difference between the two files, which is changing which bone on the armature spins).

2025-03-09.11-43-26.mp4

2025-03-09.11-43-54.mp4

Does this problem occur on original hardware or PCSX2?

Yes, it's unique to OpenGOAL

Expected Behavior

This is game breaking for many actors, some of which have no alternative or workaround. In some select circumstances, if you get lucky all you need to do is duplicate all your collision mesh, create an identical set up over the top just for the visuals again, merge them all into a single object, split apart their mesh islands, line them out in a row, assign identical weights to your code, then go through the process of trying to line up the visuals with the collision meshes. This process quickly becomes unviable on more complex actors, as well as doubles art group file sizes unnecessarily, as well as flat out not working in certain circumstances.

There are many circumstances where this doesn't work, take for instance the object above "Spinning Visuals Static Collision", in this example, that solution can't be used, since the collision is getting the wrong movement, and the visuals are getting the right movement. In this example, there's nothing you can do, and running the whole thing twice so there's a separate visual mesh over the top, with duplicated set up processes, isn't possible.

The multiple materials bug is also a significant issue due to the way collision primitives are split.

Fix to be Added

In this section, where the visuals of collision meshes are built, there is no reference to a bone index that the visuals are being given:

https://github.com/open-goal/jak-project/blob/1f6438e517ac590b6745ba1e40256ad415c82941/goalc/build_actor/common/MercExtract.cpp#L243C1-L264C6

  std::vector<jak1::CollideMesh> cmeshes;
  cmeshes.reserve(mesh_count);
  // we extracted all of the prim data for each mesh, now combine
  for (size_t p = 0; p < mesh_data.size(); p++) {
    auto& prims = mesh_data.at(p);
    std::vector<math::Vector4f> verts;
    std::vector<jak1::CollideMeshTri> tris;
    int vert_count = 0;
    for (auto& prim : prims) {
      if (prim.pat.ignore) {
        continue;
      }
      vert_count += verts.size();
      verts.insert(verts.end(), prim.verts.begin(), prim.verts.end());
      for (size_t i = 0; i < prim.indices.size(); i += 3) {
        auto& tri = tris.emplace_back();
        tri.vert_idx[0] = prim.indices.at(i);
        tri.vert_idx[1] = prim.indices.at(i + 1);
        tri.vert_idx[2] = prim.indices.at(i + 2);
        tri.pat = prim.pat.pat;
      }
    }

Unlike below, where the actual collision has "cmesh.joint_id = joint_idx".

This means that for the visuals, since there is no reference to a bone index, it is probably falling on a default or being given an adjacent one in the upstream process gltf_util.cpp.

I suggest passing the joint_idx to gltf_util.cpp, so that it can be included on this line:

jak-project/common/util/gltf_util.cpp

Line 223 in 1f6438e

math::Vector4f v_w = w_T_local * v_in;

So that the vertices can be assigned to a bone instead of world transformation on that line.

There is now also, a pre-built game files already ready to go to test the fix:
Collision Mesh Test Zone

2025-03-02_08-36-32.mp4

This means nothing is needed to set it up, all you have to do is go to the right of the village hut.

If you turn on the Actor Marks, the name of the actors describe which bug is which, and you just need to turn on collision marks as well as try standing on them.

Fix for Multiple Materials

This is a fix I've added to the extract function, what it does is change the material key to a compound key that also includes the object, so that the same material on multiple objects isn't skipped. However, it also isn't built twice, by building the materials first and referencing the built materials afterwards instead.

void extract(const std::string& name,
             gltf_util::MercExtractData& out,
             const tinygltf::Model& model,
             const std::vector<gltf_util::NodeWithTransform>& all_nodes,
             u32 index_offset,
             u32 vertex_offset,
             u32 tex_offset) {
  ASSERT(out.new_vertices.empty());

+ std::map<std::pair<int, int>, tfrag3::MercDraw> draw_by_material; // added compound key to include object {material_index, node_index}
- std::map<int, tfrag3::MercDraw> draw_by_material;
  int mesh_count = 0;
  int prim_count = 0;
  int joints = 3;
  auto skin_idx = find_single_skin(model, all_nodes);
  if (skin_idx) {
    joints += gltf_util::get_joint_count(model, *skin_idx);
  }

  for (const auto& n : all_nodes) {
    const auto& node = model.nodes[n.node_idx];
    if (node.extras.Has("set_invisible") && node.extras.Get("set_invisible").Get<int>()) {
      continue;
    }
    // gltf_mesh_extract::PatResult mesh_default_collide =
    //     gltf_mesh_extract::custom_props_to_pat(node.extras, node.name);
    if (node.mesh >= 0) {
      const auto& mesh = model.meshes[node.mesh];
      mesh_count++;
      for (const auto& prim : mesh.primitives) {
+       int mat_idx = prim.material;
        // get material
        // const auto& mat_idx = prim.material;
        // gltf_mesh_extract::PatResult pat = mesh_default_collide;
        // if (mat_idx != -1) {
        //   const auto& mat = model.materials[mat_idx];
        //   auto mat_pat = gltf_mesh_extract::custom_props_to_pat(mat.extras, mat.name);
        //   if (mat_pat.set) {
        //     pat = mat_pat;
        //   }
        // }
        // if (pat.set && pat.ignore) {
        //   continue;  // skip, no collide here
        // }
        prim_count++;
        // extract index buffer
        std::vector<u32> prim_indices = gltf_util::gltf_index_buffer(
            model, prim.indices, out.new_vertices.size() + vertex_offset);
        ASSERT_MSG(prim.mode == TINYGLTF_MODE_TRIANGLES, "Unsupported triangle mode");
        // extract vertices
        auto verts =
            gltf_util::gltf_vertices(model, prim.attributes, n.w_T_node, true, true, mesh.name);
        out.new_vertices.insert(out.new_vertices.end(), verts.vtx.begin(), verts.vtx.end());
        out.new_colors.insert(out.new_colors.end(), verts.vtx_colors.begin(),
                              verts.vtx_colors.end());
        out.normals.insert(out.normals.end(), verts.normals.begin(), verts.normals.end());
        ASSERT(out.new_colors.size() == out.new_vertices.size());

        if (prim.attributes.count("JOINTS_0") && prim.attributes.count("WEIGHTS_0")) {
          auto joints_and_weights = gltf_util::extract_and_flatten_joints_and_weights(model, prim);
          ASSERT(joints_and_weights.size() == verts.vtx.size());
          out.joints_and_weights.insert(out.joints_and_weights.end(), joints_and_weights.begin(),
                                        joints_and_weights.end());
        } else {
          // add fake data for vertices without this data
          gltf_util::JointsAndWeights dummy;
          dummy.joints[0] = 3;
          dummy.weights[0] = 1.f;
          for (size_t i = 0; i < out.new_vertices.size(); i++) {
            out.joints_and_weights.push_back(dummy);
          }
        }

+       auto key = std::make_pair(mat_idx, n.node_idx); //compound key with object

+       if (draw_by_material.find(key) == draw_by_material.end()) {
+         auto& draw = draw_by_material[key];
+         draw.mode = gltf_util::make_default_draw_mode();  // Default mode
+         draw.tree_tex_id = (mat_idx == -1) ? 0 : gltf_util::get_texture_id(model.materials[mat_idx]);  // fallback for checker pattern
+         draw.num_triangles = 0;
+         draw.no_strip = true;
+       }

-       // real draw details will be filled out in the next loop.
-       auto& draw = draw_by_material[prim.material];
-       draw.mode = gltf_util::make_default_draw_mode();  // todo rm
-       draw.tree_tex_id = 0;                             // todo rm
+       auto& draw = draw_by_material[key];
        draw.num_triangles += prim_indices.size() / 3;
-       draw.no_strip = true;
        draw.index_count = prim_indices.size();
        draw.first_index = index_offset + out.new_indices.size();
        out.new_indices.insert(out.new_indices.end(), prim_indices.begin(), prim_indices.end());
      }
    }
  }

  tfrag3::MercEffect e;
  tfrag3::MercEffect envmap_eff;
  envmap_eff.has_envmap = false;
  out.new_model.name = name;
  out.new_model.max_bones = joints;
  out.new_model.max_draws = 0;

+   for (const auto& [key, d_] : draw_by_material) {
+     int mat_idx = key.first;
+     const auto& mat = (mat_idx >= 0) ? model.materials[mat_idx] : tinygltf::Material();  // checker pattern
+     if (mat_idx < 0 || !gltf_util::material_has_envmap(mat) || !gltf_util::envmap_is_valid(mat)) {
-   for (const auto& [mat_idx, d_] : draw_by_material) {
-     const auto& mat = model.materials[mat_idx];
-     if (mat_idx < 0 || !gltf_util::material_has_envmap(model.materials[mat_idx]) ||
-         !gltf_util::envmap_is_valid(model.materials[mat_idx])) {
      gltf_util::process_normal_merc_draw(model, out, tex_offset, e, mat_idx, d_);
    } else {
      envmap_eff.has_envmap = true;
      gltf_util::process_envmap_merc_draw(model, out, tex_offset, envmap_eff, mat_idx, d_);
    }
  }

  // in case a model only has envmap draws, we don't push the normal merc effect
  if (!e.all_draws.empty()) {
    out.new_model.effects.push_back(e);
  }
  if (envmap_eff.has_envmap) {
    out.new_model.effects.push_back(envmap_eff);
  }

  for (auto& effect : out.new_model.effects) {
    out.new_model.max_draws += effect.all_draws.size();
  }

  lg::info("total of {} unique materials ({} normal, {} envmap)", out.new_model.max_draws,
           e.all_draws.size(), envmap_eff.all_draws.size());
  lg::info("Merged {} meshes and {} prims into {} vertices", mesh_count, prim_count,
           out.new_vertices.size());
}

I've also added some handling so that if multiple objects have no material (material index -1) they still each individually get the checker pattern.

2025-03-13.19-10-25.mp4

Fix for Attach To Daxter

In the example, it doesn't actually attach do Daxter, since there are other merc models around for it to attach to instead (you can see it on the WrongBoneVisualsAligned actor instead) and it appears/disappears based on if your camera is seeing the true bounds location (not the displaced location). I assume this happens when the lack of a bone index being specified for the collision mesh visuals causes it to receive a default or dummy index, and, when this happens, it is receiving a bone index that does not exist on the armature, so, it is drawing on another merc model.

I have a suspicion that this is not actually it's own issue but something that will automatically fix with the bone index fix.

Environment Information

Blender 4.3.2
OpenGOAL v0.2.21

Game Version

NTSC 1.0 (black label)

Have you set the game to something other than 60fps?

No

[gratefulforest]

Hey guys, any ideas on this? I've been trying fixes for this since I first opened the old issue 6 months ago, so any breakthroughs would be greatly appreciated!