How to reset sections of matricesWeights without affecting the geometry?

There seems to be an issue with resetting mesh weights
What is the way to make certain vertices be unaffected by bones?

In this PG, I set the weightIDs to -1 but it becomes all spaghettified.

Note: I know that I could set all the weightIDs to 0 (the first bone index) but this simply binds the vertex to the first bone. I would like the vertex to be affected by no bones, while preserving the vertex position

Hi mrlooi,

I’m not sure there’s a way to do exactly what you’re trying to do. Messing with rigging data at runtime is fraught to say the least—this is really the sort of thing that should be done in a DCC tool like Blender or Maya—and in this particular case changing this data retroactively may expose you to all sorts of implementation details within the engine code. For example, the way Babylon applies a skeleton to a mesh accumulates the influences into a matrix to do all the multiplication once; but if there are no influences, that matrix will be full of zeros, which is not a valid homogeneous transform. I don’t know for sure whether this particular code is what’s causing the exact behavior you’re seeing, but trying to manipulate this sort of mesh data at runtime will expose you to many situations where low-level assumptions may be accidentally broken, causing problems.

If you have access to the mesh and can do what you’re trying to do offline, I would definitely recommend changing the rigging using a DCC tool such as Blender. If you absolutely have to do it at runtime, though, I’d recommend splitting the mesh in two, taking the vertices you don’t want to be affected by a skeleton into a separate mesh to which you simply never apply a skeleton. That should allow you to achieve the effect of having certain vertices unaffected by the rigging without having to try to modify the rigging logic itself at runtime. Hope this helps, and best of luck!

2 Likes

Thank you @syntheticmagus , I know that you can use DCC tools but I would very much like to do everything in runtime to create a simple rigging tool all with BJS. I know that it’s possible. In fact I’m close but this is one of the last few glitches that I need to resolve

Is there really no other solution other than to split the mesh into weighted/non-weighted?
Couldn’t we technically tweak those lines in mesh.ts to make sure that if it’s all zeros it just leaves the base mesh vertex untouched?

For instance, at codeblock, line 4308

if (totalWeight < eps) {
    finalMatrix = Matrix.Identity()
} 

I’m not really familiar enough with that part of the code to be sure what all effects that might have. It sounds reasonable enough to me, but if that’s a behavior you need then I think the next step is to make an official feature request on Babylon.js. That’ll get the matter in front of the team members who know this stuff best and, if it’s as simple and innocuous a change as it looks to be, it should be a quick fix!

1 Like

Unfortunately changing applySkeleton wouldn’t quite work (PG here)
I think we’d need to change the shaders itself
Do you know where I can find the relevant shader code?

You have to set mesh.computeBonesUsingShaders to false for that CPU skinning function to be used. Otherwise the shader version is implemented here: bonesDeclaration.fx and bonesVertex.fx

1 Like

What if before this shader line here

we add if (influence <= 0) return

Setting this in the PG produces error
image

It was missing some type declarations (for VertexBuffer and Matrix) and finalMatrix needed to be changed from const to let. :slight_smile:

IDK about the shader part thou, maybe better to just create an extra bone with identity matrix to avoid any branching? Will be interesting to see what the core devs think of the issue. :slight_smile:

1 Like

For example here’s what I meant by adding an extra bone for the identity matrix, then any vertices that are transformed only by this bone should be unaffected by the skeleton (i.e. it should be the same as if you null the skeleton on line 57 since you’re doing it for all vertices I think).

Also you could try using bones.length for the matrix index for vertices that should be unaffected by the skeleton, but then you’d have to change that value if you later add or remove a bone - since identity matrix is copied to the end of the bone transforms array, the index to use for it depends on the number of bones (well according to my reading of the source code here. :slight_smile:

1 Like

Thank you @Blake ! really appreciate this :smiley:

Though the issue becomes that the mesh no longer binds to the skeleton base matrix, therefore losing its shape (in the PG it becomes much smaller)

1 Like

If you want it to look the same as before could try using getPositionData to bake the current bone transforms into the vertices. The normals are still wrong but I think the upcoming getNormalsData function could help bake those into the vertices as well. :slight_smile: :beers:

1 Like

Yea I know :slight_smile:
Though in the end this approach doesn’t solve the problem of arbitrary vertices having no weights, which is what the original post was asking about :frowning:
I think the only option is to change the shader code itself :fearful:

Hmm on rereading your first post of the thread I feel like this does address it, the vertices are “affected by no bones, while preserving the vertex position” like you want as far as I can tell… :thinking:

But if you want to do arbitrary vertices, instead of all them you like in your PG, then you could change the position and matrices data for just those vertices instead of changing it for all of them…

Or maybe an easier, better way will be found after all. :beers: :slight_smile: :zzz:

1 Like

Update on this topic

I realized that in 3D software e.g. Maya, if a mesh is skinned i.e. has matricesWeights, then these weights cannot be removed unless they ‘unbind skin’.
If a bone weight is removed (eroded via subtractive weightpaint) from a vertex, then Maya re-assigns this bone weight to a different bone instead. The effective total weight for a vertex is always normalized to 1.
And I think this approach makes sense.

So I’m going to close this topic here since the original question, while makes sense on paper, isn’t necessarily a good approach and is not compatible with approaches in existing 3D tools

Thanks a lot again @Blake and @syntheticmagus for the help!

1 Like