Problems with getWorldMatrix - positions not updated

Hi, this is a difficult problem to report - my project to animate a human skeleton now runs to 8 classes and thousands of lines of code so I cannot reproduce a playground example.

[ Please note I am not using any kind of external rigging, mesh deformations, skins - when I use the term bone I mean something that looks like a real bone in your body - there is a risk of confusion here. I am performing non-deforming transforms on meshes that resemble human bones - all relationships and joints are handled by my code - nothing to do with more usual animation and rigging examples ]

Basically the problem is that I have never been able to get the recommended method for updating vertex or mesh positions after transformations to work in order that I can immediately use their new (updated) world co-ordinates to calculate other transformations - before the render. It seems I have to wait for a render in order that I can read updated vertex positions in world space.

I have read the recommended fixes along the lines of

mesh.computeWorldMatrix();
var matrix = mesh.getWorldMatrix(true);
var local_position = new BABYLON.Vector3(0,1,0);
var global_position = BABYLON.Vector3.TransformCoordinates(local_position, matrix);

but I have never been able to get this to work in my code, tried to use it in various situations over the months, as part of a new sub-project I now encounter the same familiar problem again.

I have started a new version of my code which has the pelvis as the root or start of the chain of parent-child relationships which describe the structure of the bones of the skeleton, the root used to be a bone in the foot.

The structure simplified is…

root => joint => bone => joint => bone => joint => bone => joint => bone

The joints are rotational with varying degrees of freedom.
The root is just a special bone because it is the head of the chain - it has no parent.
The bones are all meshes, the joints are either small spheres (debug mode) or TransformNodes (non-debug mode) - performance reasons when I do not need to see the joint positions.

Now that I have changed the linkage so that the root bone is the pelvis and not the foot the consequence is that an instruction to get the skeleton to “squat” means that instead of moving down the skeleton now raises his legs and hangs suspended - this was expected, the penalty I have to pay for more flexibility is that I have to implement a new class called “gravity”.

The job of gravity is to say - “ohh the legs have folded at various joints and because the pelvis is now the root the legs will simply have raised up leaving the skeleton hovering above the ground - I better calculate how much to move the entire skeleton down so his feet are still on the ground.”

The basic idea in the code is…

  1. rotate thigh bone in pelvis joint one increment
  2. rotate lower leg about knee one increment
  3. rotate foot so its still horizontal
  4. scan absolute position of vertices in foot to find lowest - once found measure distance to floor.
  5. move entire skeleton up or down to by the amount calculated in 4 so that the lowest part of the foot is now on the ground.
  6. Render
  7. Repeat go back to 1 - next incremental movement in the animation.

Always the problem I find is that the reported absolute positions of vertices in step 4 is not up to date with the changes in the rotation of bones in steps 1 2 3 infact the positions reported correspond to the previous pass through all the steps.

This “lag” would be expected if I were not calling mesh.getWorldMatrix(true) because the render has not yet updated the world matrix for each mesh in the chains of bones, however I am calling getWorldMatrix(true).

I have actually written a temporary function in the class responsible for bones and joints that calls it on absolutely every one of my meshes before and after the leg bones have been rotated. Its deliberate absolute overkill but it just does not seem to help - it seems that whenever I transform meshes in the same piece of code that I then try to calculate geometric consequences of those transformations that the transformations have not yet been applied to the world matrix to enable me to obtain the new world co-ordinates. I have been in this position previously in other areas of my code.

I just wanted to check if there is anything obvious that I should know about?

Characteristics that might be peculiar to my example and not others…

  1. Almost all movements are pure rotations - indeed the only translations involved are ones implemented in the class gravity that are intended to push the skeleton back onto the ground so he does a squat and not a leg raise. Prior to the implementation of gravity there were no translations involved at all only rotations of bones about joints, the joint usually having been in turn rotated about another joint and so on. Is there something about long chains of parent child meshes in which each child is rotated in the local space of its parent and that parent in turn is rotated in the local space of its parent and so on.

  2. Long chains - there are hundreds of bones in my model - are there any hard limits set up in the mesh.getWorldMatrix(true) which do not act due to timeouts or lengths of child parent chains?

Babylon can cope with my chains of bones - I can animate the skeleton in real time with no evidence of it having problems - the updating must be working correctly in the render it just seems that I cannot provoke the same update inbetween render frames so that I can perform rotations, measure new relationships and then make adjustments in the form of secondary transformations in this case to move the skeleton so that its lowest points are back on the ground.

Always the pattern is that after a render everything tallies up but I cannot seem to provoke the same state of resolution in my code that the render achieves.

Eventually I intend the gravity class will become more sophisticated and automatically identify the lowest 3 non adjacent points on the entire skeleton and automatically reposition so that these form a stable base on the ground so I do need the general solution.

Thanks for any advice on what could be wrong - I know previous reports of bugs in this area have either been fixed or not verified but to me it looks as if there is still something here that is amiss.

On tablet, short answer, the bones are deforming the mesh vertices according to matrix weights on the Gpu. Javascript is unaware of change. The copy of the vertex data the cpu has is unchanged.

You can force the skeleton to be skinned on cpu, but you will have a performance hit.

1 Like

Hi - thanks but there is no deformation - this is is not using any Babylon rigging or animation - the entire rigging and animation is bespoke.

The skeleton is a real skeleton as a medical person would understand it - when I say bone I mean a mesh that looks like a bone in your body like a jaw bone or your skull or a vertebra in your spine its not the same thing as a bone used in proprietary rigging software - in my code bones are real bones as a medical person would understand the term.

My meshes all move as complete meshes - movement is not achieved through any kind of deformation or external rigging - I am not using any kind of priority rigging or animation, the code I have written controls all movements and relationships.

But thanks anyway…

Note that getWorldMatrix does not take any parameter, so getWorldMatrix(true) won’t force a recomputation: pass true to computeWorldMatrix to force a recomputation.

However, you should not need to force recomputation if you call computeWorldMatrix() for all meshes: the recomputation will be automatically done if it has to be. Still, try to pass true to it and see what happens.

The Scene._evaluateActiveMeshes function, which is responsible for finding the meshes to display and is called by Scene.render => Scene._renderForCamera is simply looping through the meshes and doing:

mesh.computeWorldMatrix();

to update the world matrix of all meshes. So, if you are doing the same you should get the same result.

1 Like

Hi Evengi,

Thanks…

I have tried computeWorldMatrix(true) in the past on different occasion, infact I have tried both calls one after the other, however an experiment just now with my “overkill” code delivered a different result which I need to investigate - I had not used the “overkill” code on previous occasions so the experiment just now is probably the first time I have tried the combination of computeWorldMatrix(true) with “overkill” code but I have tried it before in the normal way ( ie not overkill ).

I just tried a second experiment where I downgraded my overkill code to only do the mesh in question ( with computeWorldMatrix(true) ) and that delivered the familiar wrong result.

Right now it looks to me as if I need the combination of the overkill code AND computeWorldMatrix(true) - this suggests that when I call on a single mesh computeWorldMatrix(true) it is not working its way down the chain the way I intend it to do.

This gives me a lead - I will gradually downgrade the overkill code and see when it stops working and then try and work out why computeWorldMatrix(true) does not seem to be able to traverse my chains but my overkill code seems to be able to do this.

More investigation - thanks for moving on a problem that has been a periodic issue - as long as you can provoke a change in the code you can usually work it out.

I wonder is there a kind of datestamp on chains for performance reasons - for instance if I force an update on the matrix for say 3 (below) and then ask for an update on say 2 will the code say - ohh I already updated this when I did the update for 3 so I do not need to do it again? I was thinking maybe a kind of incremented integer that acts like a datestamp and says “no need all transforms in the chain have the same datestamp” ?

root = 1 = 2 = 3 = 4

I am just intrigued because I cannot see any obvious slowdown due my “overkill” code yet it must be visiting the same meshes many times each time I use it, I just added every mesh reachable from every one of my bone classes irrespective of whether it would naturally appear later on - kind of “throw everything you can find in the list”.

I think there is a documentation bug at least in one place but I think I have seen it in other places as well…
https://doc.babylonjs.com/resources/frame_of_reference

quote:

mesh.computeWorldMatrix();
var matrix = mesh.getWorldMatrix(true);
var local_position = new BABYLON.Vector3(0,1,0);
local_position.addInPlace(new BABYLON.Vector3(1, 1, 1));
var global_position = BABYLON.Vector3.TransformCoordinates(local_position, matrix);

Is that not at odds with the information you provided?
On the basis of my experimentation (incomplete) I would say I think you are right but it seems to me that the doc is wrong, I think this is repeated elsewhere.

Thanks again
Jon

Update: The combination of “overkill” and mesh.computeWorldMatrix(true) is working, my skeleton is keeping both feet on the ground - the remaining bug is the failure to update all matrices without the overkill - perhaps something in my code, I will dig deeper. A pragmatic (inefficient) fix but not the correct one yet but very pleased to be moving forward.

You are correct, PR submitted to correct.

Actually computeWorldMatrix returns the matrix so the getWorldMatrix is not needed.

var matrix = mesh.computeWorldMatrix(true); //true forces a recalculation rather than using cache version
var local_position = new BABYLON.Vector3(0,1,0);
var global_position = BABYLON.Vector3.TransformCoordinates(local_position, matrix);
var matrix = mesh.computeWorldMatrix(true);
var local_position = new BABYLON.Vector3(0,1,0);
local_position.addInPlace(new BABYLON.Vector3(1, 1, 1));
var global_position = BABYLON.Vector3.TransformCoordinates(local_position, matrix);

Thanks for pointing it out. If you come across other docs errors please let us know where they are.

2 Likes

Hi JohnK - thanks, yes I will report doc bugs if I see them.

Just a thought on giving more scope for answers? How about, in a playground, connecting a few boxes to form a chain in the manner of your bones and show what calculations you need prior to rendering, what you actually get and how that differs after rendering.

Hi John - it would be a major job even for a few boxes - there are layers of abstraction, I am using classes that have grown and grown over the months I have been working on the project - after all that work I might end up with an example that simply does not show the problem. I am in a much better position now because I now have something that actually works (the overkill code) so I can compare against the non-working example and shake it to see what drops out.

At this stage I think it is likely to be more productive if I try to track the problem in my own code and then see if the problem is in my own code - if the problem is in Babylon then yes it might later be try to code up something for the playground,

Since the issue seems to be obtaining data during the pre-render calculations that are used during rendering I though perhaps joining a few boxes using parenting and applying the world matrix might give some insight into where the issue lay. Sometimes going back to basics can give a clue that is useful in solving a problem in an already complex code.

However you now have strategy that works for you. Let us know how it goes. :slightly_smiling_face:

Update: Yesterday I said that it appeared that I needed both my “overkill” code and of course the correct arguments to computeWorldMatrix ( documentation bug ).

Today I re-ran everything and used conditionals to bypass the “overkill” calls to computeWorldMatrix and was surprised to find no problem.

I had previously encountered this type of problem in other areas of code - if I forget what I thought I saw yesterday I would conclude that all previous errors in the behavior of my code were due to the documentation bug which I believe I have followed every time I have been in this situation.

The difference between todays results and yesterdays result would most likely be either a straightforward mistake on my part (possible - I had been coding for around 9 hours ) or something that is not properly initialised elsewhere.

I am going to run for now with my overkill code receiving a flag telling it not to do all the additional calls only to do it for the mesh of interest and simply keep an eye on it.

Its not too difficult to spot when the code is not working correctly as my skeleton vibrates up and down on account of code designed to keep his feet on the floor receiving out of date information because of the failure to update all matrices in the chain

I will flag Evengi_Popovs reply as solution for now.

Thanks to everyone who replied - and I will keep monitoring this to make sure it does not re-appear.

Jon

2 Likes