How do I extract mesh vertex data properly?

Questions
1

Hey ya’ll

Just wondering what is the right method to extract vertex data onto the CPU (esp vertex data bounded to a skeleton)?
I’ve tried all sorts of ways and they don’t seem to work well + look hacky.

For example, I’ve tried

mesh.computeBonesUsingShaders = false; // set to CPU
await new Promise<void>((resolve) => {  
        // wait for vertex data to be loaded to the CPU post render loop?
	scene.onAfterRenderObservable.addOnce(() => {
		resolve();
	});
});
mesh.refreshBoundingInfo(true);
mesh.computeWorldMatrix(true);
mesh.updateFacetData();
const vertices = mesh.getVerticesData(VertexBuffer.PositionKind);

But I don’t think it always works. The vertices data still looks different from the shader/GPU stuff

@Pryme8 @sebavan @Evgeni_Popov @syntheticmagus

2

I think your only option will be to make sure all your displacements are happening on the CPU.

Otherwise if like animations and other things are happening through the GPU can be considered fairy dust and not really exist in any tangible way that you can reference.

I might be wrong though, but I assume you are trying to get some info from a mouse click on a GPU deformed mesh right?

You could do a GPU picking method but that would require you to pack all your data on the shader and when you preform the pick have the meshes shaders display that data and then collect the pixel information from the screen point the decode that data. It’s hard to get more than just IDs or position something nested in that though.

For the skeleton render it was way easier to just toggle it to CPU animation mode. But if you can’t for performance reasons then the GPU pick might be the only option and just do like IDs and have an associated map with the IDs and what data you want ready to reference once you pull the GPU ID.

What data is coming back wrong in your setup?

2 Likes
3

So actually I’m looking to get the actual vertices to do stuff like skinning/deformations. I don’t care about performance since it’s just a one time thing.
For example, a user places a bone on the arm of the mesh. I would need the mesh vertices at that point in time to do skinning on the bone. And ofcourse the skinning algorithm needs to run on CPU data.
The problem I’ve found is that the CPU vertex data is not the same as what’s on screen. The arm is one place but the CPU data suggests otherwise

A much more advanced usecase than skeleton rendering but I think BJS could make vertex updating etc easier for devs to do more interesting stuff :slight_smile:

4

There’s an internal function _getPositionData that can return the vertex positions after applying the skeleton:

const vertices = mesh._getPositionData(true); // true to apply skeleton

Or if you don’t want to use it directly, it could still serve as reference for how to apply the skeleton. :slight_smile:

5

Thanks!
I’m guessing this function is an improvement over getVerticesData for CPU extraction?

6

Well it’s used to refresh the bounding info for example and other places where the vertex positions need to have the skeleton and/or morphing applied in JS, but it’s not part of the official API…

1 Like
7

This should be a public method! Thanks Blake : )

1 Like
8

It looks like there’s some other internal updating that the function does too, so maybe better to move just the parts that apply the skeleton and morphing into a public function (that uses the same helper as the private version would)… or maybe better to just add a flag to the existing function to skip the internal updating? Curious what @sebavan will think about this one. :slight_smile:

9

Yea I really think this should be the default for getting vertices instead of getVerticesData(PositionKind) :slight_smile:

10

Yeah I’m not super keen on exposing that specific function as it handles a lot of internals. I would prefer have an additional function with only the data retrieval part

12

Okay here’s a little PR to add a separate exposed function that just gets the data and applies the skeleton/morph, which is also called by the hidden version that handles the internals, to get the ball rolling. :slight_smile:

EDIT: welp seems I went astray somewhere, after I get up and have breakfast I’ll try to find where… :thinking: Okay back on it, hopefully can get it validating… And hurray, looks like it’s good now. :slight_smile:

2 Likes
13

Heya, just a quick update: the user/API version of getPositionData() is available in the latest release now. :slight_smile:

Examples:

// Get unmodified vertex positions
positions = mesh.getPositionData();

// Apply skeleton
positions = mesh.getPositionData(true);

// Apply morph
positions = mesh.getPositionData(false, true);
3 Likes
14

cheers, thanks Blake!

1 Like