# Mesh of fingers jitter on mobile **URL:** \ **Category:** Questions\ **Tags:** mesh\ **Created:** [October 8, 2023, 7:45am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760 "2023-10-08T07:45:31Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 8, 2023, 7:45am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/1 "2023-10-08T07:45:31Z")
1. Mesh of fingers jitter on some mobile devices(Mali G77 & Mali G610), but fine on PC and iphone. 2. Visualize the bones with the inspector, the bones are not jittering. 3. jittering is severe on fingers, moderate on forearm, and fine on other parts of the body. Could it be floating points errors accumulated along the bone chain? Any suggestions on how to debug it will be helpful [https://youtube.com/shorts/abRyygeMI7w?feature=share](https://youtube.com/shorts/abRyygeMI7w?feature=share) ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 9, 2023, 9:34am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/2 "2023-10-09T09:34:12Z")
Welcome aboard! It looks like it could be a problem with floating point errors indeed. You can try to set `mesh.computeBonesUsingShaders = false` for all meshes with bones, to see if the problem comes from the GPU. When this property is `false`, all calculations are done on the CPU. ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 10, 2023, 9:53am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/3 "2023-10-10T09:53:19Z")
Thank you. it inspired me and turns out that > skeleton.useTextureToStoreBoneMatrices=false is enough to solve it. But size of uniforms are limited, so I tried to fix the shader this way ```auto mat4 readMatrixFromRawSampler(sampler2D smp,float index) { float offset=index *4.0; float dx=boneTextureWidthInv; float base=dx*(offset+0.5) vec4 m0=texture2D(smp,vec2(base,0.)); base+=dx vec4 m1=texture2D(smp,vec2(base,0.)); base+=dx vec4 m2=texture2D(smp,vec2(base,0.)); base+=dx vec4 m3=texture2D(smp,vec2(base,0.)); return mat4(m0,m1,m2,m3); } ``` The finger stop jittering, but the console log many errors (and they are basically glsl shader code). Is it because readMatrixFromRawSampler() is called in many shader, not just the skinning shader? And it’s weired that it can’t compile but the game keeps running as if no error… Edit: I forget to add “;” at the end of a line… typescript has reshape my brain… ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 10, 2023, 9:58am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/4 "2023-10-10T09:58:23Z")
Also try this: ```auto vec4 m0=texture2D(smp,vec2(saturate(dx*(offset+0.5)),0.)); ``` no jittering, but console log many errors (and they are complaining sth like “Unable to compile effect”) ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 10, 2023, 7:12pm UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/5 "2023-10-10T19:12:03Z")
`readMatrixFromRawSampler` is not used if you set `useTextureToStoreBoneMatrices = false`, so any change there won’t have any effect. However, bone matrices are now passed through uniforms, and you have a limited number of these that you can use in a shader. If you have too many bones, this method won’t work as expected and you will get some error in the console log, and it seems it’s what you get… To be sure, you should report here the errors you get in the console. ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 12, 2023, 2:36am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/6 "2023-10-12T02:36:37Z")
Yes, it is `useTextureToStoreBoneMatrices = false` that solved the problem, and narrow down to: 1. Passing floats to GPU through texture (compared to uniform) has bug on some device 2. readMatrixFromRawSampler has bug on some device (texture2D() driver bug?) I skip some texture updates by ```auto if (this.isUsingTextureForMatrices && mesh._transformMatrixTexture && Math.random()>0.9) { mesh._transformMatrixTexture.update(mesh._bonesTransformMatrices); } ``` and when texture is not updated, the mesh isn’t jittering, does this means readMatrixFromRawSampler is quite stable and can rule out speculation2? Any suggestions on how to debug this? ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 12, 2023, 8:55am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/7 "2023-10-12T08:55:22Z")
`mesh._transformMatrixTexture` is not null? Are you exporting a model from Max? If yes, can you export to a .gltf instead of a .babylon and see if that helps? ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 16, 2023, 11:24am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/8 "2023-10-16T11:24:46Z")
> `mesh._transformMatrixTexture` is not null? if useTextureToStoreBoneMatrices = false, then it is null. But for some devices the shader fails to compile because too many uniform. So I go back to > useTextureToStoreBoneMatrices = false and try to fix it. First try: replace texture2D with texelFetch, still jittering Second try: use fixed point number. I use int16 to represent float32. ```auto this._transformMatrixTexture = RawTexture.CreateRGBATexture( this.i16transformMatrices, (this.bones.length + 1) * 4, 1, this._scene, false, false, Constants.TEXTURE_NEAREST_SAMPLINGMODE, Constants.TEXTURETYPE_SHORT ); ``` and I replace the Constants.TEXTUREFORMAT\_RGBA with Constants.TEXTUREFORMAT\_RGBA\_INTEGER in CreateRGBATexture. But the avatar do not show on the screen and console log: GL\_INVALID\_OPERATION: Mismatch between texture format and sampler type ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 16, 2023, 12:08pm UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/9 "2023-10-16T12:08:57Z")
Try `TEXTURETYPE_HALFFLOAT` instead of `TEXTURETYPE_FLOAT` to see if it’s a problem with float texture support. Normalized SHORT or INTEGER won’t work because we need the values to go outside the -1…1 range (the texture stores matrices). > [@yangrui\_zhang](#): > > if useTextureToStoreBoneMatrices = false, then it is null. But for some devices the shader fails to compile because too many uniform. So I go back to So, you are using a model exported from MAX. Can you export this model as a .glTF file (instead of a .babylon) and see if that helps? What’s more, exporting to a .glTF file means you can test it with other glTF viewers (such as [https://gltf-viewer.donmccurdy.com/](https://gltf-viewer.donmccurdy.com/)). ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 17, 2023, 3:46am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/10 "2023-10-17T03:46:38Z")
> Try `TEXTURETYPE_HALFFLOAT` instead of `TEXTURETYPE_FLOAT` to see if it’s a problem with float texture support Wow. I get the same jittering on PC after > ToHalfFloat(this.\_transformMatrices[i]) It’s surprising that some devices decide to use HalfFloat, even if `TEXTURETYPE_FLOAT` is passed in to the `CreateRGBATexture` ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 17, 2023, 6:39am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/11 "2023-10-17T06:39:37Z")
This is fixed by using TEXTURETYPE\_INT. 1. SpectorJs can not ensure the internalFormat the hardware is using. I capture this on the mobile, it indicates boneTexture is using RGBA32F. But my last post indicates that the problem is caused by texture using half float. ![js](https://us1.discourse-cdn.com/flex024/uploads/babylonjs/original/3X/1/c/1cc0f0d5c1b8f5d3d1db589e22a616aae1fa469e.jpeg) 1. I read a blog claiming that “type is a hint for precision, but GL can choose any internal precision to store the texture”. I’m not sure if he is right, but on [glTexImage2D - OpenGL ES 3 Reference Pages](https://registry.khronos.org/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml) I find a table : | **Sized Internal Format** | **Format** | **Type** | **Red** | **Green** | **Blue** | **Alpha** | | | --- | --- | --- | --- | --- | --- | --- | --- | | GL\_RGBA16F | GL\_RGBA | GL\_HALF\_FLOAT, GL\_FLOAT | f16 | f16 | f16 | f16 | | | GL\_RGBA32F | GL\_RGBA | GL\_FLOAT | f32 | f32 | f32 | f32 | | Both GL\_HALF\_FLOAT and GL\_FLOAT can lead to GL\_RGBA16F? 1. for anyone who come across the same issue: First, create a texture with format Constants.TEXTUREFORMAT\_RGBA\_INTEGER Second, turn float32 into int32. In my project, all model is smaller than 2\*\*11 cm, so I think it is fine. > this.i32transformMatrices[i] = Math.round(this.\_transformMatrices[i] \* (2\*\*20)) Third, chang the BoneDecalaration.fx > precision highp isampler2D; > uniform isampler2D boneSampler; > uniform float boneTextureWidthInv; > #else > uniform mat4 mBones[BonesPerMesh]; > #ifdef BONES\_VELOCITY\_ENABLED > uniform mat4 mPreviousBones[BonesPerMesh]; > #endif > #endif > #ifdef BONETEXTURE > #define inline > mat4 readMatrixFromRawSampler(isampler2D smp,float index) > { > int base=int(index\*4.0); > vec4 m0=vec4(texelFetch(smp,ivec2(base ,0),0) )/exp2(20.0); > vec4 m1=vec4(texelFetch(smp,ivec2(base+1,0),0))/exp2(20.0); > vec4 m2=vec4(texelFetch(smp,ivec2(base+2,0),0))/exp2(20.0); > vec4 m3=vec4(texelFetch(smp,ivec2(base+3,0),0))/exp2(20.0); > return mat4(m0,m1,m2,m3); > } ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 17, 2023, 8:09am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/12 "2023-10-17T08:09:34Z")
There is another bonesDeclaration.fx in folder ShadersWGSL . ```auto var boneSampler : texture_2d; ``` Should I change it into this? ```auto var boneSampler : texture_2d; ``` The doc says it is for WebGPU. My concern is, will WebGPU be enabled if Babylon detect the device support it, and the shader will fail to execute (because I mess up i32\f32)? ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 17, 2023, 8:58am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/13 "2023-10-17T08:58:19Z")
> [@yangrui\_zhang](#): > > Both GL\_HALF\_FLOAT and GL\_FLOAT can lead to GL\_RGBA16F? In my understanding of the spec, if a GPU indicates that it supports FLOAT textures, it shouldn’t be allowed to switch to HALF\_FLOAT if you created a texture as FLOAT… > [@yangrui\_zhang](#): > > Should I change it into this? Only if you want to use these changes in WebGPU. Babylon.js will not automatically switch to another engine. If you explicitly create a WebGL engine, you are guaranteed to work with WebGL. ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 18, 2023, 1:44am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/14 "2023-10-18T01:44:23Z")
> uniform highp sampler2D boneSampler; just find out that highp precision solve this elegantly I read a blog, it says even if CPU pass f32 to GPU, ALU can use f16 when doing calculation. In opengl, mediump may be default? ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 18, 2023, 9:30am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/15 "2023-10-18T09:30:26Z")
That’s the first time I hear that setting a precision has an actual impact on the output! I wonder if we should add “highp” on our end, the problem being that if it’s not supported, it will generate an error (according to the spec)… What do you think @sebavan? ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 18, 2023, 10:50am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/16 "2023-10-18T10:50:41Z")
> > > https://github.com/AnalyticalGraphicsInc/cesium/issues/6739 observed that Cesium…'s float batch tables didn't seem to be working at high enough precision on some devices. > From https://groups.google.com/forum/#!topic/webgl-dev-list/aFmFGeyRVRQ, devices are free to select the precision of \`sampler2D\` when it isn't explicitly set. > We should update our batch table code to use \`uniform highp sampler2D\` when backing the batch table with float textures. > > As part of this, please roll back changes from https://github.com/AnalyticalGraphicsInc/cesium/pull/7421 and test on a device that shows red on https://likangning93.github.io/webgl-examples/tests/floatTexturePrecision/ but no red on https://likangning93.github.io/webgl-examples/tests/floatTexturePrecisionHighpSampler/. > > Good candidates as of this writing are the iPhone 7 or newer and the first generation iPad Pro or newer. > **[WebGL best practices - Web APIs | MDN](https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#be_precise_with_glsl_precision_annotations)** > > WebGL is a complicated API, and it's often not obvious what the recommended ways to use it are. This page tackles recommendations across the spectrum of expertise, and not only highlights dos and don'ts, but also details why. You can rely on this... MDN writes:" [In WebGL 1, “highp float” support is optional in fragment shaders](https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#in_webgl_1_highp_float_support_is_optional_in_fragment_shaders)" Does it mean, WebGL2.0 always support highp float, and in WebGL1.0 vertexShader support highp float? what about highp sampler2D ? ---
**Author:** ![sebavan](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/sebavan/32/57_2.png) [@sebavan](https://forum.babylonjs.com/u/sebavan)\ **Post date:** [October 18, 2023, 9:58pm UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/17 "2023-10-18T21:58:01Z")
should we add an extra mode for it ? ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 19, 2023, 12:26am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/18 "2023-10-19T00:26:59Z")
Do you mean just like > bool skeleton.useTextureToStoreBoneMatrices add a boolean > bool skeleton.useHighpSampler and decide whether add precision-annotation in glsl or not? ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [October 19, 2023, 4:23pm UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/19 "2023-10-19T16:23:25Z")
Could you provide us with a repro? We would like to check on other platforms. Also, can you check if Babylon.js is started in WebGL1 or WebGL2 mode on the platforms where it fails? ---
**Author:** ![yangrui\_zhang](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/yangrui_zhang/32/35625_2.png) [@yangrui\_zhang](https://forum.babylonjs.com/u/yangrui_zhang)\ **Post date:** [October 20, 2023, 1:39am UTC](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760/20 "2023-10-20T01:39:47Z")
1. My fix has merged into master and take effect, otherwise I could just give you url to this project… 2. Can reproduce it on Mali Gpu by taking a random character playing Idle animation(Idle to make it more noticable, like my youtube video). 3. Can reproduce the jittering on PC by RawTexture.CreateRGBATexture(…, Constants.TEXTURETYPE\_HALF\_FLOAT) 4. Another more noticeable way is to scale Float32Array \_transformMatrices. You can prove that scaling every element of a transform matrix make no different, unless it exceed 65535 and overflow float16. In the video below, I scale every element with t (t is time). Some bone’s matrix has element whose value is around 200, so when t reach 300, mesh of hand disappear. other parts of mesh disapper as t increases… (this can help you quickly identify if a platform is using f16. On PC however you scale the element , nothing happens) 1. My phone print the info in chrome console: “{“driver”:“ANGLE (ARM, Mali-G610 MC6, OpenGL ES 3.2)”,“vender”:“Google Inc. (ARM)”,“webgl”:“WebGL 2.0 (OpenGL ES 3.0 Chromium)”,“os”:“Android 10”}”. Float16 is a new feature aim to improve performance, so I guess it is newer gpu that has this problem: GL spec deault lowp sampler2D in vertex shader, and mali gpu decide to use float16 for lowp. [Next page](https://forum.babylonjs.com/t/mesh-of-fingers-jitter-on-mobile/44760.md?page=2)