# Problem with instancedBuffers in Babylon React Native

**URL:** <https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904>\
**Category:** Bugs\
**Tags:** babylon-native\
**Created:** [September 13, 2022, 8:01pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904 "2022-09-13T20:01:06Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 13, 2022, 8:01pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/1 "2022-09-13T20:01:06Z")

</div>

@bghgary @Cedric I’m having trouble with instancedBuffers in Babylon React Native, specifically changing UVs per instance, which results in unpredictable changing of vertex position rather than UVs, but no error.

I found this open issue:

> <https://github.com/BabylonJS/BabylonReactNative/issues/232>
>
> Hello! I'm trying to color mesh instances on \`BabylonReactNative\` scene using th…is example: https://playground.babylonjs.com/#YPABS1
> But when i move code that works in playground to \`BabylonReactNative\` scene all instances keeps appearing black.
> Is there any specific \`BabylonReactNative\` issues with \`instancedBuffers\` ?

But there’s also a mention of some work having been done on this in the merged PR:

> <https://github.com/BabylonJS/BabylonNative/pull/990>
>
> This PR needs https://github.com/BabylonJS/Babylon.js/pull/11900 that's in beta.…6. package.json has been updated to reflect this change. VT need also this PR : https://github.com/BabylonJS/Babylon.js/pull/11917 this PR will be updated to reflect the change (NPM update) but it won't have any impact on Native code.
> It will be difficult to fully understand changes without reading this comment. 
> 
> \# Context:
> bgfx instancing is quite limited: only 5 vec4 attributes can be used for instancing (world matrix + color basically). These attributes named i\_data0 .. 4 and D3D semantic texcoord3..7 are bound with the instance buffer data when available.
> No more than 5 vec4 special streams! see https://github.com/bkaradzic/bgfx/issues/2008#issuecomment-578819111
> Most of this work would have been unnecessary if it was possible to set \`instance divisor\` to any stream. Like OpenGL/WebGL allows (and I guess, any other graphic API). In bjs, any stream can have a divisor==1. 
> More over, streams for instancing are stored in a different buffer (with a lifetime of 1 frame). Where bjs treats any stream (instance or vertex buffer) the same.
> 
> The change is divided in 2 parts:
> \- make difference in recorded vertex streams so Native Engine knows if it must be stored as an instance buffer or a regular vertex buffer.
> \- Find and modify vertex attributes in shaders so bgfx can bind them as instanced buffer. This work is dependant on the rendering API.
> 
> \# Vertex Buffer vs Instance Buffer
> 
> Now, instance divisor is forwarded to native when recording.
> If the divisor is 1, there are pushed into the \`m\_vertexBufferInstanceRecords\` map. A divisor == 0 is a regular vertex stream. When binding a vertex array for rendering, these buffer arrays are merged together into a bgfx instance buffer. This instance buffer lives for 1 frame (no destruction/storage). All the instance streams must be grouped inside this instance buffer in this pattern:
> \[i\_dataA\[0\], i\_dataB\[0\], i\_dataC\[0\]....
> i\_dataA\[1\], i\_dataB\[1\], i\_dataC\[1\]...\]
> 
> \## Direct3D
> Vec4 datas are stored in reverse order of semantics: https://github.com/bkaradzic/bgfx/blob/4581f14cd481bad1e0d6292f0dd0a6e298c2ee18/src/renderer\_d3d11.cpp#L2701
> Vertex buffer is stored in ascending order in \`VertexArray\` because \`m\_vertexBufferInstanceRecords\` is \`std::map\<bgfx::Attrib::Enum, InstanceVertexBufferRecord\>\` so vertex buffer iteration is in reverse order in \`VertexArray::SetVertexBuffers\` as pointer to bgfx instance array is increased linearly. Same for OpenGL.
> 
> \## Metal and OpenGL
> Order of vec4 is the same as the location. No reverse iterator here.
> 
> \# Shader attribute definition
> 
> \## BabylonJS
> Before https://github.com/BabylonJS/Babylon.js/pull/11900 there was 1 attribute named \`color\`. it either contained Mesh Vertex colors OR instance Color. This was simple: depending on divisor and bound vertex buffer, the same shader code use the color to render the mesh. It was not possible to have Mesh vertex color AND Instance color at the same time but it was not a need. But as native shaders need name to properly bind the attribute, name needs to be different and \`instanceColor\` has been added.
> 
> \## Direct3D
> Quite simple: in \`VertexVaryingInTraverserD3D\`, add new attribute names from BabylonJS, bind them as expected semantic for bgfx and rename them as well. As bgfx binds with semantics, the new \`instanceColor\` has the appropriate semantic.
> 
> \## OpenGL
> Binding is based on name, hence the renaming (i\_data0...i\_data5). As long as i\_dataX and location ares in increasing order, it's fine. First instance attribute, first served.
> 
> \## Metal
> Binding based on attribute location. The difference here is bgfx is binding the last streams (a mix between OGL and D3D). It means, if for some reason (order of attributes in bjs shaders) attributes appear like this to the shader:
> \- location 0: instanceColor
> \- location 1: position
> \- location 2: world0
> \- location 3: world1
> \- location 4: world2
> \- location 5: world3
> It needs to be remapped to :
> \- location 0: position
> \- location 1: instanceColor
> \- location 2: world0
> \- location 3: world1
> \- location 4: world2
> \- location 5: world3
> That's why there are 2 passes. 1st one for stream, second for instanced ones.
> 
> \## Other technical possibilities
> 
> It's always possible to have a different technical path but with time allowed, this is the best I could do.
> 
> Other possibilities:
> \- modify bgfx to allow instance divisor per stream. It would be in sync with webGL. This tiny bit of work cover a change in API contract and also reimplementing instances for a few platforms (D3D11, D3D12, Metal, OpenGL, Vulkan) while keeping the backward compatibility. This also implies discussing the work with Branimir before.
> \- discard HW instancing and add loop on instance instead. As many \`submit\` call as mesh to render. Instance attributes need to be replaced by uniform. This means some work in Shader Traversers and also some questions: if a vertex buffer is used with a divisor ==1, how is it known when compiling shader that 1 (or more) particular attribute must be replaced by a uniform? Then, what's the performance will be? Especially on mobile.
> 
> \# Tests
> All platforms tested OK!
> 
> Related : https://github.com/BabylonJS/BabylonReactNative/issues/232 #992

So I’m a bit confused about the status.

Here’s some PG examples of the types of `instancedBuffers` usage I’m experiencing trouble with:

> **[Babylon.js Playground](https://playground.babylonjs.com/#64K1I1%236)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

> **[Babylon.js Playground](https://playground.babylonjs.com/#P7GNPG%239)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

---

<div class="post-metadata">

**Author:** ![bghgary](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/bghgary/32/48532_2.png) [@bghgary](https://forum.babylonjs.com/u/bghgary)\
**Post date:** [September 13, 2022, 8:29pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/2 "2022-09-13T20:29:30Z")

</div>

I’ll let @Cedric take this one.

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 14, 2022, 8:38am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/3 "2022-09-14T08:38:53Z")

</div>

Instancing with BabylonNative/BabylonReactNative is very limited because of constraints on our rendering back end.  
Basically, you can only instanciate the world matrix and the color. That’s a total of 5 vec4.  
For now, it’s not possible to instanciate uv or anything else.  
If you need different UV per instance, I would try to store an UV offset in the color. Like an index in an atlas texture.  
So, with this technic, you will have to write your vertex shader accordingly.

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 9:34am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/4 "2022-09-14T09:34:37Z")

</div>

Thanks @Cedric

So just use a vec4 and instancedBuffer of stride length 4? Name can be anything, so I could call it uvOffset4?

Something like the following?

```auto
mesh.registerInstancedBuffer("uvOffset4", 4);

// Dynamically set uOffset and vOffset somewhere ...

instance.instancedBuffers.uvOffset4 = new Vector4(uOffset, vOffset, 0, 0);

```

Then in the shader, something like:

```auto
attribute vec4 uvOffset4;
vec2 uvOffset = vec2(uvOffset4[0], uvOffset4[1]);
vMainUV1 = uvUpdated + uvOffset; 

```

Is that right?

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 14, 2022, 9:45am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/5 "2022-09-14T09:45:42Z")

</div>

No, create a buffer named `color`, just like if you want to instanciate colors.  
Then, in the shader, get the color vec4 value and use it to store something else than colors.

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 9:47am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/6 "2022-09-14T09:47:28Z")

</div>

Ok, I thought maybe the name doesn’t matter, but all good, I will use color.

Thanks again @Cedric !

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 9:49am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/7 "2022-09-14T09:49:53Z")

</div>

Will just update that code snippet in case someone else stumbles on it.

```auto
mesh.registerInstancedBuffer("color", 4);
// Dynamically set uOffset and vOffset somewhere ...
instance.instancedBuffers.color = new Color4(uOffset, vOffset, 0, 0);

```

Then in the shader, something like:

```auto
attribute vec4 color;
vec2 uvOffset = vec2(color[0], color[1]);
vMainUV1 = uvUpdated + uvOffset; 

```

Better?

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 14, 2022, 9:58am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/8 "2022-09-14T09:58:42Z")

</div>

yes! and it leaves you 2 floats for more instancing infos 🙂

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 11:02am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/9 "2022-09-14T11:02:20Z")

</div>

Hi @Cedric, I’ve tried this in Babylon React Native but am now getting a shader compilation error:

`BJS - [20:53:19]: Error: Exception in HostFunction: ERROR: 0:158: 'color' : redefinition`

I am creating this shader as a material plugin for `StandardMaterial`.

Then I tried commenting out `attribute vec4 color;` and got the warning `WARNING: Fail to create vertex buffer. Number of vertex buffers higher than max count or too many instanced streams.`

Do I need to apply this workaround just with a simple shader, not a material plugin of a `StandardMaterial`?

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 14, 2022, 12:20pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/10 "2022-09-14T12:20:36Z")

</div>

Do you have other buffers set as instance buffers?  
can you share access to your PG code?

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 7:31pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/11 "2022-09-14T19:31:12Z")

</div>

No that `color` instanced buffer is the only one I’ve set in the entire codebase.

Here’s a PG that replicates the issue, so I’ve done something wrong …

> **[Babylon.js Playground](https://playground.babylonjs.com/#P7GNPG%2310)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

---

<div class="post-metadata">

**Author:** ![Blake](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/blake/32/21509_2.png) [@Blake](https://forum.babylonjs.com/u/Blake)\
**Post date:** [September 14, 2022, 8:11pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/12 "2022-09-14T20:11:08Z")

</div>

Could try setting useVertexColors to false on the mesh so that the standard material doesn’t use the color attribute.

> **[Babylon.js Playground](https://playground.babylonjs.com/#P7GNPG%2311)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 8:14pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/13 "2022-09-14T20:14:33Z")

</div>

Thanks @Blake !

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 14, 2022, 11:41pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/14 "2022-09-14T23:41:45Z")

</div>

Hmmm, sorry me again @Cedric , but I just can’t seem to get this working in Babylon React Native. Here’s an adjusted PG showing exactly what I’m trying to do in the way it’s implemented in Babylon React Native (as per all the suggestions above). This code works fine in the playground, but in my Babylon React Native codebase the geometry is messed up and just unlit black but no errors or warnings.

> **[Babylon.js Playground](https://playground.babylonjs.com/#P7GNPG%2313)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 15, 2022, 11:45am UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/15 "2022-09-15T11:45:34Z")

</div>

I’m taking a closer look.

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 15, 2022, 1:02pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/16 "2022-09-15T13:02:25Z")

</div>

I did it a bit differently:

> **[Babylon.js Playground](https://playground.babylonjs.com/#3BU7P2)**
>
> Babylon.js playground is a live editor for Babylon.js WebGL 3D scenes

I removed `https://playground.babylonjs.com/#3BU7P2` and changed slightly `CUSTOM_VERTEX_MAIN_END`

Then, I used `thinInstanceSetBuffer` And I got it working with native.

---

<div class="post-metadata">

**Author:** ![inteja](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/inteja/32/22553_2.png) [@inteja](https://forum.babylonjs.com/u/inteja)\
**Post date:** [September 15, 2022, 7:03pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/17 "2022-09-15T19:03:14Z")

</div>

Thanks for looking into this @Cedric

So I need to use thin instances rather than instances?

---

<div class="post-metadata">

**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\
**Post date:** [September 16, 2022, 1:56pm UTC](https://forum.babylonjs.com/t/problem-with-instancedbuffers-in-babylon-react-native/33904/18 "2022-09-16T13:56:43Z")

</div>

For native, yes.