Handling texture coordinates across different platforms in BabylonNative

kmatzen · December 9, 2024, 6:54pm

If I construct a RawTexture on OpenGL in BabylonNative or in the browser with BabylonJS, the texture appears to be interpreted according to OpenGL texture coordinate conventions, whereas the same code with D3D11 or Metal in BabylonNative interprets it flipped as is the convention with those platforms. I took a look and saw that NativeEngine delegates the texture data upload directly to bgfx without conversion. Is there a utility function that BabylonNative provides so that I can write clean cross platform code without manually checking which graphics platform is in use and flipping textures myself?

sebavan · December 9, 2024, 8:41pm

cc @BabylonNative

Cedric · December 10, 2024, 9:04am

Hi @kmatzen

Texture is flipped vertically when rendering. UV Tex coord is flipped when sampling textures.

github.com

BabylonJS/BabylonNative/blob/6f8f211d7ddf89b9ec9f622701b9feb36654fa1b/Plugins/NativeEngine/Source/NativeEngine.cpp#L945


      
                  patchedVertexSource += "gl_Position.z = (gl_Position.z + gl_Position.w) / 2.0; }";
                  return patchedVertexSource;
              }
              return vertexSource;
          }
          
          std::string ProcessSamplerFlip(const std::string& vertexSource)
          {
              const auto* caps = bgfx::getCaps();
              // for d3d, vulkan, metal, flip the texture sampling on vertical axis
              if (!caps->originBottomLeft)
              {
                  std::string patchedVertexSource = vertexSource;
          
                  static const std::string shaderNameDefineStr = "#define SHADER_NAME";
                  const auto shaderNameDefine = vertexSource.find(shaderNameDefineStr);
                  if (shaderNameDefine != std::string::npos)
                  {
                      static const auto textureSamplerFunctions = R"(
                          highp vec2 flip(highp vec2 uv)
                          {

And it’s also flipped when preparing image

github.com

BabylonJS/BabylonNative/blob/6f8f211d7ddf89b9ec9f622701b9feb36654fa1b/Plugins/NativeEngine/Source/NativeEngine.cpp#L253


      
              image->m_format == bimg::TextureFormat::RGBA16F ||
              image->m_format == bimg::TextureFormat::RGBA32F);
          
          assert(image->m_depth == 1);
          assert(image->m_numLayers == 1);
          assert(image->m_numMips == 1);
          assert(image->m_cubeMap == false);
          
          if (bgfx::getCaps()->originBottomLeft ? invertY : !invertY)
          {
              FlipImage({static_cast<uint8_t*>(image->m_data), image->m_size}, image->m_height);
          }
          
          if (srgb && !bgfx::isTextureValid(1, false, 1, Cast(image->m_format), BGFX_TEXTURE_SRGB))
          {
              bimg::ImageContainer* oldImage{image};
              image = bimg::imageConvert(&allocator, bimg::TextureFormat::RGBA8, *image, false);
              bimg::imageFree(oldImage);
          
              assert(bgfx::isTextureValid(1, false, 1, Cast(image->m_format), BGFX_TEXTURE_SRGB));
          }

So, it should be transparent for most usages. Can you please elaborate or provide a PG that highlights your issue?

Topic		Replies	Views
Babylon Native custom shader texture sampler inverted in iOS? Questions shaders , babylon-native , glsl	5	421	August 3, 2023
Difference in shader uniforms passed to Babylon Native? Questions babylon-native	2	422	December 6, 2022
Flip Imagedata from dynamic Texture Questions	5	520	January 16, 2020
Texture on custom setVerticesData Questions	2	328	April 14, 2023
Constructing CubeTextures in BabylonNative Questions babylon-native , cubetexture	3	32	November 25, 2024

Handling texture coordinates across different platforms in BabylonNative

Related topics