I am not opposed of supporting it @Baktrian if you are willing to contribute it but it needs to be under a flag not enabled by default due to the impact on perf.
1 Like
I’m working on the feature update here. (Forgive the draft nature)
I’ve got the mask texture shader done which generates a mask texture in the shape of the UV layout, I would only need this once per mesh I would have thought.
I can flag it sure.
My current approach is not quite right, im passing the mask texture into the meshUVSpaceRenderer.fragment shader which has that seam issue regardless of how i update the fragColor.
When you say a new render pass would that be happening outside the decal shaders?
Able to find the edges of the UV where the seams are but need to now figure out why the seam stays visible and how to do the bleeding/ blending
I probably have to do this in another render pass
So im able to detect the seams, as well as generate the Mask Texture, what can I do to bring it all together ?
My idea is to try and add some padding/ texture bleeding but having some difficulty getting actually rid of the seam by covering it completely as im only able to get fragments to draw colour within UV bounds.
Generating a mask texture :
void main(void) {
gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
}
Result:
Generating UV Island edges:
varying vec2 vUV;
uniform sampler2D maskTexture; // The mask texture for seams
void main(void) {
// gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
// // Hard-coded texel size for a 1024x1024 texture
vec2 texelSize = vec2(1.0 / 1024.0, 1.0 / 1024.0);
// Sample the center pixel and surrounding pixels
float centerPixel = texture2D(maskTexture, vUV).r;
float rightPixel = texture2D(maskTexture, vUV + vec2(texelSize.x, 0.0)).r;
float leftPixel = texture2D(maskTexture, vUV - vec2(texelSize.x, 0.0)).r;
float topPixel = texture2D(maskTexture, vUV + vec2(0.0, texelSize.y)).r;
float bottomPixel = texture2D(maskTexture, vUV - vec2(0.0, texelSize.y)).r;
// Detect edges by checking if the center pixel is white but at least one of the neighbors is not
if (centerPixel > 0.5 && (rightPixel < 0.5 || leftPixel < 0.5 || topPixel < 0.5 || bottomPixel < 0.5)) {
// Current pixel is on the edge of a UV island
gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0); // Red
}
// Check if the UV coordinates are within the 0 to 1 range
else if (vUV.x >= 0.0 && vUV.x <= 1.0 && vUV.y >= 0.0 && vUV.y <= 1.0) {
// If inside the UV range, draw white
gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0);
}
}
Result:
meshUVSpaceRenderer.fragment:
varying vec2 vDecalTC; // Texture coordinates from the decal
varying vec2 vUV; // Original UV coordinates
uniform sampler2D textureSampler; // The main texture sampler for decals
uniform sampler2D maskTexture; // The mask texture for seams
void main(void) {
// Hard-coded texel size for a 1024x1024 texture
vec2 texelSize = vec2(1.0 / 1024.0, 1.0 / 1024.0);
// Sample the center pixel and surrounding pixels from the mask texture
float centerPixel = texture2D(maskTexture, vUV).r;
float rightPixel = texture2D(maskTexture, vUV + vec2(texelSize.x, 0.0)).r;
float leftPixel = texture2D(maskTexture, vUV - vec2(texelSize.x, 0.0)).r;
float topPixel = texture2D(maskTexture, vUV + vec2(0.0, texelSize.y)).r;
float bottomPixel = texture2D(maskTexture, vUV - vec2(0.0, texelSize.y)).r;
// Check if the current pixel is near a seam
bool nearSeam = centerPixel > 0.5 && (rightPixel < 0.5 || leftPixel < 0.5 || topPixel < 0.5 || bottomPixel < 0.5);
// Sample the decal texture
vec4 decalColor = texture2D(textureSampler, vDecalTC);
// Determine if the pixel is part of a decal (non-transparent pixel in the decal texture)
bool isDecal = decalColor.a > 0.5;
// Apply edge bleeding
if (isDecal && nearSeam) {
gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); // Draw green line over the seam
} else {
gl_FragColor = decalColor;
}
}
So this is the current state of the decal shader and is able to track the seams, just need a way to replace it completely now.
No colour is covering the actual seam, but we do get a green line as expected right up until the UV seam between the UV islands, before it just draws blank in the gap.
2 Likes
I will have a look as soon as possible.
1 Like
There was a technique implemented here which involves manipulating the vertices & uvs real time while painting
https://rawcdn.githack.com/sciecode/three.js/f4e363a8e0cf6c496f4191192d7eb15110442a7c/examples/js/TexturePainter.js
(three.js - texture - paint)
Gpu approach
https://sciecode.com/THREE.TexturePainter/src/TexturePainter.js
1 Like
The generation of the texture mask (TMask) looks good, though you should use a Constants.TEXTUREFORMAT_R format and a Constants.TEXTURETYPE_UNSIGNED_BYTE type, as you only need to store 0 or 1 values there.
From there, when you want to render the decal into the texture, my understanding of the algorithm is:
- you render the decal as before in the mesh texture TMesh (the
meshUVSpaceRenderershaders should not be changed) - you make an additional pass, taking TMesh and TMask as inputs, by using the code given in the article:
_MainTex is TMesh and _MaskTex is TMask. This step will generate the final texture TMeshFinal to use instead of TMesh when rendering the mesh
1 Like
Thank you for the response!
I think I’m almost there with the logic but need to understand some babylonjs tidbits:
TMeshis the decal texture being generated by the meshUVSpaceRenderer shaders.TMaskis the mask created from the mesh- in
MeshUVSpaceRenderer.ts,this.texturewhich usually holds the result of the decal texture will now hold the final texture of the decalTexture and the maskTexture which should output with the above shader. - If I assign
this.texturethe final result of the shader combining the decal with the UV Seam fix everything should work. - The part where the two are brought together in a different pass would look something like this?:
private _createFinalTexture(): void {
if (!this.texture) {
this.texture = new RenderTargetTexture(
"finalTexture",
{ width: this._options.width, height: this._options.height },
this._scene,
false,
true,
this._options.textureType
);
}
try {
this._scene.clearColor = new Color4(0, 0, 0, 1);
const finalMaterial = new ShaderMaterial(
"meshUVSpaceRendererFinaliserShader",
this._scene,
{
vertex: "meshUVSpaceRendererFinaliser",
fragment: "meshUVSpaceRendererFinaliser",
},
{
attributes: ["position", "uv"],
uniforms: ["worldViewProjection"],
samplers: ["decalTexture", "maskTexture"]
}
);
this._mesh.material = this._mesh.material as PBRMaterial;
finalMaterial .setTexture("decalTexture", this.decalTexture); // decalTexture being this.texture formerly
finalMaterial .setTexture("maskTexture", this._maskTexture);
finalMaterial .backFaceCulling = false;
this.texture.render();
} catch (error) {
console.error("Error creating mask texture:", error);
}
}
If i render the final pass (shader code is being written now), how do i actually get it to output as a render target texture without assigning the mesh? would i still be assigning the mesh? wouldn’t that limit me to the UV space for rendering?
Updated code;
Currently using a fullscreen quad, so far so good:
const fullScreenQuad = MeshBuilder.CreatePlane("image", { size: 1 }, this._scene);
fullScreenQuad.isPickable = false
fullScreenQuad.setEnabled(true);
fullScreenQuad.material = finalMaterial;
if (MeshUVSpaceRenderer._IsRenderTargetTexture(this.texture)) {
this._scene.customRenderTargets.push(this.texture);
this.texture.renderList?.push(fullScreenQuad);
this.texture.setMaterialForRendering(fullScreenQuad, finalMaterial);
this.texture.render();
}Yes for points 1 to 4.
No for point 5. Point 5 should be a simple post-process step. There’s no mesh rendering involved, and you should use the shader code from the article to generate the final texture decal from the source decal and mask textures.
1 Like
Thanks!
I’ve finally fixed it:
Some more testing, and will clean up the PR and flag the uv-seam fix, merge it into my fork and then with some feedback get it merged into the Engine!
4 Likes
So I’ve got the PR ready with all the checks complete.
Can also use this link to test it out:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14577/merge/index.html#N10DXG#271
1 Like