Feasibility of setting environment textures reflection matrix every frame

Questions
#1

In trying to avoid the camera shake of an XR device, Quest 2 as an example, I have tried a couple of methods which have involved NOT altering on a dimension basis, the cameras rotation / position for very small changes from the previous frame.

While this does work, the big gains are not observed before there is the side effect of noticing.

The primary beneficiary though are PBR materials, especially ones that do a lot of reflection. One thing I was thinking about was not altering the camera, but use the same prior frame checking to alter the reflection texture instead to counteract the rotation and stabilize the reflection.

If it worked, the major advantage is it would probably not produce a side effect until beyond observing a large improvement. And any side effect would also probably not involve ones sense of balance.

The quick setup would be:

let smallYChange = n;
let smallXChange = n2;
let texture;

const mat = texture.getReflectionTextureMatrix();
BABYLON.Matrix.RotationYawPitchRollToRef(smallYChange, smallXChange, 0, mat);

// already sort did this, by using the reference, but it manages the change
texture.setReflectionTextureMatrix(mat);

While feel free to poke holes in this anywhere, looking at the stuff in setReflectionTextureMatrix(), below, the scene markAllMaterialsAsDirty() seems ominous. Is that going be a devistating performance hit?

/**
 * Returns the reflection texture matrix
 * @returns the reflection texture matrix
 */
public getReflectionTextureMatrix(): Matrix {
    return this._textureMatrix;
}

/**
 * Sets the reflection texture matrix
 * @param value Reflection texture matrix
 */
public setReflectionTextureMatrix(value: Matrix): void {
    if (value.updateFlag === this._textureMatrix.updateFlag) {
        return;
    }

    if (value.isIdentity() !== this._textureMatrix.isIdentity()) {
        this.getScene()?.markAllMaterialsAsDirty(Constants.MATERIAL_TextureDirtyFlag, (mat) => mat.getActiveTextures().indexOf(this) !== -1);
    }

    this._textureMatrix = value;
}
#2

I would expect so. For example, this is the reason why scene.blockMaterialDirtyMechanism exists to avoid the dirty flag recomputing everything.

#3

While it is not even relative if is does not work (probably 3 hours to program), I did some more superficial checking, reflectionMatrix is one of the uniforms.

Doesn’t that means it is going to be passed every draw call anyway? So what if the value is not the same?

Probably no to reply, other than to educate, if it is bust. It’s Friday morning, should know today.

#4

I’m curious about this solution, do you just freeze the camera instead of making tiny changes to the angle?

#5

Yes, but implemented sort of backwards. I replaced the sub-cameras in the WebXRCamera with my own TargetCamera subclass. Each frame the cameras are assigned their new rotations & positions (they are both the same across left & right). The function _checkInputs() is then going to be called in the rendering process before anything is drawn.

In my override of _checkInputs(), when it is decided that something is not different enough from the prior frame, then I change it back. The effect is to freeze that particular dimension, but the change has already been made. I am changing it back.