Multicanvas Render in React, Camera Keyboard Inputs not working on engine.InputTarget change

Pryme8 · February 2, 2021, 6:59pm

Hoooookay, this is kinda hard to demonstrate unless you go to here:

and do a local deploy of the editor.

Basically I can get the camera to see the mouse inputs but not the keyboard after the inputTarget is changed.

Pryme8/pane-editor/blob/master/src/babylonManager/engine/engineCache.ts#L47


    if(cacheScene.onInit){
        const onInit = cacheScene.onInit
        onInit()
        if(!cacheScene.initialized){
            cacheScene.initialized = true
        }      
    }
    return false
}
cacheScene.views.forEach((view)=>{
    const canvas: Nullable<HTMLCanvasElement>  = document.getElementById(`${view.canvasId}`) as HTMLCanvasElement
    if((canvas) && !view.view){ 
        view.view = slot.engine.registerView(canvas, cacheScene.scene.cameras[0])
        cacheScene.scene.detachControl()   
        slot.engine.inputElement = canvas
        cacheScene.scene.attachControl(true, true, true)                                                    
    }
    if(slot.engine.activeView === view.view){
        cacheScene.scene.render()
    }

is where the engine.inputTarget is changed.

And then this is where the input is bond when the react component mounts for the first time.

github.com

Pryme8/pane-editor/blob/master/src/windows/grass/grassBaseScene.tsx#L32


    const bm:BabylonManager = this.mutators.getBM()
    const engineSlot = bm.getEngineSlot('grassBase')
    if(!engineSlot){         
        const engineSlot = bm.createEngineSlot(`grassBase`)
        const cacheScene = bm.addSceneToSlot(engineSlot, {
            uid:`grassBase`,
            canvasId:`canvas-grassBase`
        }) 
        engineSlot.engine.inputElement = this.canvasRef.current
        const scene = cacheScene.scene            
        cacheScene.onInit = ()=>{ 
            scene.cameras[0].position.z = -4;        
            (scene.cameras[0] as any).setTarget(Vector3.Zero())      
            scene.cameras[0].attachControl(this.canvasRef.current, false)
            scene.cameras[0].inputs.add(new FreeCameraKeyboardMoveInput());
            this.grassBuilder = new DynamicGrassBlade({scene})
            globalThis.scene = scene
        }          
    }
}

Just ignore everything else about DynamicGrass. If this is too much for people to understand, I can prolly make an easier component to deal with.

But its going to be hard to test it outside of this React context, so anyone that takes the extra time to take a look I would much appreciate it.

Pryme8 · February 3, 2021, 5:13pm

Nobody else is interested in multicanvas input support and react? No one even clicked the link but me!!!

Savage… </3

Deltakosh · February 3, 2021, 6:42pm

haha man! I’m with you…It happens to me a lot as well

Pryme8 · February 5, 2021, 6:00pm

Bumpidy bump bump the bump bump bumped thread.

If someone is even willing to do a quick deploy of this on their local and take a look for a hour or two I to give me some advice or possibly fix it. I would gladly provide some compensation for your time seeing how this is kind of a lot to digest (just PM me before hand so we can agree to something).

Kinda doing this to understand the connection between react and webGL/BJS and how different methods can be deployed. I was able to do this with whole system already with Functional Components but it was not as efficient for the complex state management. Anyways I’m rambling… yeah someone come save me please.

Pryme8 · February 9, 2021, 4:16pm

aaaand bump… come on… someone has to have some sort of suggestion of something to try. Guess nobody wants to make any $$$…

Pryme8 · March 12, 2021, 6:29pm

Hooookay, sense there where not takers on this I decided to make a really simplified version of the same problem.

Could someone please help me figure out how to get the keyboard inputs working here?

Line 56 of GameSystem.tsx is where the new canvas is attached when the react layer is ready.

The mouse works, but I can not for the life of me get the keyboard to work.

I have tried a ton of different ways, this only happens with a react setup.

Deltakosh · March 12, 2021, 7:02pm

So this is a shoot in the dark but can you try to set your canvas.style.tabIndex = 1 (same with zIndex)

Pryme8 · March 12, 2021, 7:16pm

nope… hmm, honestly I’m pretty stumped by this.