But when I tested it with a brand new browser, the flickering disappeared.
This may have something to do with the browser environment
Do not forget that a web browser is a big beast. Garbage collection, other processes, JS engine…many things can happen.
I found the reason for my flickering here, the jitter goes away when I disable the Immersive Web Emulator extension.
Exactly. And a physics emulation should always run smooth in the background. This is why one needs interpolation which is what I am trying to get at by documenting flicker/stutter WITHOUT rigidbodies 