Fixing DeviceOrientationCamera noize for android devices

Bugs
1

Noize on DeviceOrientationCamera on android devices.

I tried to find a solution for shaking on android devices.

  1. At first I tried to test as many devices as possible. We found out the shaking occurs on most android devices. Tested on Samsung galaxy s20, Samsung galaxy s20 FE, Samsung Galaxy A51, Xiaomi 10 and some other android devices. It worked fine on OnePlus and all newer iPhones. So it doesn’t work smooth on about 50% of all devices.

Playground here: https://www.babylonjs-playground.com/#14KRGG#198

It also doesnt work smoothly on youtube: VR 360 Video of Top 5 Roller Coaster Rides 4K Virtual Reality - YouTube

  1. We found out, that this kind of shaking doesnt happen on VR look. So I guess it gets more accurate data on device orientation than DeviceOrientationCamera. So maybe you already solved the problem, but you implemented it only on VR looks.

Playground here: https://playground.babylonjs.com/#TJIGQ1#3

  1. We looked at Karpano, who managed to solve this problem. Their simple explanation on how they managed to solve this problem from innacurate deviceorientation data event. krpano.com - Plugins - Gyro2

One of their example is embedded in 360 cities: https://www.360cities.net/

Google also managed to solve the problem on Google 360 view: https://maps.app.goo.gl/eCH9eFP6kHZRmu2S8

  1. We tried to solve the problem ourselves and we were able to smoothen the deviceorientation event data by setting (on ArcRotateCamera) camera.alpha and camera.beta with their last 10 average event values. But the problem occurs with event.alpha values when phone is at 90 degrees or more (upwards). We managed the orientation to work fine in landscape mode, but we cant fix it for portrait mode.

Our Demo (Only works on android devices): https://www.babylonjs-playground.com/#14KRGG#613

We also analyzed data from deviceorientation and absolutedeviceorientation event, and it is just not as smooth and as accurute as is it on iOS devices.

Questions? Is there anything going on on updating the results of DeviceOrientationCamera smoothness on android devices? Is there any advice you would give to our code? Can you explain us how exactly DeviceOrientationCamera gets their data, and if it is possible to smoothen it in any way?

Most likely solution:
I think the solution might be to do averages on DeviceOrientationCamera for android devices. The movement might be a litlle slower, but the camera would move smoothly. This would give much better user experience in my opinion.

Best Regards,

2

cc @RaananW :slight_smile:

3

This is more of a device issue that directly a camera issue.
You could use some noise-reducing filter, depending on the noise you are experiencing. I had experience with applying a kalman filter on sensor data (when low-pass was out of the question), but that information would have to be fed to the camera, emulated instead of directly using events as it is now.

About andoird vs. iphone - this is not an operating system issue, but a build quality issue. as much as I dislike apple’s OS, I very much appreciate their hardware quality. With Android, it very much depends on the manufacturer. Some high-end devices (like the s-line of samsung) do deliver proper sensor data, and some cheaper models will, for obvious reasons, deliver less than optimal results.

Sorry for not giving any proper solution…

1 Like
4

Thanks for the answer, I know this also a device problem. But as I described, karpano and also zappar, somehow found a solution to this problem and even described how they found a solution for this problem, but cant access their source code since it is encrypted.

Is there a possibily to add this as a reasearch/feature for next releases? (so I dont spam this reasearch in feature request also).

I will also try to find a solution for this problem in the next few days, but im just not as smart as your team :slight_smile: (my math is not on so high level)

5

We can investigate on our end if we can end a filter for the seonsor data, but as this is not a main topic it might be pushed to the backlog and stay there for quite some time… I would investigate javascript-based kalman filter, I am sure there are a few implementations that are rather straight forward. No need to reinvent the wheel :slight_smile:

2 Likes