AA for VideoTexture from webcam stream

I was wondering if someone could point me in the right direction for getting rid of jagged edges (anti-aliasing) in a VideoTexture coming from a webcam? Straight edges can appear jagged at certain viewing angles. Raising the anisotropic filtering level seems to make it worse? Also, the higher the resolution of the camera output the worse it gets.

My code is nothing fancy, very similar to this:

Does it happen with all of your webcams?
I didn’t notice the effect.
Also, have a look a this version with 4 AA samples - https://playground.babylonjs.com/#582KAL#3

2 Likes

Hello just checking in if you’d like any more help? :slight_smile: @Gnorme

1 Like

Thanks for checking in :slight_smile:

It’s not very noticeable on lower resolution cameras. The higher the resolution, the worse it seems to be.

I made just a few changes to that playground to try for high resolution when getting the camera and to prefer the environment facing camera which is usually the higher resolution one (and easier to see the effect while moving around). Try this on a mobile device with a decent camera:

I ended up having to turn away from using BabylonJS for now. I was trying to apply filters and effects to live video but on top of the AA issues, I couldn’t achieve a steady 30fps on a lot of devices when combining it with MediaRecorder to record the canvas to a video.

@sebavan and @bghgary are working on video related stuff so they might have some tips :thinking:

This is really strange as there is no AA involved for texture, only for meshes.

Could you share a screenshot of how it looks ?

Please excuse the atrocious tape job on the AC.

This angle captured it well. It’s not usually so bad.

The AA artifacts here seems to be present in the source unfortunately :frowning: and generated by the camera.

Babylon is doing nothing there but copy the data to the screen, this is so weird…

Can you check the raw data with smthg like this :frowning:

The AA artifacts don’t show up when I’m not using BabylonJS.

They don’t show up when I use the native camera app and they don’t show up when I use navigator.getUserMedia() and attach the stream to a canvas or video element tag.

I had to go back to using plain canvas and getUserMedia because of the AA.

Do you see this as well in the default case https://playground.babylonjs.com/#582KAL#2 and in any other browser or just ios ?

It’s not as obvious but yes it happens in the default case.

I can confirm it happens on 3 different iPhone models.
I’m pretty sure it happens on android and PC also but it’s been a while since I checked. I’ll be able to double check later today or tomorrow.