BabylonNative: Video / WebSockets API / W3C things?

Hi - was really excited to see the work on BabylonNative :clap:

I’m about to start a new project this year and am weighing up whether to go all in with future BabylonNative (ie. continue building in JS until a 1st release is ready).

NativeCamera is really promising but I was wondering if anything like NativeVideo is planned (for example H264 playback)? WebSockets will also be really crucial - so in general I guess I am requesting W3C type APIs that are in the browser.

I can see that Chakra is used for Windows, JavascriptCore for Apple, and V8 for Linux - which I assume are runtimes for Javascript without the client-side APIs.

For video cameras and playback, perhaps OpenFrameworks could be a good reference point:

For communication, maybe just a way to pipe messages into the app, handled like window message events:

cc @BabylonNative

We actually already have a NativeVideo implementation, but it is only to support NativeCamera at the moment. For video playback, you might be able to get away with using ExternalTexture and playback the video manually per platform.

WebSockets

There is support for web sockets coming, but it will only be for Windows, Android, iOS initially.

The rest don’t seem to be questions, so I’m not sure you are looking for. Let us know if we can answer any more questions.

1 Like

We actually already have a NativeVideo implementation, but it is only to support NativeCamera at the moment. For video playback, you might be able to get away with using ExternalTexture and playback the video manually per platform.

Great to hear - on Github I see Linux / Mac backends for NativeCamera are placeholders for now. Is it known what will handle those eventually, for example things like V4L2 or AVFoundation cams? And for NativeVideo - will this eventually open and decode, for example, an H264 video file?

There is support for web sockets coming, but it will only be for Windows, Android, iOS initially.

:metal:

The rest don’t seem to be questions, so I’m not sure you are looking for.

  • Video playback of H264 file: <video src="myvideo.mp4" />
  • Video from UVC devices: navigator.mediaDevices
  • Websockets client: new WebSocket(url, protocols)
  • Servers? require('websocket'); require('https'); :crossed_fingers::genie:

We don’t have plans to implement this at the moment. Contributions are welcome.

I’m still not sure if there is a question. If you are asking if these will be supported. I think I’ve already answered.

We don’t have plans to implement this at the moment. Contributions are welcome.

Thanks for the clarification. I’ll look into it… a tutorial & example for best practice writing plugins would be nice.

I’m still not sure if there is a question. If you are asking if these will be supported. I think I’ve already answered.

Foolish to start a discussion on discourse hub, innit

Yes, I agree, we’ve talked about creating a repo with simple examples, but we don’t have the resources to create this at the moment unfortunately. There are many plugins available to review in the repo which should give some ideas.

I’m not sure what you mean by this. Let us know if you have questions that we can help answer.