VR Avatar Support

Feature requests
1

I would like to make the case that VR avatar support is an essential part of the webXR-VR ecosystem (and “our engine”).

(1.) VR worlds will continue to grow, and continue to become more accessible. (They will also continue to become “more interesting”: club loner - https://www.instagram.com/p/CPM7B1hB-4q/ - I went, it was insane! ← in case it’s unclear, that was in VRChat).

(2.) WebXR-VR worlds will continue to grow, and continue to become more accessible (https://edmidentity.com/2021/04/25/secret-sky-2021-livesets/).

(3.) Most webxr metaverse-type experiences are being written in threejs.

(4.) Babylonjs has an advantage over threejs in including high level gaming abstractions, but it’s not something that it can “rest on,” because companies like webaverse and xrengine are rapidly building out their stack, including support for avatars (and also bots) (and also full body tracking, etc.).

(5.) Babylonjs is a “game engine,” but what are people building virtual worlds with-- game engines! VRChat is built with unity. To “keep pace” with the development of the metaverse, at the very minimum, I argue that Babylonjs should have avatar support (in addition to other forms of locomotion and collision, but I know that is coming).

(5.5) ~half point-- what is a game? what is a metaverse? what is fortnite, wherein which you have a multiplayer game, a concert venue (Travis Scott and Fortnite Present: Astronomical (Full Event Video) - YouTube), a construction sandbox engine, etc. aren’t future media a combination of all these things?~

(6.) Here is my most important point: if babylon doesn’t stay at the cutting edge of supporting metaverse-like VR features, it may be “cut off at the pass” lose the enormous future marketplace of the webxr-VR metaverse (and/or maybe even a big chunk of future gaming, if future gaming really is gonna go the direction of 5.5 ~ that’s highly speculative, tho).

Caveats:

-I’ve only worked in the engine a year; so, there’s a lot that I don’t know-- though I have built a complex multiplayer VR world in it.

-There are several community projects (again, vrspace, cryptovoxels, maybe frame?) that have avatar support; I don’t understand the relationship between “features that develop in the wild” vs. features that are developed in house, or brought in house; this is very unclear to me, particularly bc my history with the engine is only a year. e.g., I don’t understand when/if these kinds of engine/platform distinctions are competitive and/or collaborative.

2

adding @RaananW

1 Like
3

Yeah, i totally agree! It is an essential part of a lot of VR experiences.

It should be abstracted IMO. there shouldn’t be any relations to a specific engine. The avatars are essentially meshes that are loaded and placed in the right place. Any engine should be able to render them and place them in the scene correctly, just like the controllers are abstracted and are not engine-dependent.

I see that as the webxr committee’s fault. The examples are written in three.js, so three.js is considered to be the starting point for every WebXR application. It’s as if any new browser event type will be introduced using react. But there is little we can do about it, apart from providing those examples ourselves. We can argue that it’s a bit unfair, but this is what they use. And what is fair?..

1 Like
4

Frame is built with Babylon, yep.

Lots of interesting questions and thoughts here, but in my opinion the “avatar” format at the fundamental level is nothing more or less than the .gltf model format. The other items besides the root model, like the data that define the users identity, etc, are all important and interesting but those questions are being examined by various groups like the Metaverse interoperability working group (you should join their meetings?).

I agree with @RaananW that these aren’t items that Babylon should try to solve, but if the standards for avatar identity are written in such a way to be compatible with multiple rendering engines, then Babylon will be able to work with them.

While aframe does have networked aframe, we started there and ended up here, for various reasons.

Easy boilerplate is good for lots of stuff but as your project matures you may find that your needs evolve well beyond its constraints. This is typical with any helper library, though, and networked aframe is still a great choice for tons of projects.

I have chatted with @PirateJC a bit about abstracting some of the fundamental frame multiuser capabilities into an api of sorts, but all early conversations right now.

2 Likes
5

OK, I get you; yes, it should be abstracted up a level; however, in case that doesn’t happen, maybe babylonjs could support it? For now, I will be using the VRSpace avatar stuff. But, as a metaverse builder, not even having a link to that kind of library in the documentation is kind of… not super encouraging. I guess in these critiques of what we have going on, I’m specifically thinking as a metaverse builder (rather than a gamer, per say) (but if I want to build a multiplayer VR game, all this still applies).

I think I would half agree with you re: this being the fault of the webxr comittee; however, again, I would say that if we, here as a community, build a product that is irresistibly attractive to a certain kind of dev (or group of devs) it doesn’t matter that the starting place is in another library. In assembling all of these babylonjsVR examples with a particular artistic aesthetic, I hope to win some people over (if only a few).

6

Gosh, abstracting some of the Frame multiplayer stuff into a networked aframe style library for babylonjs could be very helpful. Again, I’ve been making a lot of front end-y babylonJSVR examples that are easily remixable on glitch, but I’m getting more and more interested in making a set of tiered back end examples (for beginners). I need them for my game anyway; so, I’m going to try to give them people to easily remix. (1.) just player positions, (2.) the actual server code owning objects, (3.) offline persistence is the plan.

2 Likes
7

That’s a hard question. We will support any industry and/or web standard. As babylon is always backwards compatible, if we introduce something to core, it will forever stay there. If the “standard” we define (avatar standard in that case) will not be accepted by anyone else other than us, it will be unusable. We don’t usually do these kind of things, especially not as part of the core library.
We are always happy to see extensions built on top of babylon.js, and this is a great example of such an extension. An avatar system with simple integration? yes please! just submit the PR in the extensions library :slight_smile:

EDIT - I removed a few sentences here, as what I wrote is no longer 100% the case. the WebXR examples are written in native gl and native WebXR.

2 Likes
8

I do not think something like this is really appropriate in extensions. I left there partly because I had 2 different ‘projects’, and that is difficult to manage with one repo.

The main reason to go with a dedicated repo here is, if an avatar frame work were to be written or forked from an existing system, you would surely get better management & visibility.

One way to be successful is to define an abstract client interface standard right at the start, so anyone could add a directory inside a clients directory with their framework’s implementation.

9

I agree - a dedicated repo would be the better option.

1 Like
10

I examined a lot of rigged gltf charaters for my avatars. Doc explaining the structure is here:
https://redmine.vrspace.org/attachments/download/17/vrspace-characters.doc

And the code here: vrspace/avatar.js at master · jalmasi/vrspace · GitHub

Heuristics really.

And BTW live demo https://www.vrspace.org/

3 Likes
11

I would recommend looking into the VRM format. It is based on GLTF and GLB formats.

For three.js but should be able to be adapted for Bablyon.

3 Likes
12

@Mr_SquarePeg great initiative, what’s the state of adoption here? I.e. is there any free content around we could play with?

I see we already have it supported: Babylon.js and VRM | Babylon.js Documentation

2 Likes
13

There is allot of free content. I would recommend looking up VRoid studio or cats blender plugin if you do not want to use VRoid.

1 Like
14

I don’t want to use either, I just want download a file in this format, so I can try to load and animate a character.

1 Like