Scene to USDZ converter

Hello guys,

it seems that Mr. Doob already did this and model-viewer will follow. I think that would be a very nice feature for Babylon as well.

Best

1 Like

I wonder what’s the point is…

GLTF is trying to becoming a standard for 3D file format (at least in the field of real time rendering) to avoid having to deal with multiple existing formats (fbx, obj, …), throwing a new file format (even if it exists for some time now) in the mix does not sound like a good idea to me…

Also, it comes with its own problems: Should You Use the USD and USDZ 3D File Formats?

I believe nobody wants to use USDZ but it is necessary to be able to use AR functionality on iPhones. The converter should solve exactly this problem that you are not forced to create a USDZ but it converts the scene on the fly.

Ok, thanks for the explanation! Apple at its best…

2 Likes

Yeah…Do we know how the conversion is done? Maybe we could port it

It seems that it’s already integrated to three.js: three.js/USDZExporter.js at dev · mrdoob/three.js · GitHub

This link could be helpful: GitHub - google/usd_from_gltf

Would love to have this feature in babylon :pray:
We have a project where it will be very much needed and it would be a great addition for those of us working in AR where usdz is just unavoidable :upside_down_face:

3 Likes

If anyone is motivated, I would gladly merge!

1 Like

The google project usd_from_gltf is very cumbersome to work with.

I built a docker image to do the work

But it required a multi step built where I first need to download and compile the USD source, which takes around 30minutes to build on my mac pro, and then you need to use that to then use the google python script to pipe in your gltf file into it and hopefully it spits out a usdz file.

The problem with the whole thing is that usdz (on iphone) does not support draco, which means the files are huge.
there are also multiple gotchas with it as you can read here

Even though three.js has support there will be edge cases where the conversion wont work flawlessly.

With that in mind, I think it’s a great idea to port the lib to babylon and see how well it works with different models.

The lib more or less loops through and creates the text representation that is the USD format.
there are a lot of string concats and a lot of room for error, but it can evidently be done.

How would we go about doing a port?
We need to figure out the differences between how three.js materials, textures, meshes are layed out.
and then go about porting one by one.

1 Like

We wrote our own USDZ converter in our app (based on BabylonJS). In theory USDZ supports everything we need but the viewer in IOS is very limited and buggy. We had to use a lot of try and error to make it work.
Here some of the issues:

  • transformations are buggy, in the end we have to bake every transform to the meshes
  • only one set of UVs is supported, in some cases we can’t export the normal map because of this
  • if i rember correctly there are also issues with transparency

So we have a USDZ export in our app but its far from perfect. There are models that we can’t export correctly. The USDZ files are also very big. Thera are also size limaitations in IOS.
The fact that the zip has to be uncompressed and we use the ASCII version of USDZ makes it even worse. … i couldn’t find a documentation for the binary version of USDZ.
We are planning to move the USDZ conversion to the server side. This way we can use the official USDZ tools and can atleast create a binary USDZ.

@Kesshi

I read about the file size problem already. But this shouldn’t be a problem since the user don’t have to download the file.

You you mind to share your code for the converter? In this community are a lot of smart helpful people. I am convinced that some problems could be solved.

The file size is a problem because the loading time is not very good in the IOS viewer and it gets worse with bigger files. There is also a limit. If the file is to big the file will not load at all.
Sorry i’m not able to share our code.