New build system in our main repository

cc @RaananW to have a look

There is a .gitignore in the ./, which has dist. My guess is that it is intentional to not have the dist output under source control.

A couple of potential options to get available locally:

  1. write a script using curl or wget to download from CDN?
  2. update git repository and then build with npm run build:umd then you will see your files in ie: ./public/umd/babylonjs/dist. You can follow other paths for remaining needed files.
    • babylon.js
    • babylon.js.map
    • babylon.max.js
    • babylon.max.js.map
    • babylon.module.d.ts

I think the entire ./legacy is uneeded currently and can be disregarded, but should let Raanan confirm that and likely he has a better way! :smile:

1 Like

Yep, legacy should be disregarded and is only there too slowly port everything, as there is still a todo list.

If you want the files locally you can either build it yourself, download from the cdn or use npm to download the latest package. These are the ways to get the files locally. The cleanest way was suggested by @brianzinn - run “npm run build:umd”. If you want it all in one directory you can run “npm run prepare-snapshot” to create the cdn structure locally in the .snapshot directory.

I can gladly try to automate the release creation. In fact i have already added the task yesterday, but it hasn’t been tested fully. What we won’t do is continue updating our git repository with daily changed in production code (both minified or not). It bloats the repository for no reason and doesn’t scale.

I don’t know how to use npm unfortunately.
I understand that it is easier for developers to have made all the changes in the repository, but it may complicate things for users if they now have to build the files for local use.
I prefer having the files locally because it’s faster to load than using url.

I don’t understand what the said file is for if it is no longer updated. Is this only for the official release?

As for the version error that I explain in bold in my post. is this a bug? The cdn version is 5.0.3, but when you download the 5.0.3 zip, the local files are 5.0.0-rc.4.

I have to say that the change in the repository has some upsides, but there’s also a step back from before for users who like to update patches regularly.

Am I right in thinking that if I now want to download any of the Babylon.js files, eg max, min, loaders, gui etc. in JavaScript form they will not be available in the Babylonjs repository and then one way would be a browser search for jsdelivr babylon.js select the appropriate link and download the required file from the jsdelivr page?

I’m doing this as a workaround. and it works. (I don’t know if .d.ts also go in the cdn?)
I share it in case someone wants to do the same.
'when this code is executed, it creates or updates the local javascript file from the remote file
PS: It is necessary if the server is local (easyphp, wamp) to activate extention : extension=php_openssl.dll

<?php	
$arrContextOptions = array("ssl" => array("verify_peer"=>false, "verify_peer_name"=>false));
$babylonFile = file_get_contents("https://preview.babylonjs.com/babylon.js", false, stream_context_create($arrContextOptions));	
$localProjet = "../my_project/libs/babylon/";
file_put_contents($localProjet."babylon.js", "".$babylonFile."");	
?>

Do the same with the other files gui, materials, inspector…

Yes, the files will not be available in the repository anymore. There are different reasons behind it (and the main one being that we were updating a log of minified files on a daily base, which led to the repository being bloated with unneeded changes).
There are different ways of downloading all of the files, the best and simplest one is downloading them from our CDN. there is no need to use jsdeliver, we have an updated CDN for you :slight_smile:
Setting up the repository for build shouldn’t take long (apart from the time it takes for the repo to clone). If you have node 12+ and npm 7+ installed, it is (theoretically) as easy as:

  1. clone repository
  2. run npm install
  3. run npm build:umd
  4. run npm prepare-snapshot

All files will be in the .snapshot directory

Once you have the repository already cloned you only need to run the last 2 steps next time to get the files you need.

Also, running npm install babylonjs in a new empty directory will work perfectly as well. It will pack the files in the node_modules directory, but otherwise it is the built version of the package that you are looking for. local and ready to be used. Run npm update babylonjs in the same directory, and you will get the latest stable version (if any is available).

3 Likes

Thank you. That clears up a lot for me, except for one point.

A
I understand that I can “download” the required files in html as in

<script>src="https://cdn.babylonjs.com/babylon.js"></script>

B
What I do not know if by “our CDN” you also mean a site that lists all the possible current Babylon.js javascript files which can be individually downloaded?

My assumption is that only A is true.

Would you please confirm if its A, B or A and B.

A is right :slight_smile:

@JohnK , @Dad72 ,

I have added auto-release creation to the publish pipeline. Each github release will include a zip file with a snapshot of the CDN structure:

Releases · BabylonJS/Babylon.js (github.com)

Starting this version it will include this zip file. Hope this solves the issue :slight_smile:

2 Likes

Thanks RaananW for adding this. This gets the files for local use.
But why everything goes in a zip file and not directly put this zip structure in the repository?
Anyway, thanks for this addition.

We try to avoid big files in the repo because else it takes ages to clone/fork and build (for our CI)

I wrote a solution to update quickly with PHP. here .
I take the content in : https://preview.babylonjs.com/babylon.js and I rewrite everything to a file locally. (dynamic copy paste). This solution works well.

2 Likes

I do not know where it is, but there is a CDN site where almost every release is. I myself, was thinking about putting a reference to the URL of the specific version as a <!-- Commented specific version Url --> when published, because that is when I know the current version the scene was made with.

If for any reason the current CDN no longer works, then it is a 2 minute change. Don’t need to remember what version to try. Some of my old scenes stop running when the location of extractMinMax() was moved.

I am currently re-thinking a dedicated Github pages repo. It is just a floating garbage bag of scenes. I think you can add pages to any repo now, so each scene can be its own repo. Us mere mortals, cannot pull off a monorepo like @RaananW . Has anyone tried this yet?


Edit:


Edit 2:
Forgot to mention that candidates for using a single scene Github pages are probably not going to have any typescript. I keep that to libraries. Transpiling requires a 64 mb node_modules sub-directory.

Obviously, I would git ignore that, but seems wrong for all but the most demanding scenes.

Hey, I wanted to upgrade babylonjs to the latest version in an old project, not es6, … and I didn’t find any link or info about from where I can download babylonjs :frowning: , other than this

which is not quite completed, I mean I have loaders, inspector, gui, … from where I can take those? …

I solved my problem downloading the assets from github releases, … all good, … but it wasn’t my first option :frowning:
We have thousand pages of documentation and we don’t have a page clear page where to write click this to get this… I don’t think it is ok to try workarouds like @Dad72 to get the babylonjs files. Cheers!

Tagging @PirateJC about docs - might be interesting to add a “releases” section on the docs with a list of versions, like what we’d find in Download Python | Python.org

1 Like

Have done a basic list of all packages for CDN in the new draft docs as a suggestion and thoughts of adding CDN versions when available.
Could add the basic page to the current docs if wanted.

4 Likes

Would be much appreciated!

1 Like

Yes it’s not the ideal solution, but it works for my use case. This saves me having to download the zip file to take the scripts one by one and copy them into my projects. My solution makes things faster, without downloading and miscellaneous manipulation. I run the php file and all my projects are updated in 5 seconds locally. :smiley:

1 Like

@RaananW had already submitted a PR for the page I was going to change.

1 Like