The easiest way is to remove all textures from a file beforehand with some tool like Blender (or with some relevant CLIs, or with just small Babylon script like you mentioned and then save the clean file).
I think it is not possible to not to load textures from GLB, but it should be possible for GLTF with some additions to the GLTF Loader. Here is a couple of threads which may put some light on it
1 Like