You can try doing
BABYLON.SceneLoader.OnPluginActivatedObservable.addOnce(function (loader) {
loader.compileMaterials = true;
});
There are some other suggestions in this thread How to preload mesh/materials so that there isn't stutter when a model enters the camera view? - #19 by kvbh