This leads me to conclude that there is a timing issue, so I tried loading the model in the scene.onReadyObservable callback, and that also fixes the issue:
To make sure I’m understanding the problem correctly, I tried to summarize the issue and solution in the diagram below.
This is just my interpretation based on your explanation—please let me know if I misunderstood anything.
I believe @Evgeni_Popov could shed more light on this issue and possibly propose a permanent solution. I just did some quick research into the cause of the problem.