First you should have a look at the profile and see where the time is really spent (download vs runtime).
Then you could rely on MSFT_LOD extension @bghgary could provide guidance to have a LOD system where the assets would have several resolution helping in reducing your overall time to first frame.