In case someone stumbles on this issue, it’s finally been fixed , apparently if you install the preview version:
Good news, we are tracking to a public release.
To help get to that date, we have a fix that is in the latest in-flight builds of HoloLens OS. If you have access to a test HoloLens device, you can get the latest build here that has the fix:
Insider preview for Microsoft HoloLens | Microsoft Learn
If anyone does go to insider preview to help test and works, please let me know here. If any issues, can hit up too.