Ooh, I just saw @msDestiny14’s post that all you have to do is set the excluded mesh’s visiblity to 0.99 to make it so that it doesn’t affect the highlight layer at all! 
https://playground.babylonjs.com/#4AARMT#5
I think the addExcludedMesh documentation is a little unclear thou, since it says that the excluded mesh wont impact the highlight layer but really the mesh’s visibility must be less than 1 for that to be true. IDK how to word that in the documentation thou or how to describe the effect when visibility is 1. 