What you might be looking for is “simplification”, which means to reduce faces of the mesh while keeping rendering result “similar” as before.
To start with, gltfpack tool from the meshoptimizer repo is a good choice, with a widely used and optimized simplification algorithm. Or you can call meshoptimizer api for lower-level control.
glTF-Transform has a js api, making it more friendly to js/ts devs, but it has not integrated latest features of meshoptimizer.
If you are looking for a solution fully in babylon.js, it has a “simplify” function, but the underlying algorithm might not be as optimized as meshoptimizer.
Or here is a quick demo of using meshoptimizer simplify in babylon.js, but it’s far from complete.