So!
Actually everything works as expected but it is a bit complex to understand 
Some prerequesites:
- GPU particles are not created. The system has a define pool and only iterate through them. Dead particles are recycled and new ones are built from the same structure. For instance the system creates a buffer of 100 particles at start and the GPU goes through all of them every frame. It updates every particles and then render the same buffer. At the beginning until we reach max capacity we ask the GPU to only process the first few ones while every particles in the buffer are emitted
- This means that when a particle dies, it is simply repositioned at the source with a fresh set of values
- When targetDuration is hit, the system stop recycling, meaning that the dead particles remain and become zombies
So on the examples provided by @roland:
- The first one SEEMS to work but this is only because the deadColor is set to black. If I set it to anything else you can see the zombies particles: Babylon.js Playground (babylonjs.com)
- So on the contrary for the second one, if I set the colorDead to black, zombies are invisible (but still computed): Explosion | Babylon.js Playground (babylonjs.com)
This is why with GPU particles you need to add this code if you want to really terminate the system when targetDuration is hit:
ps.onStoppedObservable.addOnce(() => {
ps.dispose();
})
Mystery solved 