My Niagara Emitter exceeds the max CPU particle count

I implemented a fake star background (on top of an actual HDRI star background) using NiagaraEmitter. That allows me to have layers of closer (fake) stars moving in front of the distant, fixed ones.

However, I easily reach over 1,000,000 particles like this.

These particles are never all visible at the same time, I basically filled in the background of the whole game word.

But how do I go about this now? Is there an elegant way to imit the particle spawning to the actually visible ones? Should I make the Emitter depening on the viewport maybe (instead of fixing it to the world)?