Make attached particle system stay upright

I want to attach a particle system to a physical mesh, but I don’t want it to rotate with that mesh but stay upright.
How would you go about doing it?

I see nothing fitting in the emitter options or the particle system’s options.