Controller doesn't work in PIE for me

I tried to use both a PS5 and an XBox One controller in the editor in PIE and it doesn’t seem to work.

Strangely it works in standalone and in a package.
Is anyone aware of a setting that could fix this?