Project Settings → Collision
Create a new Object Channel named Ghost. Default Ignore.
For each actor you want to block the ghost you set custom collision and tick block for ghost.