In my case, collision/overlaps were not working on actors added at runtime.
The objects placed in editor beforehand (before PIE) were working fine, but when I tried to place actors dynamically at runtime in the level that I loaded from the main menu level, they did not register overlaps or GetHitResultUnderCursor() or any other collision related stuff. Adding the levels to the Levels list fixed this problem.