If you do not need the dragged widget to also show while placing the building, there’s no need for drag and drop. Just pass the focus back to the viewport as soon as you want to move the actor around. This can also be achieved by juggling Input Modes.