I found a solution!!!
Since “Get Actors in Selection Rectangle” is not accurate based on camera position, I found this code you can add after the selection rectangle to make the selection and deselection become accurate to the rect itself instead of the world position.
Just follow this guy’s examples in his pictures and it will fix it.
Basically it gets the actor’s location and converts it to screen location then checks if it is inside the rectangle or not.
This is something I was thinking about doing but I didn’t know how.