Looking at the documentation for FloatingPawnMovement it mentions:
“During swept (non-teleporting) movement only collision of UpdatedComponent is considered, attached components will teleport to the end location ignoring collision.”
With this in mind, making the skeletal mesh component the root component of TestPawn and setting it’s collision to BlockAll instead of OverlapAll causes the mesh to correctly collide with the box mesh.