Seems like this isn’t solved yet even in UE 5.2.
The fix for using 1, 2 indexes broke the singleplayer VR pawn UI interaction in my case.
So, I added a workaround to disable ticking on the widget interaction components for remote (not locally controlled) pawns at begin play.
That seems to work.