How to calculate cube rotation while maintaining the same view perspective

I’m trying to create a function that would allow me to change the rotation of a cube while keeping the cube’s same front perspective/viewpoint/design. This wording may be confusing so hopefully the images help.

The first image (purple side in front) shows the initial starting rotation of the image. If the player were to swipe in the “right” direction; I would like to interpolate to the second image (black side in front). And if the player were to swipe in the “down direction”, I like to interpolate to the third image (yellow side on front). How would I approach calculating the destination rotation while maintaining the same end perspective? Any help would be greatly appreciated.

Thx