All you really need to do is be able to place the images you are using as a mouse cursor at specific screen coordinates. Two basic ways to do this are drawing directly to the canvas (via the HUD) or by using Slate/UMG widgets.
For the first option, look at AHUD, specifically DrawHud()
There’re a lot of ways to do this using Slate or UMG. You can create an individual SImage widget for each cursor and manually place them. Or you could create a container widget that updates the positions of any number of cursor images based on the screen coordinates you’d like to use.
Your container widget could be a UUserWidget that uses an SOverlay widget sized to fill the full screen, and then adjust margins/offsets of the children.
Look at UGameViewportClient, AddViewportWidgetContent and AddViewportWidgetForPlayer will be helpful.
Once you can place an image anywhere on the screen, you just need to update the image positions based on player input. You can get the position of the mouse on the screen with APlayerController::GetMousePosition, or use gamepad/keyboard/whatever input you choose.
You could also have one player’s cursor be the “real” mouse cursor and only create a “fake” cursor for additional players. Look at APlayerController::SetMouseCursorWidget and UGameViewportClient::AddCursorWidget SetHardwareCursor, SetVirtualCursor