How to display camera from client to server and vise versa?

Development Programming & Scripting Multiplayer & Networking
, ,
1

Let me preface this by saying I am pretty new to multiplayer in unreal, but…

I am making a mobile game and I want to host a P2P server. I have this working correctly already, my issue arises when I try to replicate one camera to the other. It is a puzzle game and I want to make a multiplayer mode where you and a friend try to see who can complete the puzzle the fastest. I want to show the listen-server player’s camera to the client player’s screen and vise versa.

Right now, I am using a scene capture component 2D (replicated) on the player camera, that is “setting” a render target (RT_ SecondCamera0), that is then setting a material instance (MI_SecondCamera0). There is a UI - image element that is getting set to that material instance. This works completely fine in a single player environment, taking the screen and displaying it to the UI image element. The issue arises when I try to replicate it.

My current iteration of attempts is as follows:
Whenever the host player clicks play game, there is a multi-cast event in the ‘game state’ that fires off, making both players “SetTextures” (also a multicast function) which:

If server:
Sets the capture component 2D’s render texture to RT_SecondCamera0
and
sets the CameraMaterial (a material instance variable inside the player) to MI_SecondCamera0

If client:
Sets the capture component 2D’s render texture to RT_SecondCamera1
and
sets the CameraMaterial to MI_SecondCamera1

I then, after these have been completed, back in the game state, if the player is the listen-server/host, fire off another multicast event in the game mode that gets both players, and sets their UI images to the other one’s material instance. (server to MI_SecondCamera1, client to MI_SecondCamer0)
I currently have 2 different UI image elements one for server, one for client, but I plan to have only one if I can get this figured out.

With this current setup, the server just shows it’s own material on one UI element and a blank black screen on the other. The client does not show either UI element at all. I have noticed that camera material returned from the client is null.

I suspect that issue is either replicating material, and/or I probably should not be trying to set the UI through the game mode since it only exists on the server, and it needs to be set on both. However, I went through the game mode so that I can get references to both players, which have references to their current camera materials, and their UI elements…

Thanks for any help, I can add pictures of the node setups if needed, however, I thought it might just cause more confusion. Let me know if you need any more information. Fixes to my current attempt or a better way to do it much appreciated.

Unreal 5.4

2

Hello,
I don’t think it’s a beneficial for you to try to replicate a texture. Imagine replicating for example 4mln pixels (2k texture) every networking tick… it would be a lot of data to send.
Of course it’s doable with pixel streaming, but it’s usually not very smooth and responsive in higher resolutions.
The usual solution in your case, would be to have both puzzles in one world - 1 for your player and another to mimic the second player. Then you only replicate puzzle objects when they change, and player can switch between cameras smoothly. It’s much more optimal than trying to replicate the whole camera view, but for sure harder to do.

1 Like
3

Hmm, that is an interesting idea. I could definitely make that work. I would just have to rework a bunch of code. Thank you for the idea. I’ll let you know how it goes.

4

My original thought process was that I could just send the texture and not have to deal with all of the functions and events getting replicated, I could just show what the other player sees. But I didn’t think about all the data that needs to get sent over for one texture, which would probably be more memory than the events/functions anyways.

5

The process of recreating events on client side may also give you some more flexibility, as you have all objects present, not only a texture.
But you can check this thread: Replicate Textures from Host to Client they mention a plugin that can replicate a texture. Maybe this will work in your case.. but consider this only if your texture can have very small size an doesn’t need to be replicated very often.

1 Like
6

UE doesn’t replicate materials, textures, skeletal meshes and so forth. This is noted in the docs. Last few paragraphs of the “Replication” section.

Replicate data which will provide the information needed to mimick the desired result.

For your overall issue I’d do spectating. Another option would be split screen.

1 Like
7

I got this working like 90%, however, I am having an issue when I click an actor “button” it calls “SetActorTransform” on another actor that it has a reference to, moving and rotating it. This works from Server to Client but no the other way around.

All actors involved are marked to replicate, and replicate movement.

I have tried calling the event on server (this seems to do nothing at all, just ends the execution of code) and multicast. I have tried using a rep-notify to then set actor transform. I do not know what else to try. Would you happen to know what a possible issue could be?

I think the issue might be that the actor does not have access to the server or something of the sorts. Is that something that could happen?