Real Time Collaborative Development

Hey what’s up unreal users! I was wondering if anyone knows if there would be the possibility of having tools to work with a team in real time similar to something that the Hero Engine or Snowdrop Engine does? It would really kill a lot of development time if the devs within the team can make changes to the game together in real time, instead of having to export/import changes constantly.

I have the same question. It would be a time saver.

I guess you guys asking for something like this:
=FOWsc1AqtzQ
No, it is not currently possible, It would cool feature though, for making big worlds (not just MMO), but I bet it’s something that will more likely come in form of 3rd party extension.

Will Epic find this feature worth to be implemented ? Could we expect this feature in a future engine version ?

I’d definitely love to hear some ideas around what we could implement to make this a great experience. There’s some fundamental functionality we’re working on at the moment that makes more advanced/realtime collaboration feasible (hot reload for example), plus there’s some obvious workflows that don’t currently scale with teams that we’re definitely going to target (a.k.a. matinee). Beyond that we don’t have anything concrete, but I would say there’s interest from quite a few developers internally on this topic so please feel free to share your thoughts.

I just joined this community (and started subscription) to focus exactly on the thing around this subject. I look the collaboration/sharing also from end-user perspective and wider scope for sharing/co-creating whole worlds.

I try to keep this post as short as possible, but also give some initial background of how concrete we are - and why I’m really excited to hear that UE team is also internally interested in this topic.

  • My background is 15 years of entrepreneur-like developer focused high-end development (initially on C++, then C# for more than a decade)
  • With my game-industry focused friend we concepted C#/.NET integration to Half-Life engine back in around 2007ish - to have real database and Enterprise grade sw development joined to game engine level
  • Since 2010+ I’ve been focused to bring collaborative architectural sw development to mainstream-doable
  • Since 2012 I’ve been building open source platform to reach the collaboration of sw dev combined to information/web content sharing
  • Exactly two years ago we started university-collaboration project to build “The Ball” platform to focus on all of the above

Now where are these initiatives now?

In a nutshell what is The Ball in VERY CONCRETE fashion:

  • Microsoft Azure cloud runnable platform - that scales on per-end-user basis linearly and most importantly cost-WISE
  • Complete open source, free to use and “maintain” By anyone; right now I maintain open dev concept versions of it
  • It’s been ran in production use for elementary school pupils digital content publishing

My personal goal is to bring game development (Minecraft modding and UE4+) to elementary schools => based on simple controlled file sharing and toolchain execution built on top of that.

Why I think it’s much simpler than usually thought of?

What you see in the demos is concept level web-app toolchain. When combined with game-engine integration, it can bring character profile editors, quests, tasks, collaboration features on the web side, but still affect on the aspects that Ray just described.

If UE team will indeed have “fundamental functionality to make advanced/realtime collaboration feasible” on editor toolchain, all that’s needed is effective file/delta-content sharing; like automated version control, but more flexible than GitHub and alike - and this is what the Ball is initially designed to do.

Of course keeping feet to the ground, using something like Dropbox works out-of-the-box for immediate content syncing/store (with hot-reload, delta supporting tools) and achieves much of the same functionality for normal developer team workload.

My vision/aim for larger scenario is the ability to involve different technologies to be collaboratively developed. Such as web-UI for character management, statistics reporting (Battlefield - alike), and especially for external community-driven game-extension development.

I once wanted to do something like this with 3ds Max. I approached it as a hybrid between something like Git and generic sandbox serialization. It definitely needs infrastructure in place before it’s possible to accomplish, but I think it’s doable considering how Epic have described the construction of UE4’s editor. The biggest obstacle that kept my Multiplayer-Max plugin from working is how local clients handle the creation, addressing, and manipulation of data.

I considered it least-effort by treating the collaborative realm as a server with collaborators as clients. Resources (such as models/textures) are stored server-side under version control. Clients connect to the server and ensure they’re using up to date resources before any maps can be loaded. If a new resource is introduced to the server, it must be available on all connected clients before it can be introduced to the simulation(easy on LANs, longer on American broadband).
The maps themselves would function as a type of pushed repository where a save is similar to a commit. Clients work in the map’s scene independently fleshing out whatever aspect of the level they choose. At this point, you can opt into “full” serialization for what they’re working on to give that real time feedback or leave it as simple as a camera geometry object representing what they’re looking at and a bounding box for the workspace they’re approximately working in.
With this, data wouldn’t persist on the server unless it was committed. In the long term, this allows teams/leads to sift through aspects of the collaboration and treat major decisions as branches/merges instead of picking one file at a time in a huge folder.

I couldn’t get it working the way I wanted, but it’d be cool if Epic could get something like that running with UE4.

Note: I was actually thinking about Kismet / Blueprint when I first wrote this post, but it would still help with matinee.

ShareJS is an Operational Transform library for NodeJS & browsers. It lets you easily do live concurrent editing in your app.
Source: sharejs.org <- Have a look at the demo on his page.

Written by an ex-Googler who worked on the original Google Wave.

Currently working for Lever Inc out in San Fran where he integrated ShareJS in DerbyJS.

I’ve been following his work for years. If you think that OT/ShareJS would help with matinee or other real-time collaboration problems (I think it would), I would definitely drop him a line.

Thanks.

Thanks for the feedback - there definitely seems to be opportunities here. I’m most interested in really understanding the workflows that would best benefit from collaboration and doing some experiments to find what really works well. For example collaborative geometry editing or world editing seems to have a lot of potential given the successes of Minecraft and similar games. Collaborative blueprint editing reminds me of pair programming which seems like an interesting experiment but maybe not terribly effective (although I’m sure there’s some folks who would swear otherwise). Along those lines I’ve always been fascinated by the prospects of an entire team always operating on a living, persistent experience for example, much like how many several MMO teams operate.

At any rate - we should continue the discussion! As I said before there’s interest on our side here, so there may be a great opportunity to work with the community to make some of this happen. At the very least it’s fun to brainstorm ideas on. :slight_smile:

I personally think it’s easiest to put up concept-demo and showcase/demonstrate scenarios and possible workflows. I just purchased UE4 capable workstation, which is arriving early next week. I’ll learn my way around the editor etc and then get back to this with some real demo…

That’s again of course from my vision/perspective - and while I’m pushing that way, I’m closely following this discussion as well to see if that vision meets the real UE4 community or do I have to adapt to better meet the needs as well…

Just a suggestion here -

Why not make a listen server that we can host on our own server box. Then have a toggle box with some “networking magic” that submits changes through the network between machines. :slight_smile:

Just a thought.

HeadClot

This could work… so the changes are made to the game on hosting server…

Here is what I think could be networked:

  1. Scene editing (terrain,lights,materials etc)
  2. Blueprint development
  3. Real-time playtesting-while some build things, other test them in the same time

This is really good starting point - to concretize what :slight_smile:

I got my rig and I also came to technical conclusion how it can be simplified and just done: the collaboration MUST be doable within version control level as well.

Thus => as end result of active ongoing collaboration, the results should be committable at all times without any merge conflicts, that are not automatically solvable. Then the game server looks at those automatic-immediate commits and re-flushes based on that “single truth”.

End result: the collaboration is “first class auditable citizen” producing exactly same developer artifacts than any coherent/controlled team.

What would be nice would be support for Autodesks FBX “send to” feature.

Something that I think is a bit more important is content control and security. The thing about content is it tends to grow like a weed and although there are more than a few different hosting solutions I’ve yet to see anything that would limit access as to whats needed as to apposed to random downloading.

To put it into content.

Over a few years your group collects a few terabytes of content through licensing, subscription and development and bring on a new hire. Under the current ways and means they would be able to freely download at will and then quit.

As much as it would be nice to add sharing features directly into the core of UE4 I also feel there is a potential for content pirating and for remote development the other side of the coin must be looked into as well taking into consideration the evil that people do.

I’ll up vote HeadClot suggestion that a listen server should be used for remote development and that direct client to client used only by small groups.

Agreed - Security must be looked into with things like this.

My 2 cents on security for larger teams and smaller teams.

Give the project lead / project leads the ability to define via GUI on the listen server backend to set security access for different team members. I am not sure what the security levels for this should be however there should be the ability to define who people are via nickname or name.

User Defined Groups should be a thing to keep things tidy so that we do not have to dig to find some one.

Example -

Project Leads -
Lead A
Lead B
Lead C

Artists -
Artist A
Artist B
Artist C

Programmers -
Artist A
Artist B
Artist C

Further more allow the leads to push tasks to certain people via the listen server back end to the client.

Anyway that is my 2 cents on the matter :slight_smile:

  • HeadClot

Im new here, though I must admit, though we are moving from HeroEngine to Unreal, I have gotten spoiled of not having to rely on nightly builds. I forgot the hassles that can be, especially for those that have never worked with them. I got spoiled for sure. On that note… What I am suggesting is more support for this topic than anything. I hope something I say can be helpful.

For security and team management: User defined groups are very important i think. Having a core predefined ACL set up be it stand alone or via predefined groups would be perfect. However, because different teams and companies are not all the same, i think allowing it to be adjustable would be important. That is where one difficulty would be, that i can see. It must be dynamic enough to serve most teams and companies.

I am in 100% agreement with this. Though I think with BP’s and any c++ we should have it a little less “real time” if you will. That can cause a lot of confusion if you have a few people working on the same thing. So I am thinking for version control, it may be best to allow the programmers or visual scripters to push the updates as they are finished or want/need testing.

So blueprints are important yes, but I think it might be best to have that as a one button push, and then update to everyone connected.

The most practical way I see for the actual networking is to allow a core install on a master, and run from that. I am thinking almost cloud styled software. This would require a single instal for the teams ‘server’, and yet not eliminate the subscription model you have. It could be an internal server, a vps, or remote hosted. That should be left up to the individual teams. Yet, if we have a localized listener, that could make it easier in the short run.

I always fantasized about this back when mapping for UT. I’d be constantly chatting with fellow mappers while we mapped and I’m sure a couple of times we discussed how cool would it be if we could make a map at the same time, and we could see what the other mapper was doing, see their meshes being moved around and how they work, possibly even race them to complete an area etc… The thinking being that whatever things they were touching would be “locked” while they worked on them, lighting builds etc would be temporary client-side until you both committed to a proper build, and no doubt a million other considerations.

Would be totally awesome but seems like an engine would kinda have to be designed with it in mind from the beginning to go to that extent.

Perhaps a slightly easier alternative might be if you could at least fly around and “spectate” while they edited (not propagating everything and rebuilds still having to be done periodically spectator-side). An Epic LD could make a map “live” and people could spectate and watch how it goes on (of course, they could always just stream their screen, but it’d be cooler to have the ability to get any view you want and possibly click on things to spawn a tag or note for the person working, view it on an oculus?). This way, you could still almost achieve the fantasy above, if you had a viewport open spectating your fellow mapper’s session, and they had one open spectating you. Starting to sound a little creepy.

I’d like this feature as well! It would definitely speed up the development process, and be another awesome reason to use Unreal Engine 4. I think this will draw more developers to Unreal Engine 4 as I don’t believe there is another popular engine out there that has this feature. :smiley:

The thing is you can do that now and could for a very long time if engines like UDK or UE4 are used as the world builder and not as the principle design application. It’s not a software problem but a project management problem and a division of work loads is whats needed.

Personally I would never build a map just using UE4 and prefer to do my over all construction using 3ds Max as it’s the application I know best and a complete putz using brushes and for years has already had collaborative tools that makes the job much easier with out the need to lock out any part of the project whats so ever.

Using Xrefer for example, which is read only, you can see in real time the progress of some element or component in development, including real time updates and how their work will interact or connect with what you are working on.

Overall though what I believe makes the big difference is UE4 is taking a much more modular approach with the idea that more than a single individual can only access a given area or requirement at a time and that a map or environment by it’s self could very well be just as easy to manage as any other project as a stand along.

Not creepy at all.