Excluding _BuiltData from source control?

Since lightmass builds are essentially derived data from the map, I would like to exclude the _BuiltData.uasset files from source control to save space and upload time while iterating on big maps.

But any time I build lighting it updates the map and not just the _BuiltData, even if I tell it to build lighting only.

If I then save only the built data and not the map, the build won’t apply the next time I restart. If I do save the map as well, then when the guy I’m working with updates the map it will invalidate his lighting build, and he will build, save, and invalidate my lighting build ad infinitum.

Is there a way to get what we want, built data excluded from source control? For some maps the built data can be nearly a gig and there is no reason to want to check it in every time when iterating on the map.

I know for gameplay only stuff I can make a sublevel and avoid the massive checkin sizes for audio, collision layout, etc., but is there a way for scenery changes? I had thought this was the main point of _BuiltData aside from the runtime loading of different lighting scenarios stuff.

1 Like

The built data is the lighting. And you have to save both the map and that when you rebuild, otherwise the map doesn’t know how to access the built data.

You are saying you don’t want to check in the built data. That means anybody who syncs and loads your map will have no lighting information and thus the lighting will be automatically invalidated if they save the map. I don’t really understand what you are trying to achieve by doing that. You understand that nobody else but you would ever be able to see your rebuilt lighting, right? Are you saying that’s what you want, you want others to be able to make changes anyways without invalidating lighting? Pretty sure that is not possible by design.

The main reason for it was to allow for streaming in separate lighting copies for the level, ie different baked times of day (edit, I see you even mentioned that in your last sentence).

What I was looking for is, while a map is undergoing lots of development, let everyone build their own lighting and keep it out of source control to save disk space on the server and bandwidth for the devs and artists.

I’m collaborating over the internet so in some cases it can be faster to rebuild lighting than to upload and download the _BuiltData. And Comcast has an annoying data cap in my area.

Is it just the internal layout pointing to atlased texture areas and stuff that makes the map need to know about the specific _BuiltData file rather than being able to generically point to one built from the same map? If so, do those atlassed areas vary from build to build, even if everything else stays constant?

For server storage savings we only store latest BuildData file in source control. But this won’t help your network limits.

How do you have that set up? Does Perforce have an option for only storing the latest version of a file? We’re using Subversion so I don’t think it is easily possible.

On perforce there is setting called Change filetype. Then there is checkbox that says: “Server limits the number of revisions stored to 1”
Not sure how to do that in Subversion.

That’s awesome functionality!
A side note for newbies like me: you need to set that option for each BuiltData file. But you can select several at once and set it in P4V.

FYI, you can setup Perforce to automatically do this for you by editing the “typemap” file.

Run “p4 typemap”
and add:

binary+S1 //…/*_BuildData.uasset

This will only store one version of BuildData files on your server for all newly added *BuildData.uasset file (existing ones will need to be manually setup).

2 Likes

Thanks! This is amazing!

Hi, just wanted to weigh in on the subject of BuiltData files and source control system. Perforce typemap solves the server-side problem, but there’s still the problem of syncing BuiltData for all maps each time you sync your project. It’s even more relevant today when many people work from home and are limited by their network connection. This motivated me to develop a plugin, that helps to solve this problem: On-Demand BuiltData. It gives you tools to omit BuiltData files during the “morning” sync, and then automatically sync individual BuiltData files at the moment they are needed.

1 Like