Need help from the community or a dev that specializes in texture streaming and textures in general.

Development Rendering
1

Hello everyone,
For the past month i am trying to upgrade my project to 4.10. Unfortunately i cant because i get constant Vram warnings and crashes.

After a while and with the contribution of the very helpful answerhub guys i have managed to better understand what is the root for all of my problems.
This is where i need YOU! I could use some answers on how and why unreal treats textures the way it does. Is it normal? Is it a bug? Is it a bug with paper2d only projects?

Lets take things from the start.

The second i upgraded from 4.9 to 4.10 and pressed play i got greeted with a big red warning that my texture pool is exceeding the default value (1000mb) by about 3000mb! Whaaaaaat?
After reading around i learned about the texture pool value, where i can change it etc. etc. It now appears that my level is using about 4gigs of texture Vram! Needless to say that after a few secs i would get a glorious crash.

Although i am not an expert by any means when it comes to textures (and almost everything else… ) i had done my homework before i started creating my game and i knew about the limitations (like keep your textures below 2048x2048) and the do and dont’s of assets.
Because my game is 2d i have no need for mip maps or anything fancy.Hell! They dont even have to be power of two.Why would i add the extra translucency size cost for a sprite when not needed? Another good thing about 2d is that 90% of a level is comprised of the same, repeating sprites. So i was in shock when i realized that my gtx980 was crashing because it ran out of memory.

My experiments lead me to a conclusion that does not make sense to me. It appears that Unreal is loading EVERYTHING that is within my project no matter if i only use 1 texture inside my level! Is that normal? Is that the way an engine should work? No wonder it is crashing. It is loading >5000 textures into memory. Shouldnt unreal only use what is needed and not everything?

Another super weird thing is that when i go to project settings and disable texture streaming then my texture memory consumption goes back to normal. Unfortunately it still crashes in standalone game or if a package my game. At least it does not crash in standalone window anymore.

Is there a Dev out there that can help me understand why is this happening? I cant work anymore and it is hurting me really bad, both financially and mentally.

This is a picture of an empty map with only 1 sprite and 1 flipbook (my character bp).

73719-screenshot 2016-01-10 19.51.53.png
73719-screenshot 2016-01-10 19.51.53.png1920×1080 304 KB

And this is the same map with the player in the same spot but without me deleting a few hundred sprites. Notice that with the stream texture option unchecked ( or console command r.TextureStreaming 0 ) everything goes back to normal!

EDIT: Nothing goes back to normal. It is still the same but because i am not streaming anything anymore thats why the red warning is not appearing.

Any help would be greatly appreciated.

Thank you!

2

This is weird, and I have no real clue as to why this happens but I’d thought I’d bump because I’m currently beginning work on a bigger paper2d game and I’d like to avoid any issues haha.

3

Hey,

If unreal is loading everything to memory regardless if the level needs it or not , then we are is serious trouble my friend.

The bigger the game the worst it will become.

I can imagine myself saying in the minimum specs for Spartan. 1/3 of the game? Gtx980, Full game? Titan with 12Gb of Vram! XD

They are awfully quiet about something so serious and now that they have officially stopped Paper2d development i think that i am screwed…

Lets hope is something simple to fix or i am the one doing something wrong.

Cheers.

4

I’m curious about that too, i haven’t ran into this issue yet but i’m getting ready to build a large 2D world, some input from a dev would be really appreciated and reassuring

5

Hey,

do you happen to use this technique for your game?

6

Hello Demoneyejin,

You caught me by surprise! To be honest i had to read a bit about it to kind of understand what you were asking.

I do use png’s for flipbooks but for anything larger i use FXB or one texture that i tween realitime during gameplay through a json file.

So i guess… no?

7

Actually! it sounds like you do. Tweening through a json in game is exactly part 1 of that technique. the part two would be to do it to the sprite in place of the flipbook. So you’re half way there haha.

I was actually looking to learn how I can do this tweening via json files. Do you happen to have any source of learning for that so I can try to implement some samples?

8

Here you go sir: http://creature.kestrelmoon.com/

So any ideas what is happening with the insane amounts of Vram usage? A suggestion maybe?

9

Well I was figuring that it might be really large sprite sheets created from bone animations. How big are your sprite sheets? Also I was wondering… Are you using level streaming ? or are you separating things by levels?

10

Everything is below 2048x2048. And yes i do use level streaming. But i unload everything but the map that i am currently using but i also have the same issues on a single map. I am trying to narrow it down but what troubles me is why Unreal loads ALL of my textures into memory even though i am only using for example only 2. Weird…

11

The one thing I was thinking was because level streaming needs to call up data at whatever time, it requires having that data loaded. On your single map test: are there level stream options you might have enabled for the project? also how are you loading your textures/assets into the game? is it via C++ and or blueprints? or just one of the two?

12

I am conducting a few experiments. I will report back with any findings.

13

I reached out to the creators of Dustforce and asked what they did to manage all their large sprite sheets in their game and how they avoided using a ton of vram: the answer was that they had to create their own custom engine that “streamed” in sprite sheets as they were needed. so they would pretty much use one large sprite sheet for a character and a few other things and the engine would call specific parts of it, load it onto vram and then use it in the game until it wasn’t used anymore. It was freed after that.

I wonder if there’s a way we can stream paper2d sprite textures as needed.

14

Unreal has already build in texture streaming.Obviously i do not use Mipmaps but even if i set everything to power of two (use Mipmaps) and stream my textures i still get the same results. Maybe this is a paper2d only bug or limitation?

15

Check out this picture that i just took. Brand new paper2d template project that i added 2000 textures to the content browser. WITHOUT adding anything to the level i get these results.

Screenshot 2016-01-20 05.03.30.jpg
Screenshot 2016-01-20 05.03.30.jpg1920×1080 383 KB

16

whoa! that’s not right. yeah this is clearly a bug and or some kinda memory leak. have you put this on the answerhub ? if you list this as a bug it will get checked out or at least responded with. I’d post that picture along with everything else you have done.

17

I’ve been trying for over a month to convince (in answerhub under bug reports) them that there is something wrong with Paper2d and Textures in general but to no avail.
Oh man… How much i miss Noland…!

18

I managed to replicate this on the standard Paper2d template!

It appears that i was right. Unreal does not release the textures and in fact it adds them up all across the levels. Unless i am super wrong and there is something else that i am not doing correctly, then in this current situation, it is impossible to create a game.

I will post the link to the project here in case someone wants to play around and give me some feedback about my observations.

19

Don’t know much about texture streaming, but it seems to me not really an issue of texture streaming (since all your textures don’t even have mip maps), but more likely a serious/silly bug/feature of ue4’s resource management system.

Will try your project see what’s really happening…

20

If the problem is fixed when streaming is off, then there is something terribly wrong engine wise.

As Nate said, it must be a bug. I will try your project too and even make a bug report myself about it.

Can you give the link to your answerhub post? I cant seem to find it myself.

Also did you try this at 4.11 ? Maybe it is fixed. I will try it when I get home though maybe you want to check it out.