Is high-fidelity real-time rendering for interior design app achievable on mobile devices like the iPhone 12 and later?

[Ardavans1]

Exploring the potential for high-fidelity real-time rendering in mobile devices for interior design tool – how feasible is it to deliver console-quality visuals and smooth customization experiences on high-end smartphones like the iPhone 12 and later?

Quality reference (without animation):

Studio

[jblackwell]

That’s a pretty complex problem space you’re asking about, and ultimately, the answer depends on what you want to render and how you want to integrate it into the scene.

Good-looking assets in an AR background? Doable. Sure, not too much geometry in an AR setting. Textures might be low-res, but you can pull a number of tricks to get decent resolve. Lighting would be the real trick, and getting solid scene integration could be a problem. As you tack on complexity however, your scope will increase and your ability to get good perf will decrease.

I would personally be stunned if the video you showed me was an actual product of a mobile real-time renderer. The image is too clean, physics feel too good, and especially with that cloth folding on the couch, there’s no way a mobile renderer could handle all that at that fidelity.

So I guess what I’m asking is, what specifically do you want to pull off?

[Ardavans1]

Thanks, Interesting feedback! my goal is to test a real-time interior customization tool that leverages Unreal Engine’s high-end rendering quality, allowing users to adjust wall colors, flooring, and furniture in a mobile-optimized environment. The goal is to maintain high-quality graphics without overloading the GPU, even when implementing real-time customization features, without needing the animation or walkthrough feature

https://www.youtube.com/watch?v=17QZBUU11Ac (ofcourse with updated rendering quality)

[jblackwell]

I mean, if they could do it back then, you can probably do it now. It looks like what they’re doing is using texture swaps but keeping the lightmap bake the same, which is pretty much what you’d have to do to make it performant, but you couldn’t swap the models then.

What does ‘high-end rendering’ mean for you and your project? If it means photoreal archvis-style work, with real-time GI, you’re going to have problems.

Lumen is really expensive even on consoles. It’s technically supported for mobile now, but given the expense I’d be surprised if you could run it on the iPhone 12. It would have to use lumen’s software mode because only the iPhone 15 and later support hardware ray-tracing, and software RT has some quality issues that aren’t easy to address. That said, I don’t nessesarily think what you want to do is impossible.

Here’s what I’ll say: a photoreal, archvis-quality interior customizer is definitely possible with static lighting. It can run very fast, and you’ll have the performance headroom for good textures, dense meshes, and more.

A photoreal, archvis-quality customizer with dynamic lighting is maybe possible, if:

1. You keep your scene size as small as possible.
2. You keep your texture count to an absolute minimum.
3. You keep your light count, especially your shadowed light count, to an absolute minimum.
4. You keep your assets as low-poly as possible while still hitting your quality bar.
5. You hand-optimize the proxy meshes/SDFs for lumen to be as low-res as possible while still portraying your assets well.
6. You tune the lumen settings to reduce the nessesary sample count and scene size as low as it can go while still giving good visuals.
7. You make use of an upscaler like TSR to move from a low target resolution to a higher one.
8. You’re comfortable with a ~30fps update rate.

Real-time GI on mobile has been done before, in some cases quite well, but you should know that achieving that requires a lot of planning and optimization; it also limits accessibility, because the nicer your visuals are, the fewer people can experience them. You may even want to look into solutions like enlighten, which is a different GI plugin that’s lower-quality than lumen but far cheaper. There’s also a pretty good chance you’ll need to forsake ray-traced reflections in order to have acceptable performance, but lumen has a pretty good fallback.

If you’re comfortable working within all these limitations (and maybe waiting for another version of UE that’s better optimized to roll around), then I say go ahead. If you need help bringing lumen into performance, let me know.