Ways to appoach in game (runtime) art creation

This is more a general question/discussion on the best way to approach this. I’m working on a game about creating art, with the ability to draw on surfaces with high fidelity. Initially, drawing on a whiteboard, or canvas, with customizable brushes/markers is the goal.

I’ve prototyped various approaches:

  • Render Targets

This is easy to use for 2D art, however, I found it difficult to get the fidelity I seek, without tiling numerous render targets, and there is less direct control (programmatically).

  • Procedural Meshes

I’ve been prototyping this and it seems like an interesting approach. It seems performant enough for real-time painting usage, and offers fantastic opportunities for 3D art. I am however, running in to issues with Z-fighting when the procedural mesh intersects with itself, and I’m not sure how to solve that.

  • Dynamic Meshes

I haven’t looked in to this too much yet. I’ve read that it has less support for runtime usage, but also that it is meant to replace procedural meshes.

So that’s where I’m at so far. It seems each option has pros and cons. Procedural Meshes seem the most flexible, but I’m not sure about the performance implications for complex creations, and I have no idea how to handle Z fighting (e.g. drawing over previously drawn material) gracefully.

Any tips or advice appreciated!

I would say look into C++ and how render targets are done, then create something like it but for your needs, in C++. You can make your own struct that you paint in then function that copies it over to render target, so you can use it in unreal freely.

So create RGBA picture/texture struct, create own functions that can work fast with it, than use render target only to copy that into unreal.