Is it possible to create textures dynamically

Development Rendering
, , , , ,
1

One question I’ve been wondering is it possible to generate textures dynamically through code or blueprints. for instance generating noise patterns, making a ray casting engine in UE4, making a drawing game or stamping textures to mesh instead of using decals.

In ActionScript 3 there is a class called BitmapData which is just bitmap data with functions like set and getPixel, which sets the pixel color at the specified X and Y coordinates of the BitmapData.

Does UE4 for have some sort of equivalent API.

2

I haven’t done too much with it, but the Substance Plugin has some cool features for generating textures in realtime. Check it out, maybe it can do some of the things you are wanting!

Have fun! =)

3

Doesn’t the standard material system cover what you are trying to acheive? or are you trying to alter the texture/material on the fly?

Materials can be a very powerful system and are for much more than bog standard texturing.

4

I’ve never heard of it but I’ll check it out

5

No, the material system is not what I need for this problem or using it in that way is needlessly complicated. The closest thing to the what I want is a render target however to my understanding there is no way to draw to a the texture on a pixel level.

6

Here’s some example code to get you started. You’ll need to adapt/change this for your own purposes. I’ve copied this out from my own more specialized code which dealt with fixed-size buffers (which is why it’s hard-coded to 64x64), but it shouldn’t be too hard to generalize it to arbitrary sizes or color depths. This is mostly just to get you started, it’s not a complete solution.

.h:

class MYPROJ_API FMyProjDynamicTextureUtilities
{
public:
	// Create a dynamic texture intended to be used for passing non-texture data
	// into materials. Defaults to 32-bit RGBA. The texture is not added to the
	// root set, so something else will need to hold a reference to it.
	static UTexture2D* CreateTransientDynamicTexture(int32 Width, int32 Height, EPixelFormat PixelFormat = PF_A32B32G32R32F);
	// Updates a region of a texture with the supplied input data. Does nothing
	// if the pixel formats do not match.
	static void UpdateTextureRegion(UTexture2D* Texture, int32 MipIndex, FUpdateTextureRegion2D Region, uint32 SrcPitch, uint32 SrcBpp, uint8* SrcData, bool bFreeData);
	// Convenience wrapper for updating a dynamic texture with an array of
	// FLinearColors.
	static void UpdateDynamicVectorTexture(const TArray<FLinearColor>& Source, UTexture2D* Texture);
	// Sets up a component's material instance parameters (on all materials) for
	// use with the supplied UTexture. The proper parameters (specified by
	// IndexParameterName and TextureParameterName) should exist on the
	// material, otherwise this will not have the proper effect.
	static void SetDynamicTextureAndIndex(class UStaticMeshComponent* Component, class UTexture2D* Texture, int32 Index, FName IndexParameterName, FName TextureParameterName);
};

.cpp:

// NOTE hard-coded to 64 * 64 texture sizes! I just copied this from some of my own code and modified it for you a little. You'll want to change a bunch of stuff in here for your own purposes.

void FMyProjDynamicTextureUtilities::UpdateTextureRegion(UTexture2D* Texture, int32 MipIndex, FUpdateTextureRegion2D Region, uint32 SrcPitch, uint32 SrcBpp, uint8* SrcData, bool bFreeData)
{
	if (Texture->Resource)
	{
		struct FUpdateTextureRegionsData
		{
			FTexture2DResource* Texture2DResource;
			int32 MipIndex;
			FUpdateTextureRegion2D Region;
			uint32 SrcPitch;
			uint32 SrcBpp;
			uint8* SrcData;
		};

		FUpdateTextureRegionsData* RegionData = new FUpdateTextureRegionsData;

		RegionData->Texture2DResource = (FTexture2DResource*)Texture->Resource;
		RegionData->MipIndex = MipIndex;
		RegionData->Region = Region;
		RegionData->SrcPitch = SrcPitch;
		RegionData->SrcBpp = SrcBpp;
		RegionData->SrcData = SrcData;

		{

		ENQUEUE_UNIQUE_RENDER_COMMAND_TWOPARAMETER(
			UpdateTextureRegionsData,
			FUpdateTextureRegionsData*, RegionData, RegionData,
			bool, bFreeData, bFreeData,
			{
				int32 CurrentFirstMip = RegionData->Texture2DResource->GetCurrentFirstMip();
				if (RegionData->MipIndex >= CurrentFirstMip)
				{
					RHIUpdateTexture2D(
						RegionData->Texture2DResource->GetTexture2DRHI(),
						RegionData->MipIndex - CurrentFirstMip,
						RegionData->Region,
						RegionData->SrcPitch,
						RegionData->SrcData
						+ RegionData->Region.SrcY * RegionData->SrcPitch
						+ RegionData->Region.SrcX * RegionData->SrcBpp
						);
				}
				// TODO is this leaking if we never set this to true??
				if (bFreeData)
				{
					FMemory::Free(RegionData->SrcData);
				}
				delete RegionData;
		});

		}
	}
}

void FMyProjDynamicTextureUtilities::UpdateDynamicVectorTexture(const TArray<FLinearColor>& Source, UTexture2D* Texture)
{
	// Only handles 32-bit float textures
	if (!Texture || Texture->GetPixelFormat() != PF_A32B32G32R32F) return;
	// Shouldn't do anything if there's no data
	if (Source.Num() < 1) return;

	UpdateTextureRegion(Texture, 0, FUpdateTextureRegion2D(0, 0, 0, 0, Texture->GetSizeX(), Texture->GetSizeY()), Texture->GetSizeX() * sizeof(FLinearColor), sizeof(FLinearColor), (uint8*)Source.GetData(), false);
}

UTexture2D* FMyProjDynamicTextureUtilities::CreateTransientDynamicTexture(int32 Width, int32 Height, EPixelFormat PixelFormat /*= PF_A32B32G32R32F*/)
{
	auto* Texture = UTexture2D::CreateTransient(Width, Height, PixelFormat);
	if (Texture)
	{
		Texture->CompressionSettings = TextureCompressionSettings::TC_VectorDisplacementmap;
		Texture->SRGB = 0;
		Texture->UpdateResource();
	}
	return Texture;
}

void FMyProjDynamicTextureUtilities::SetDynamicTextureAndIndex(class UStaticMeshComponent* Component, class UTexture2D* Texture, int32 Index, FName IndexParameterName, FName TextureParameterName)
{
	if (!Component || !Texture) return;
	for (int32 i = 0; i < Component->GetNumMaterials(); i++)
	{
		auto* DynamicMaterial = FMyProjInstanceProcedures::TryGetDynamicMaterial(Component, i);
		if (!DynamicMaterial) continue;
		FLinearColor CalculatedIndex(FMath::Fmod((float)Index, 64.0f) + 0.5f, FMath::FloorToFloat((float)Index / 64.0f) + 0.5f, 0.0f, 0.0f);
		CalculatedIndex /= 64.0f;
		DynamicMaterial->SetVectorParameterValue(IndexParameterName, CalculatedIndex);
		DynamicMaterial->SetTextureParameterValue(TextureParameterName, Texture);
	}
}
7

Well at a glance you’re telling me it’s completely possible but there also doesn’t appear to be a high level solution, like “Use the UDrawTexture Class” or something like that.

Truly if that’s the case there really should be something like that, it’d be really handy to just be able to say something as simple as say

SetPixel(10, 20 ,0xFF8A00);

I suppose beggars can’t be choosers but the Epic should really get on that.

Thanks for the code, I’ll have to take a long hard look at all this. I really don’t know what’s going on here.

8

You can make your own with that interface using the definition I provided above, if you want.

The reason you won’t typically see something like ‘SetPixel’ in UE4 in C++ is because it’s inefficient, inconvenient and typically not how you would generate a procedural texture. ‘SetPixel’ forces you to make a function call, which even if inline forces you to operate over single pixels at a time, and this is typically now how I would approach doing procedural textures. More typically you would operate over whole buffers at a time using SIMD or exchanging data with the GPU. The example I provided above where an array is used as the input source is not typically how you would fill a texture – you would operate over the memory buffer with your own code. I just posted it that was as an easy-to-understand example.

I hope that helps clear things up.

9

Your absolutely right for large textures setting each pixel one by one would be inefficient. You obviously want to involve the the GPU to generate the textures as much as possible. My gripe is more on the fact that there is no flat out built in simple solution.

10

Are you sure I can cast FLinearColor* to uint8* ? You ar doing something like :
(uint8*)Source.GetData(). That would return null I guess.

11

It’s fine to cast FLinearColor* to uint8* (assuming you are avoiding writes that might cause the strict aliasing rule to come into effect). This is used to write the data as bytes into the buffer.