Note: This post requires some knowledge about RHI/RDG and compute shaders in Unreal Engine.
I am trying to create a compute shader which has as part of its parameters a set of structured buffers, but I do not know how to access the memory of those buffers.
For the current time being, there are two input structured buffers and three output structured buffers.
I wish to fill the input buffers with some data (it being static const data or some generated data) as well as retrieve the data from the output buffers (and ultimately have it in a TArray variable).
I have looked at the following resources:
The general consesus across these is to use RHILockStructuredBuffer, however there are some issues with this:
-
It blocks the thread on which it runs which means that it is more difficult to do async copying. One of the resources proposed a solution by extending a class, but this does not seem very elegant and I hope there is some way to achieve this via the Unreal Engine codebase itself, rather than extending it.
-
Using this function does not seem very RDG-friendly. In the powerpoint slides they suggest to avoid writing boilerplate code and (for example) popose to add a pass via FComputeShaderUtils::AddPass(). However, to use the Lock/UnlockStructuredBuffer functions, you need to use them within a lambda that is passed as argument to GraphBuilder.AddPass(), making it an awkward situation.
I found about GraphBuilder.QueueBufferExtraction()*, but all it does (to my knowledge), is allow you to keep a buffer which was initially allocated just for the pass (that is, once the pass is over, the buffer would disappear - please correct me if I’m wrong) by giving you a reference to a pooled buffer.
My issue with this is:
- I cannot find a way to get to the underlying data to even issue a standard FMemory::MemCpy().
- This is just for retrieving the data. What about storing data in an input buffer? After all, I am dealing with FRDGBufferRef for the input shader, whereas for the QueueBufferExtraction() function returns TRefCountPtr of FPooledRDGBuffer.
I’m getting rather confused around all this and I seek help from others. All suggestions are welcome!
Below I have provided source of calling the shader.
ENQUEUE_RENDER_COMMAND(MarchingCubesShader)
([&](FRHICommandListImmediate& RHICmdList)
{
//Render Thread Assertion
check(IsInRenderingThread());
int VoxelCount = ScheduleParams.Width * ScheduleParams.Height * ScheduleParams.Depth;
int MaxVertexCount = VoxelCount * 15; // max 5 triangles per voxel -> max 15 vertices
FRDGBuilder GraphBuilder(RHICmdList);
FRDGBufferDesc ConstCubeEdgeFlagsDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(int), 256);
FRDGBufferDesc ConstTriangleConnectionTableDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(int), 256 * 16);
FRDGBufferDesc InputDensityDataDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(float), VoxelCount);
FRDGBufferDesc OutputVertexPositionsDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(FVector), MaxVertexCount);
FRDGBufferDesc OutputVertexNormalsDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(FVector), MaxVertexCount);
FRDGBufferDesc OutputTriangleIndicesDesc = FRDGBufferDesc::CreateStructuredDesc(sizeof(int), MaxVertexCount);
FRDGBufferRef ConstCubeEdgeFlagsBuffer = GraphBuilder.CreateBuffer(ConstCubeEdgeFlagsDesc, TEXT("SB_EdgeFlags"));
FRDGBufferRef ConstTriangleConnectionTableBuffer = GraphBuilder.CreateBuffer(ConstTriangleConnectionTableDesc, TEXT("SB_TriangleConnectionTable"));
FRDGBufferRef InputDensityDataBuffer = GraphBuilder.CreateBuffer(InputDensityDataDesc, TEXT("SB_DensityData"));
FRDGBufferRef OutputVertexPositionsBuffer = GraphBuilder.CreateBuffer(OutputVertexPositionsDesc, TEXT("SB_VertexPositions"));
FRDGBufferRef OutputVertexNormalsBuffer = GraphBuilder.CreateBuffer(OutputVertexNormalsDesc, TEXT("SB_VertexNormals"));
FRDGBufferRef OutputTriangleIndicesBuffer = GraphBuilder.CreateBuffer(OutputTriangleIndicesDesc, TEXT("SB_TriangleIndices"));
FRDGBufferUAVRef ConstCubeEdgeFlagsUAVRef = GraphBuilder.CreateUAV(ConstCubeEdgeFlagsBuffer);
FRDGBufferUAVRef ConstTriangleConnectionTableUAVRef = GraphBuilder.CreateUAV(ConstTriangleConnectionTableBuffer);
FRDGBufferUAVRef InputDensityDataUAVRef = GraphBuilder.CreateUAV(InputDensityDataBuffer);
FRDGBufferUAVRef OutputVertexPositionsUAVRef = GraphBuilder.CreateUAV(OutputVertexPositionsBuffer);
FRDGBufferUAVRef OutputVertexNormalsUAVRef = GraphBuilder.CreateUAV(OutputVertexNormalsBuffer);
FRDGBufferUAVRef OutputTriangleIndicesUAVRef = GraphBuilder.CreateUAV(OutputTriangleIndicesBuffer);
FMarchingCubesCS::FParameters* MarchingCubesCSParameters = GraphBuilder.AllocParameters<FMarchingCubesCS::FParameters>();
MarchingCubesCSParameters->Width = ScheduleParams.Width;
MarchingCubesCSParameters->Height = ScheduleParams.Height;
MarchingCubesCSParameters->Depth = ScheduleParams.Depth;
MarchingCubesCSParameters->VoxelScale = ScheduleParams.VoxelScale;
MarchingCubesCSParameters->CubeEdgeFlags = ConstCubeEdgeFlagsUAVRef;
MarchingCubesCSParameters->TriangleConnectionTable = ConstTriangleConnectionTableUAVRef;
MarchingCubesCSParameters->DensityData = InputDensityDataUAVRef;
MarchingCubesCSParameters->OutputVertexPositionsBuffer = OutputVertexPositionsUAVRef;
MarchingCubesCSParameters->OutputVertexNormalsBuffer = OutputVertexNormalsUAVRef;
MarchingCubesCSParameters->OutputTriangleIndicesBuffer = OutputTriangleIndicesUAVRef;
// Get a reference to our shader type from global shader map
TShaderMapRef<FMarchingCubesCS> MarchingCubesCSRef(GetGlobalShaderMap(GMaxRHIFeatureLevel));
// Compute the Group dimensions used for dispatching
FIntVector MarchingCubesCSGroupCount = FComputeShaderUtils::GetGroupCount(NUM_THREADS_PER_GROUP_XYZ,
FIntVector(ScheduleParams.Width, ScheduleParams.Height, ScheduleParams.Depth));
ValidateShaderParameters(MarchingCubesCSRef, *MarchingCubesCSParameters);
FComputeShaderUtils::AddPass(GraphBuilder, RDG_EVENT_NAME("Marching Cubes Test"), MarchingCubesCSRef, MarchingCubesCSParameters, MarchingCubesCSGroupCount);
GraphBuilder.QueueBufferExtraction(OutputVertexPositionsBuffer, OutputData.VertexPositionsBuffer, FRDGResourceState::EAccess::Read, FRDGResourceState::EPipeline::Compute);
GraphBuilder.QueueBufferExtraction(OutputVertexNormalsBuffer, OutputData.VertexNormalsBuffer, FRDGResourceState::EAccess::Read, FRDGResourceState::EPipeline::Compute);
GraphBuilder.QueueBufferExtraction(OutputTriangleIndicesBuffer, OutputData.TriangleIndicesBuffer, FRDGResourceState::EAccess::Read, FRDGResourceState::EPipeline::Compute);
// final step
GraphBuilder.Execute();
});
P.S. I hope UE 5 will have a better RDG structure and documentation to make compute shaders easier to work with. I beg you Epic!!
Edit: I probably need to post some more source, let me know if you need clarification!
although the buffers have to be locked because of the CPU-GPU boundary.