How to query the material rendered at any pixel in the Level Editor Viewport

anonymous-edc · October 29, 2025, 4:55pm

Hello!

I am trying to create a tool that would do the following:

User right-clicks on anywhere in the level editor viewport.
The tool then will take that exact pixel that the user’s mouse right-clicked on, and query the material being rendered at that exact pixel, if any.
In the right-click context menu, there will be a sub-menu for the tool, something like “Material Finder Actions >”
Within the “Material Finder Actions” sub-menu, there will be options to either “Browse to Material/Material Instance” or “Edit Material/Material Instance”, similar to the actor actions that are available in that context menu.

Since there could be multiple materials assigned to an asset, I would strongly prefer to not simply get all Materials/Material instances that are applied to the selected actor. I am looking to get the material that is rendering at just the one pixel.

I understand that there are ways to raycast to the selected point and to get the actor that is there, but I’m struggling to find a way where we can get the singular Material/Material instance that is at that specific pixel under the mouse. Is there any way to query for the Material/Material instance rendered at any pixel in the Level Editor Viewport? If so, how?

Thanks,

Christina

Sam_Lapere · October 31, 2025, 12:56am

Hi,

thanks for reaching out. If you want to sample the material of the specific mesh section under the mouse cursor, this bit of code may be helpful (you may have to adapt the part that does the line tracing by your specific needs):

void APawn::GetMaterialAtPixel()
{
	APlayerController* PlayerController = UGameplayStatics::GetPlayerController(this, 0);
	FVector WorldOrigin, WorldDirection;
	if (PlayerController->DeprojectMousePositionToWorld(WorldOrigin, WorldDirection))
	{
		FHitResult HitResult;
		double TraceDistance = 10000000;
		FCollisionQueryParams CollisionQueryParams;
		CollisionQueryParams.bReturnFaceIndex = true;
		CollisionQueryParams.bTraceComplex = true;
		CollisionQueryParams.AddIgnoredActor(this);
 
		GetWorld()->LineTraceSingleByChannel(HitResult, GetActorLocation(), GetActorLocation() + WorldDirection * TraceDistance, ECollisionChannel::ECC_Visibility, CollisionQueryParams);
		const AActor* HitActor = HitResult.GetActor();
		
		if (HitActor)
		{
			UStaticMeshComponent* StaticMeshComp = HitActor->FindComponentByClass<UStaticMeshComponent>();
			
			if (StaticMeshComp)
			{
				UStaticMesh* StaticMesh = StaticMeshComp->GetStaticMesh();
				
				// Assuming LOD 0
				int32 LODIndex = 0;
				const FStaticMeshLODResources& LODResource = StaticMesh->GetRenderData()->LODResources[LODIndex];
				
				// The triangle index you are looking for
				int32 TriangleIndex = HitResult.FaceIndex;
 
				// The index buffer is a flat array of vertex indices. Each triangle uses 3 indices.
				// The "global" index of the first vertex of the triangle in the index buffer.
				uint32 GlobalIndexStart = TriangleIndex * 3;
 
				// Iterate through all sections in the LOD to find which one contains this index range
				for (int32 SectionIndex = 0; SectionIndex < LODResource.Sections.Num(); ++SectionIndex)
				{ 
					const FStaticMeshSection& Section = LODResource.Sections[SectionIndex];
 
					// Check if our triangle's starting index falls within this section's index buffer range
					uint32 SectionFirstIndex = Section.FirstIndex;
					uint32 SectionLastIndex = SectionFirstIndex + Section.NumTriangles * 3;
					if (GlobalIndexStart >= SectionFirstIndex && GlobalIndexStart < SectionLastIndex)
					{
						// Found the section! The triangle belongs to this SectionIndex
						// You can now access section-specific data, like the Material
						UMaterialInterface* SectionMaterial = StaticMeshComp->GetMaterial(Section.MaterialIndex);
 
						if (SectionMaterial)
						{
							UE_LOG(LogTemp, Warning, TEXT("Static mesh name: %s Section material name: %s"), *(StaticMesh->GetName()), *(SectionMaterial->GetName()));
						}
					}			
				}
			}
		}
	}
}

Let me know if this helps or if you have further questions.

Best,

Sam

Sam_Lapere · November 6, 2025, 3:52am

Hi,

>> Is there a way to get the LOD index that was used for the HitResult?

Determining the exact LOD of a mesh at the hit point of a line trace at runtime is not directly exposed in the FHitResult struct. Changing to a different LOD is primarily handled internally by the engine based on screen size, not typically for gameplay logic queries via line traces. That’s why I used LOD 0 in my example, but it may be possible to estimate the LOD using similar logic as the one used by the engine as in this code snippet:

float ComputeScreenSize(float Radius, float Distance, const ULocalPlayer* LocalPlayer)
{
	if (!LocalPlayer || !LocalPlayer->ViewportClient)
		return 0.0f;
 
	const float ScreenWidth = LocalPlayer->ViewportClient->Viewport->GetSizeXY().X;
	const float ScreenScale = Radius / Distance;
	return ScreenScale * ScreenWidth;
}
 
int32 EstimateLODIndex(UStaticMeshComponent* MeshComp, const FVector& CameraLocation, const ULocalPlayer* LocalPlayer)
{
	if (!MeshComp || !MeshComp->GetStaticMesh() || !LocalPlayer)
		return INDEX_NONE;
 
	const FStaticMeshRenderData* RenderData = MeshComp->GetStaticMesh()->GetRenderData();
	if (!RenderData || RenderData->LODResources.Num() == 0)
		return INDEX_NONE;
 
	// Calculate screen size
	FVector Origin;
	FVector BoxExtent;
	MeshComp->GetLocalBounds(Origin, BoxExtent);
	float Radius = BoxExtent.Size();
 
	FVector MeshLocation = MeshComp->GetComponentLocation();
	float Distance = FVector::Dist(CameraLocation, MeshLocation);
	float CalculatedScreenSize = ComputeScreenSize(Radius, Distance, LocalPlayer);
 
	// Get screen size thresholds
	const TArray<FStaticMeshSourceModel>& SourceModels = MeshComp->GetStaticMesh()->GetSourceModels();
	for (int32 LODIndex = 0; LODIndex < SourceModels.Num(); ++LODIndex)
	{
		float ScreenSize = SourceModels[LODIndex].ScreenSize.Default;
		// Compare against thresholds
		if (CalculatedScreenSize >= ScreenSize)
			return LODIndex;
	}
 
	// Return the last LOD index
	return SourceModels.Num() - 1;
}

You can than replace int32 LODIndex = 0; in the previous code with the following lines:

ULocalPlayer* LocalPlayer = PlayerController->GetLocalPlayer();
FVector CameraLocation = PlayerController->PlayerCameraManager->GetCameraLocation();
int32 LODIndex = EstimateLODIndex(StaticMeshComp, CameraLocation, LocalPlayer);

This may not work in all cases. I believe the issue you are seeing is that there is a discrepancy between the collision mesh and the rendered mesh. If you open the static mesh editor for a specific mesh, there is a setting called “LOD for Collision” which defaults to 0. This determines the LOD mesh that will be used for complex (i.e. per-face) collisions, regardless of the actual LOD of the rendered mesh in the scene (there is only one collision mesh possible). Unfortunately Unreal does not provide a method to perform line traces against the visible geometry LOD, but the following steps may work (I have not tested this):

1) perform a “simple” line trace from the mouse pointer into the scene (use a FCollisionQueryParams struct with bTraceComplex set to false)

2) if a static mesh is hit, estimate its LOD with the code above and temporarily change its “LOD for Collision” to the estimated LOD

3) perform a more accurate “complex” line trace (as in the first code sample) against the collision LOD mesh

4) revert the mesh’s LOD for Collision setting back to its original value

Hopefully that helps, let me know if you have more questions. If the above doesn’t work, would you be able to provide a minimal repro project containing a problematic mesh, so I can further investigate?

Thanks,

Sam

Sam_Lapere · November 10, 2025, 2:36am

Hi,

thanks for providing the code sample, it seems to work fine when I tested it with a simple multi-sectioned mesh, but I haven’t done any elaborate testing. The code sample I provided to estimate LODs might not work for Nanite meshes, given that Nanite has its own automatic LOD implementation. In that case you can remove the LOD estimation part from the code.

>> instead of the raycast hitting the complex collision (which they explained does not have a material assigned), they think I could be hitting the Nanite Fallback mesh, which is apparently where the complex collision is derived from

The Nanite fallback mesh and the complex collision mesh are actually not necessarily the same. You can visualize the fallback mesh in the static mesh editor (under the eye icon in 5.6 or under the Show menu in older versions, you have to close the menu again to see the actual fallback mesh). You can also visualize the complex collision mesh from the same menu, and it should be identical to the original source mesh (which can be visualized with Lit Wireframe).

Below are some visualizations of a mesh with multiple sections and materials (first is complex collision mesh, then Nanite mesh wireframe and finally Nanite fallback mesh wireframe), which show the difference between the complex collision mesh and Nanite fallback mesh:

[Image Removed]

I’m not exactly sure why it isn’t working though. For the problematic meshes, can you test again after setting their Complex Collision as “Use Complex Collision As Simple” in the static mesh editor’s Details panel? That will force using the original mesh as the collision mesh.

Best,

Sam

Sam_Lapere · November 11, 2025, 1:54am

Hi,

>> I added up the total number of triangles in each section of LODResource, and I notice that the total is equal to the total number of “Fallback Triangles” for the mesh I’m testing with

that is odd. I searched the engine code and the string used to display the “Fallback Triangles” is called StaticMeshTriangleCount (line 897 in StaticMeshEditorViewportClient.cpp) and is defined as below (on line 881 in StaticMeshEditorViewportClient.cpp):

const FText StaticMeshTriangleCount = FText::AsNumber(StaticMeshEditorPtr.Pin()->GetNumTriangles(CurrentLODLevel));

The CurrentLODLevel is the result of executing a lambda function. I would suggest to place some breakpoints around this code and step through the relevant parts to inspect the values of these variables at runtime and see if it’s what you expect.

Regarding the meshes, I understand you cannot provide these for confidentiality reasons, but would it be possible to create or find a non-confidential alternative mesh that still exhibits the same behaviour when picking materials, so we can investigate this further? Without having access to such an asset it will be difficult to debug this further.

Thanks,

Sam

anonymous-edc · November 4, 2025, 1:30am

Hi Sam,

Thank you so much for the response! This code snippet is helpful -- I am likely not going to be doing this logic from within an APawn like you have, since this would be a tool within the Level Editor, which I believe does not have a player controller. However, I was able to figure out a way to raycast using FSceneView::DeprojectScreenToWorld rather than PlayerController->DeprojectMousePositionToWorld.

I have a follow-up question: Is there a way to get the LOD index that was used for the HitResult? I see that you have the code to assume LOD 0 -- why is this? When I tested this out, I end up getting (for example) MaterialA when my mouse hovers over a section of a static mesh that is very very clearly MaterialB. I am wondering if there might be a way to query what LOD was used during the raycast, and then use that same index when we filter the LODs to later find the Section that the triangle resides within.

Another idea: I also notice that the asset I am testing this out with has a ComplexCollisionMesh -- perhaps my issue is that I should be getting the RenderData > LODResources from here? I notice that these are different from the LODResources from the UStaticMesh’s RenderData. Using this has also been able to get me the correct material, so I’m not sure if this is the correct path or if it’s a red herring. Please let me know your thoughts on this and if I am correct/wrong.

Best,

Christina

anonymous-edc · November 4, 2025, 6:33pm

UPDATE: So I have tried with a couple different assets, and it seems like the ComplexCollisionMesh is not the correct route either. But I notice that for the few assets I have tested without a ComplexCollisionMesh, your logic works great and LOD 0 works fine. But with the more complex assets I have (that also happen to have ComplexCollisionMeshes, unsure if that’s related or not), the logic that you have provided me is not returning the proper Material. I am still locally investigating to see if I can figure out why this is happening and what the solution is. Please let me know if you have any insight into this, and please also let me know why you have gone with a default LOD of 0.

anonymous-edc · November 5, 2025, 10:06pm

I have yet another update. So I believe the issue here is that for complex static meshes I am testing with, their collision does not perfectly wrap the static mesh, leading to the hit location to be slightly off, which I think is then causing me to retrieve the incorrect material.

Is there any function or any other way that I can do a ray trace to the actual visible geometry of the static mesh, not to the mesh’s collision? I have been doing some digging around but haven’t found something like this. Please let me know if all of this sounds like I’m on the right investigation path here, or if you think there may be another issue at play here. Also, I would like it to be assumed that I cannot edit/modify the collisions in this scenario.

Best, Christina

anonymous-edc · November 7, 2025, 1:23am

Thanks for the response, Sam!

Unfortunately, I believe I am either misunderstanding a part of the steps you mentioned, or perhaps it is still not working for a different reason. Before I continue to investigate, I figured I should go ahead a post what I have locally so that you are able to take a look sooner rather than later. I will follow up on this post if I am to find where I am going wrong.

Below is what I have right now, and it all lives within the action mapping of a Level Editor Viewport Right-Click context menu. The only differences between what I have and what you sent me is that I put it all in this one section (I understand it’s not good programming practices, but this is just temporary as I am investigating the possibility of this tool. I will clean it up and refactor it afterwards), and I also do not use the player controller (since we are in the editor).

Unfortunately, I will not be able to provide an example problematic mesh for privacy reasons.

Another note/update: I have spoken with some colleagues, and some of them have suggested that instead of the raycast hitting the complex collision (which they explained does not have a material assigned), they think I could be hitting the Nanite Fallback mesh, which is apparently where the complex collision is derived from. I just wanted to call this out in case this information is helpful to you.

I will keep investigating further to try and get this figured out. If you have any ideas to help me further, or if you see any issues with the code snippet I have pasted below (much of which was piecing together what you have previously suggested me), please do let me know. I feel like we are getting close to the solution!

FVector OutWorldPosition, OutWorldDirection;
if (FLevelEditorViewportClient* LevelViewportClient = GCurrentLevelEditingViewportClient)
{
	if (FViewport* ActiveViewport = LevelViewportClient->Viewport)
	{
		FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues(
			ActiveViewport,
			LevelViewportClient->GetScene(),
			LevelViewportClient->EngineShowFlags)
			.SetRealtimeUpdate(true));
		FSceneView* View = LevelViewportClient->CalcSceneView(&ViewFamily);
 
		FIntPoint MousePos;
		LevelViewportClient->Viewport->GetMousePos(MousePos);
 
		const FIntPoint ViewportSize = ActiveViewport->GetSizeXY();
		const FIntRect ViewRect = FIntRect(0, 0, ViewportSize.X, ViewportSize.Y);
		const FMatrix InvViewProjectionMatrix = View->ViewMatrices.GetInvViewProjectionMatrix();
		FSceneView::DeprojectScreenToWorld(MousePos, ViewRect, InvViewProjectionMatrix, OutWorldPosition, OutWorldDirection);
 
		// 1) perform a "simple" line trace from the mouse pointer into the scene(use a FCollisionQueryParams struct with bTraceComplex set to false)
		FHitResult HitResult;
		double TraceDistance = 10000000;
		FCollisionQueryParams CollisionQueryParams;
		CollisionQueryParams.bReturnFaceIndex = true;
		CollisionQueryParams.bTraceComplex = false;
 
		UWorld* World = GEditor->GetEditorWorldContext().World();
		World->LineTraceSingleByChannel(HitResult, OutWorldPosition, OutWorldPosition + OutWorldDirection * TraceDistance, ECollisionChannel::ECC_Visibility, CollisionQueryParams);
		const AActor* HitActor = HitResult.GetActor();
 
		if (HitActor)
		{
			UStaticMeshComponent* StaticMeshComp = HitActor->FindComponentByClass<UStaticMeshComponent>();
 
			if (StaticMeshComp)
			{
				UStaticMesh* StaticMesh = StaticMeshComp->GetStaticMesh();
				// save LODForCollision value
				int32 OriginalLODForCollision = StaticMesh->LODForCollision;
 
				//2) if a static mesh is hit, estimate its LOD with the code above and temporarily change its "LOD for Collision" to the estimated LOD
				const FStaticMeshRenderData* RenderData = StaticMeshComp->GetStaticMesh()->GetRenderData();
				int32 LODIndexToUse = 0;
				if (RenderData && RenderData->LODResources.Num() != 0)
				{
					// Calculate screen size
					FVector Origin;
					FVector BoxExtent;
					StaticMeshComp->GetLocalBounds(Origin, BoxExtent);
					float Radius = BoxExtent.Size();
 
					FVector MeshLocation = StaticMeshComp->GetComponentLocation();
					float Distance = FVector::Dist(OutWorldPosition, MeshLocation);
 
					// Compute screen size
					const float ScreenWidth = ActiveViewport->GetSizeXY().X;
					const float ScreenScale = Radius / Distance;
					float CalculatedScreenSize = ScreenScale * ScreenWidth;
 
					// Get screen size thresholds
					const TArray<FStaticMeshSourceModel>& SourceModels = StaticMeshComp->GetStaticMesh()->GetSourceModels();
					for (int32 LODIndex = 0; LODIndex < SourceModels.Num(); ++LODIndex)
					{
						float ScreenSize = SourceModels[LODIndex].ScreenSize.Default;
						// Compare against thresholds
						if (CalculatedScreenSize >= ScreenSize)
						{
							LODIndexToUse = LODIndex;
							break;
						}
					}
				}
 
				StaticMesh->LODForCollision = LODIndexToUse;
 
				// 3) perform a more accurate "complex" line trace(as in the first code sample) against the collision LOD mesh
				CollisionQueryParams.bTraceComplex = true;
				World->LineTraceSingleByChannel(HitResult, OutWorldPosition, OutWorldPosition + OutWorldDirection * TraceDistance, ECollisionChannel::ECC_Visibility, CollisionQueryParams);
 
				const FStaticMeshLODResources& LODResource = StaticMesh->GetRenderData()->LODResources[LODIndexToUse];
 
				// The triangle index you are looking for
				int32 TriangleIndex = HitResult.FaceIndex;
 
				// The index buffer is a flat array of vertex indices. Each triangle uses 3 indices.
				// The "global" index of the first vertex of the triangle in the index buffer.
				uint32 GlobalIndexStart = TriangleIndex * 3;
					
				DrawDebugPoint(World, HitResult.Location, 10, FColor(255, 0, 0), true, 30, 0);
 
				// Iterate through all sections in the LOD to find which one contains this index range
				for (int32 SectionIndex = 0; SectionIndex < LODResource.Sections.Num(); ++SectionIndex)
				{
					const FStaticMeshSection& Section = LODResource.Sections[SectionIndex];
 
					// Check if our triangle's starting index falls within this section's index buffer range
					uint32 SectionFirstIndex = Section.FirstIndex;
					uint32 SectionLastIndex = SectionFirstIndex + Section.NumTriangles * 3;
					if (GlobalIndexStart >= SectionFirstIndex && GlobalIndexStart < SectionLastIndex)
					{
						// Found the section! The triangle belongs to this SectionIndex
						// You can now access section-specific data, like the Material
						UMaterialInterface* SectionMaterial = StaticMeshComp->GetMaterial(Section.MaterialIndex);
 
						if (SectionMaterial)
						{
							UE_LOG(LogTemp, Warning, TEXT("Static mesh name: %s Section material name:         %s"), *(StaticMesh->GetName()), *(SectionMaterial->GetName()));
						}
					}
				}
 
				// 4) revert the mesh's LOD for Collision setting back to its original value
				StaticMesh->LODForCollision = OriginalLODForCollision;
			}
		}
	}
}

anonymous-edc · November 10, 2025, 6:55pm

Hi,

I noticed that for the problematic static meshes I am testing with, their Collision Complexity in the static mesh editor’s Details panel are all already set to “Use Complex Collision As Simple”.

Another thing that I noticed -- Regarding the following line of code:

const FStaticMeshLODResources& LODResource = StaticMesh->GetRenderData()->LODResources[0];

I added up the total number of triangles in each section of LODResource, and I notice that the total is equal to the total number of “Fallback Triangles” for the mesh I’m testing with (and I was able to conclude this by looking at the number of “Fallback Triangles” listed in the static mesh editor viewport’s top left corner). Perhaps this is the part that’s wrong? Should we be using the HitResult’s FaceIndex and mapping that to the material based on the sections of the Fallback Triangles?

Best,

Christina

anonymous-edc · November 11, 2025, 1:38am

Hey Sam,

In the Level Editor, there is a view mode for the viewport called Nanite Visualization > Picking. This appears to be close to that I am trying to achieve -- it finds the material rendered at the point I select (for assets that are nanite-enabled). I am looking into the possibility of using some of the same logic for my scenario, but I’m having a few difficulties as I am not very familiar with nanite under the hood. I was wondering if you had any insight into whether this route would be a plausible solution, and if you have any advice in regard to this idea.

Best,

Christina

Sam_Lapere · November 11, 2025, 2:24am

Sorry, I didn’t see your last post about the Nanite visualisation mode and replied to your previous one.

On first sight, I think that emulating the Nanite picking behaviour might be the more robust approach, but it may not be straightforward as most of the logic is performed on the GPU in a compute shader (see lines 405-435 in NaniteVisualize.cpp and picking related logic in NaniteVisualize.usf). I would have to investigate this more and discuss the feasibility of this approach with my team mates. Once I know more I’ll get back.

Sam

Sam_Lapere · November 11, 2025, 2:40am

Looking a bit deeper into this, this might be quite easy to achieve (no need for GPU compute shaders). I think what you are looking for is called *PickedMaterialSection.ShadingMaterialProxy which you can get from NaniteVisualize.cpp and then get its name with *PickedMaterialSection.ShadingMaterialProxy->GetMaterialName())).

Let me know if that works,

Best,

Sam

anonymous-edc · November 11, 2025, 5:59pm

Thanks for the responses Sam!

Yes, I was also looking to get the PickedMaterialSection -- but I am unsure how to get the material index for the material sections. Here is what I have right now:

// StaticMeshComp is the UStaticMeshComponent* from the hit result actor
 
const FPrimitiveSceneProxy* SceneProxy = StaticMeshComp->GetSceneProxy();
const Nanite::FSceneProxyBase* PickedNaniteProxy = (const Nanite::FSceneProxyBase*)SceneProxy;
const TArray<Nanite::FSceneProxyBase::FMaterialSection>& PickedMaterialSections = PickedNaniteProxy->GetMaterialSections();
// After this, NaniteVisualize.cpp uses the PickingFeedback.MaterialIndex as the 
// value for the Material Section index, which I do not have in my scenario.

I am currently trying to find a way to get the index that I need. Please let me know if you find anything in the meantime.

Thanks,

Christina

anonymous-edc · November 12, 2025, 7:38pm

Hi Sam,

I have created a test asset that seems to allow me to repro the issues I am having. Please try using different color faces of the cube to see if you end up getting the correct material. Some triangles work properly and some do not. I noticed that the problematic assets I have are all nanite enabled, so that is definitely where the problem lies. Additionally, I am still facing issues with getting the material index for the material sections in my previous comment. Do I need to do a GPU pass to somehow get the index there? I am not quite familiar with how to do this if so.

Best,

Christina

Sam_Lapere · November 13, 2025, 4:23am

Hi,

thanks a lot for the repro asset. After duplicating the asset and disabling Nanite, I can confirm that the picking code I provided only works for the non-Nanite version. Picking materials on the Nanite mesh is almost always wrong (they seem off by one index). I will file a bug report for that with Epic.

>> I am still facing issues with getting the material index for the material sections in my previous comment. Do I need to do a GPU pass to somehow get the index there?

Yes, you have to emulate the picking logic of AddVisualizationPasses(), PerformPicking() and DisplayPicking() in NaniteVisualize.cpp. Look for the lines in DeferredShadingRenderer.cpp that call AddVisualizationPasses(), it’ll show how to get a picking feedback buffer. The FNanitePickingFeedback struct in Engine/Shaders/Shared/NaniteDefinitions.h holds the material index you need.

Hopefully that helps.

Sam

Sam_Lapere · November 24, 2025, 9:13pm

Hi,

the bug with material picking for Nanite meshes is now available on the public issue tracker at this link, so you can track its progress.

Best regards,

Sam

anonymous-edc · November 24, 2025, 9:49pm

Hi Sam,

Thanks for logging the bug and for sharing it! For communications sake, I should let you know that the investigations and work I was doing on this have been paused as I am pivoting to other things. But if I end up picking this back up sometime in the future, I will likely revisit this thread and reach out if needed. Thank you for all the help, and I’ll keep an eye out for if this bug gets resolved!

Best,

Christina

Sam_Lapere · November 25, 2025, 12:17am

No problem and thanks for getting back. I will close this ticket for now, but feel free to open a new one and refer to this case when you have time to look at this feature again and encounter more issues.

Best regards,

Sam