How to take photos at the same time for multiple cameras in pure C++(without blueprint or sth)?

ThanosSteve · November 29, 2023, 3:57pm

hi, thanks for help me with C++ in UE, I’m working on a crowd simulator, and it needs to take high resolution(like 4K) photos from a top-down view for many(like 5,000) cameras
at different locations at the same time for one specific frame.
Now I have achieved one camera taking photos with a SceneCaptureComponent2D which is attached on my CameraActor(Blueprint), and the Component2D has a Texture Target.

In C++, I create a TArray to store the cameras spawned by the afore-mentioned Blueprint Class, it works only when I create 1 camera, but when I create more than 1, all the cameras save the same pic.
I don’t know why and I want to achieve the multiple cameras taking photos at different locations problem in pure C++. BUT HOW?
Here are my codes:

//take pics
for (int i = 0; i < Cameras.Num();i++) {
    if(Cameras[i])
        TakeScreenshot(i);
}

void AReplayCrowdSystem::TakeScreenshot(int CameraIndex)
{
    if (Cameras[CameraIndex])
    {

        FString FilePath = FPaths::ProjectContentDir() +...;
        Cameras[CameraIndex]->SavePath = FilePath;
        UE_LOG(LogTemp, Warning, TEXT("camera location:%s"), *Cameras[CameraIndex]->GetActorLocation().ToString());
        Cameras[CameraIndex]->SavePic();

the SavePic()is in the blueprint.

ThanosSteve · November 30, 2023, 1:59pm

Anyone could help me?

3dRaven · November 30, 2023, 9:23pm

header


#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "Engine/TextureRenderTarget2D.h" 
#include "ReplayCrowdSystem.generated.h"


UCLASS()
class CAPTURECAMS_API AReplayCrowdSystem : public AActor
{
	GENERATED_BODY()

public:
	// Sets default values for this actor's properties
	AReplayCrowdSystem();

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:
	// Called every frame
	virtual void Tick(float DeltaTime) override;

	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	bool Debug = false;

	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	TArray<ACameraActor*> Cameras;
	
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString RelativeDirectory;

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Prefix;

	UFUNCTION(BlueprintCallable)
	void TakeScreenshot(int CameraIndex);

	UFUNCTION(BlueprintCallable)
	void TakeScreenshots();
	
	UFUNCTION()
	void CompleteImage();

	UPROPERTY(EditDefaultsOnly, BlueprintReadWrite)
	class USceneCaptureComponent2D* CamCapture;

	UPROPERTY()
	USceneComponent* Root;

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	class UTextureRenderTarget2D* RT;
	
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FVector2D RTSize = FVector2D(1024, 1024);
	
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	TEnumAsByte<ETextureRenderTargetFormat> TargetFormat;
	


private:
	class UCameraComponent* Cam;
	
	void InitCapture();

	int32 NumberOfImagesSaved;

};

CPP


#include "ReplayCrowdSystem.h"
#include "Camera/CameraComponent.h"
#include "Components/SceneCaptureComponent2D.h" 
#include "Kismet/KismetRenderingLibrary.h"
#include "Camera/CameraActor.h" 
#include "Camera/CameraComponent.h" 
#include "ImageWriteBlueprintLibrary.h" 

// Sets default values
AReplayCrowdSystem::AReplayCrowdSystem()
{
    Root = CreateDefaultSubobject<USceneComponent>("Root");
    SetRootComponent(Root);
    // Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
    PrimaryActorTick.bCanEverTick = false;

    Cam = CreateDefaultSubobject<UCameraComponent>("Cam");
    Cam->SetupAttachment(Root);
    CamCapture = CreateDefaultSubobject<USceneCaptureComponent2D>("Cam Capture");
    CamCapture->SetupAttachment(Root);
    CamCapture->bCaptureOnMovement = false;
    CamCapture->bCaptureEveryFrame = false;
}

// Called when the game starts or when spawned
void AReplayCrowdSystem::BeginPlay()
{
    Super::BeginPlay();
    InitCapture();


}

// Called every frame
void AReplayCrowdSystem::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);
}


void AReplayCrowdSystem::InitCapture() {
    if (CamCapture == nullptr) {
        CamCapture = NewObject<USceneCaptureComponent2D>(this);
        CamCapture->RegisterComponent();
        CamCapture->bCaptureOnMovement = false;
        CamCapture->bCaptureEveryFrame = false;        
        CamCapture->DetailMode = EDetailMode::DM_Epic;
    }
    if (RT == nullptr) {
        RT = UKismetRenderingLibrary::CreateRenderTarget2D(GetWorld(), RTSize.X, RTSize.Y);        
    }
}


void AReplayCrowdSystem::TakeScreenshots() {
    NumberOfImagesSaved = 0;
    for (int i = 0; i < Cameras.Num(); i++) {
        if (Cameras[i])
            TakeScreenshot(i);
    }
}

void AReplayCrowdSystem::CompleteImage()
{
    NumberOfImagesSaved++;
    if (Debug == true) {
        if (GEngine) {
            GEngine->AddOnScreenDebugMessage(-1, 10, FColor::Cyan, "Saved [" + FString::FromInt(NumberOfImagesSaved) + "]");
        }

        if (NumberOfImagesSaved == Cameras.Num()) {
            if (GEngine) {
                GEngine->AddOnScreenDebugMessage(-1, 10, FColor::Green, "All Saved");
            }
        }
    }
}


void AReplayCrowdSystem::TakeScreenshot(int CameraIndex)
{
    if (Cameras[CameraIndex])
    {
        InitCapture();

        FString FilePath = FPaths::ProjectContentDir() + RelativeDirectory;
        FString FileName = Prefix + FString::FromInt(CameraIndex) + ".png";

        UE_LOG(LogTemp, Warning, TEXT("camera location:%s"), *Cameras[CameraIndex]->GetActorLocation().ToString());

        if (CamCapture)
        {            
            SetActorLocation(Cameras[CameraIndex]->GetActorLocation());
            SetActorRotation(Cameras[CameraIndex]->GetActorRotation());
            FMinimalViewInfo viewInfo;
            Cam->GetCameraView(0, viewInfo);
            CamCapture->SetCameraView(viewInfo);
            RT->RenderTargetFormat = TargetFormat;
            CamCapture->bAlwaysPersistRenderingState = true;
            CamCapture->CaptureSource = SCS_FinalColorLDR;
            CamCapture->TextureTarget = RT;
            CamCapture->SetCameraView(viewInfo);
            CamCapture->CaptureScene();
            RT->UpdateResourceImmediate(false);
            //UKismetRenderingLibrary::ExportRenderTarget(this, RT, FilePath, FileName);
                                    
            FImageWriteOptions Options;
            Options.bAsync = false;
            Options.bOverwriteFile = true;
            Options.Format = EDesiredImageFormat::PNG;            
            Options.OnComplete.BindUFunction(this, "CompleteImage");
            Options.CompressionQuality = 100;

            UImageWriteBlueprintLibrary::ExportToDisk(RT, FilePath +"/"+ FileName,Options);            
            RT->UpdateResourceImmediate(true);
        }
    }
}

Project
captureCams.zip (5.2 MB)

ThanosSteve · December 1, 2023, 4:31am

Thanks a lot! Your code work quite well after I add it to my system, but only one problem is still bothering me:
in my system, I got a csv file including crowd’s trajectoy, at each timeslot, I wany to replay the characters’ [location, rotation] and their animation(walk, a animation sequence file in my content dir).
I’ve done it by spawn the characters in the same timeslot(like timeslot i), and use function Character->GetMesh()->PlayAnimation(Anim, true); to play the animation.
But after I add your code, all work well except the characters in the pics saved do not play animation.
Could you help me with it？Thanks and I know I am being stupid, sorry for taking your time.
after adding code:(Pic takes when animation is not playing)

what I want(pic that I take before):(Pic takes when animation is playing)

My code:
Header file:

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Character.h"
#include "CrowdSpawnSystem.h"
#include "MyCameraActor.h"
#include "Engine/TextureRenderTarget2D.h"
#include "CaptureActor.h"
#include "ReplayCrowdSystem.generated.h"

UCLASS()
class POPULATIONSYSTEMFULLPACK_API AReplayCrowdSystem : public ACharacter
{
	GENERATED_BODY()

public:
	// Sets default values for this character's properties
	AReplayCrowdSystem();

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:	
	// Called every frame
	virtual void Tick(float DeltaTime) override;

	// Called to bind functionality to input
	virtual void SetupPlayerInputComponent(class UInputComponent* PlayerInputComponent) override;


	//Crowd Trajectory Array
	TArray<FCharacterTrajectoryInfo> CrowdTrajectoryArray{};

	//Default FileName to read, Default Dir is FPaths::ProjectContentDir()
	FString DefaultFileName = FPaths::ProjectContentDir() + "Trajectory.csv";

	//Read a Trajectory File to Memory, returns the max_timeslot of the file
	// !!!!!make SURE that the CSV file's structure is Right:
	// "%d,%d,%s,%f,%s\n"
	// ped.timeslot, ped.ID, *ped.Location.ToString(), ped.Rotation.Yaw, *ped.BP_Name!!!!! 
	int ReadTrajectoryFile(const FString& FilePath);

	//maximum of timeslots
	int max_timeslot = 0;

	//place the characters of an exact timeslot into the scene, return next startIndex to search(for continuously replay)
	//!!!!!Ensure that the trajectory file is ordered by timeslot
	//timeslot: the timeslot to replay;
	//startIndex: from which Index Of The Array(Row Of CSV File) to search, set to 0 as default
	int ReplayTimeslot(int timeslot, int startIndex = 0);

	//Blueprint to use
	TSubclassOf<AActor> characterBPToSpawn;

	//Animation
	UAnimSequence* Anim;

	//Character Array in a timeslot
	TArray<ACharacter*> CharactersInTimeSlot;

	//Replay All Timeslots from 1 to max_timeslot
	FTimerHandle Timer;
	void ReplayAllTimeslots();
	int current_timeslot=1;

	bool CaptureMode = false;
	TArray<FVector> CameraLocations;
	TArray<AMyCameraActor*> Cameras; //Cameras to capture crowd
	//for CameraToUse to take Screenshot
	void TakeScreenshot(int CameraIndex);

	//
	//ACaptureActor* CaptureActor1;

	UPROPERTY(EditDefaultsOnly, BlueprintReadWrite)
	class USceneCaptureComponent2D* CamCapture;

	UPROPERTY()
	USceneComponent* Root;

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	class UTextureRenderTarget2D* RT;

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FVector2D RTSize = FVector2D(4096, 2304);

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	TEnumAsByte<ETextureRenderTargetFormat> TargetFormat;

	UFUNCTION()
	void CompleteImage();

private:
	class UCameraComponent* Cam;

	void InitCapture();

	int32 NumberOfImagesSaved;

};

CPP:

#include "ReplayCrowdSystem.h"
//#include "Components/SceneCaptureComponent2D.h"
//#include "Camera/CameraActor.h"
//#include "Camera/CameraComponent.h"
#include "GameFramework/CharacterMovementComponent.h"

#include "HighResScreenshot.h"

#include "MyCameraActor.h"
#include "MyBlueprintFunctionLibrary.h"
#include "ImageUtils.h"
#include "Misc/FileHelper.h"
#include "Engine/LocalPlayer.h"
#include "Engine/GameViewportClient.h"

#include "Camera/CameraComponent.h"
#include "Components/SceneCaptureComponent2D.h" 
#include "Kismet/KismetRenderingLibrary.h"
#include "Camera/CameraActor.h" 
#include "Camera/CameraComponent.h" 
#include "Runtime/ImageWriteQueue/Public/ImageWriteBlueprintLibrary.h"

// Sets default values
AReplayCrowdSystem::AReplayCrowdSystem()
{
 	// Set this character to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
	PrimaryActorTick.bCanEverTick = true;

    //TODO; Load BPs of characters
    FString CharacterBlueprintPath = TEXT("/Script/Engine.Blueprint'/Game/CrowdSpawnSystem/Character_BPs/ThirdPersonCharacter_5.ThirdPersonCharacter_5_C'");
    UClass* CharacterBlueprintClass = LoadClass<ACharacter>(nullptr, *CharacterBlueprintPath);
    if (CharacterBlueprintClass == nullptr) {
        UE_LOG(LogTemp, Warning, TEXT("Failed to load asset: %s"), *CharacterBlueprintPath);
    }
    else {
        UE_LOG(LogTemp, Warning, TEXT("Succeeded to load asset: %s"), *CharacterBlueprintPath);
        characterBPToSpawn = CharacterBlueprintClass;
    }

    //Load animation
    FString AnimSeqPath = TEXT("/Script/Engine.AnimSequence'/Game/CrowdSpawnSystem/Animations/ThirdPersonWalk_1.ThirdPersonWalk_1'");
    static ConstructorHelpers::FObjectFinder<UAnimSequence> AnimSeqFinder(TEXT("/Script/Engine.AnimSequence'/Game/CrowdSpawnSystem/Animations/ThirdPersonWalk_1.ThirdPersonWalk_1'"));
    Anim = AnimSeqFinder.Object;

    //initialize capture
    Root = CreateDefaultSubobject<USceneComponent>("Root");
    SetRootComponent(Root);
    Cam = CreateDefaultSubobject<UCameraComponent>("Cam");
    Cam->SetupAttachment(Root);
    CamCapture = CreateDefaultSubobject<USceneCaptureComponent2D>("Cam Capture");
    CamCapture->SetupAttachment(Root);
    CamCapture->bCaptureOnMovement = false;
    CamCapture->bCaptureEveryFrame = false;

}

// Called when the game starts or when spawned
void AReplayCrowdSystem::BeginPlay()
{
	Super::BeginPlay();
    //read trajectory file
    max_timeslot = ReadTrajectoryFile(DefaultFileName);
    UE_LOG(LogTemp, Warning, TEXT("max_timeslot:%d"), max_timeslot);

    //init camera class
    //FString CameraBlueprintPath = TEXT("/Script/Engine.Blueprint'/Game/CrowdSpawnSystem/4KCamera1.4KCamera1_C'");
    FString CameraBlueprintPath = TEXT("/Script/Engine.Blueprint'/Game/CrowdSpawnSystem/Cameras/CameraCapture.CameraCapture_C'");
    UClass* CameraBlueprintClass = LoadClass<ACameraActor>(nullptr, *CameraBlueprintPath);

    //init camera locations
    CameraLocations = { {2590.0,4190.0,5000.0} , { 1500.0,1500.0,5000.0 }, { 1500.0,4000.0,5000.0 }, { 1500.0,6500.0,5000.0 }, { 3500.0,1500.0,5000.0 },
                        {3500.0,4000.0,5000.0},{3500.0,6500.0,5000.0} };

    //spawn cameras
    FActorSpawnParameters spawnParams1;
    spawnParams1.SpawnCollisionHandlingOverride = ESpawnActorCollisionHandlingMethod::AdjustIfPossibleButAlwaysSpawn;
    for (const FVector& location : CameraLocations) {
        AMyCameraActor* CameraToUse1 = GetWorld()->SpawnActor<AMyCameraActor>(CameraBlueprintClass, location, { -90.0,0.0,0.0 }, spawnParams1);
        Cameras.Add(CameraToUse1);
    }
    
    //TakeScreenshot(CameraToUse1);

    //CaptureActor1 = GetWorld()->SpawnActor<ACaptureActor>(ACaptureActor::StaticClass(), GetActorTransform(), spawnParams1);
    
    //replay trajectory and take pics
    GetWorld()->GetTimerManager().SetTimer(Timer, this, &AReplayCrowdSystem::ReplayAllTimeslots, 1.0, true, 0.1f);
    //for (int i = 1; i <= max_timeslot; i++) {
    //    /*for (int j = 0; j < CharactersInTimeSlot.Num(); j++) {
    //        CharactersInTimeSlot[j]->Destroy();
    //    }*/
    //    CharactersInTimeSlot.Empty();
    //    int next = ReplayTimeslot(i);
    //    UE_LOG(LogTemp, Warning, TEXT("next_timeslot:%d"), next);
    //}
    
    
    //InitCapture();

    
}

// Called every frame
void AReplayCrowdSystem::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);

}

// Called to bind functionality to input
void AReplayCrowdSystem::SetupPlayerInputComponent(UInputComponent* PlayerInputComponent)
{
	Super::SetupPlayerInputComponent(PlayerInputComponent);

}

int AReplayCrowdSystem::ReadTrajectoryFile(const FString& FilePath)
{   
    int max_temp = 0;
    if (FPlatformFileManager::Get().GetPlatformFile().FileExists(*FilePath)) {
	    TArray<FString> CSVRows;
	    FFileHelper::LoadFileToStringArray(CSVRows, *FilePath);
        for (const FString& CSVRow : CSVRows)
        {
            TArray<FString> CSVColumns;
            CSVRow.ParseIntoArray(CSVColumns, TEXT(","), true);

            // add trajectory to CrowdTrajectoryArray
            if (CSVColumns.Num() == 6)//
            {
                FCharacterTrajectoryInfo NewStruct;

                // Parse and assign values from the CSV columns to FCharacterTrajectoryInfo struct
                NewStruct.timeslot = FCString::Atoi(*CSVColumns[0]);
                NewStruct.ID = FCString::Atoi(*CSVColumns[1]);
                NewStruct.Location.InitFromString(*CSVColumns[2]);
                NewStruct.Rotation.InitFromString("P=0.0 Y=" + CSVColumns[3] + " R=0.0");
                NewStruct.VelocityValue = FCString::Atof(*CSVColumns[4]);
                NewStruct.BP_Name = *CSVColumns[5];

                // Add the struct to the array
                CrowdTrajectoryArray.Add(NewStruct);
            }

            //find maximum value of timeslot
            //ensure that timeslot is at column[0]!!!!!
            int timeslot_temp = FCString::Atoi(*CSVColumns[0]);
            max_temp = FMath::Max(max_temp, timeslot_temp);
        }
    }
    else {
        UE_LOG(LogTemp, Warning, TEXT("File doesn't exist!!!"));
    }
    return max_temp;
}

int AReplayCrowdSystem::ReplayTimeslot(int timeslot, int startIndex)
{
    int NextTimeslot_StartIndex = startIndex;
    FActorSpawnParameters spawnParams;
    spawnParams.SpawnCollisionHandlingOverride = ESpawnActorCollisionHandlingMethod::AdjustIfPossibleButAlwaysSpawn;
    for (int i = startIndex; i < CrowdTrajectoryArray.Num(); i++) {
        if (CrowdTrajectoryArray[i].timeslot == timeslot) {
            //replay
            //characterBPToSpawn = characterBPToSpawn;//TODO
            //UE_LOG(LogTemp, Warning, TEXT("Rotation: %s"), *CrowdTrajectoryArray[i].Rotation.ToString());
            ACharacter* Character = GetWorld()->SpawnActor<ACharacter>(characterBPToSpawn, CrowdTrajectoryArray[i].Location, CrowdTrajectoryArray[i].Rotation, spawnParams);
            if (Character) {
                CharactersInTimeSlot.Add(Character);
                if (Character->GetMesh()) {
                    Character->GetMesh()->PlayAnimation(Anim, true);
                    //Character->GetMesh()->SetAnimInstanceClass(nullptr);
                }
            }
            NextTimeslot_StartIndex = i + 1;
        }
        //TODO if(CrowdTrajectoryArray[i].timeslot != timeslot)
    }

    return NextTimeslot_StartIndex;
}

void AReplayCrowdSystem::ReplayAllTimeslots()
{
    for (int j = 0; j < CharactersInTimeSlot.Num(); j++) {
        CharactersInTimeSlot[j]->Destroy();
    }
    CharactersInTimeSlot.Empty();
    int next = ReplayTimeslot(current_timeslot);

    
    //take pics
    NumberOfImagesSaved = 0;
    for (int i = 0; i < Cameras.Num();i++) {
        if(Cameras[i])
            TakeScreenshot(i);
    }
    /*for (int i = 0; i < CameraLocations.Num(); i++) {
        CaptureActor1->TakeScreenshot(i, CameraLocations[i], current_timeslot);
    }*/
    
    /*if (Cameras[0])
        TakeScreenshot(Cameras[0]);*/

    UE_LOG(LogTemp, Warning, TEXT("next_timeslot:%d"), next);
    current_timeslot++;
    if (current_timeslot > max_timeslot) {
        GetWorld()->GetTimerManager().ClearTimer(Timer);
    }
}

void AReplayCrowdSystem::TakeScreenshot(int CameraIndex)
{
    //if (Cameras[CameraIndex])
    //{

    //    FString FilePath = FPaths::ProjectContentDir() + TEXT("/Pic")+ FString::Printf(TEXT("%d"), CameraIndex)+ TEXT("/CapturedImage")+
    //                        FString::Printf(TEXT("%d"), current_timeslot) + TEXT(".png");
    //    Cameras[CameraIndex]->SavePath = FilePath;
    //    UE_LOG(LogTemp, Warning, TEXT("camera location:%s"), *Cameras[CameraIndex]->GetActorLocation().ToString());
    //    Cameras[CameraIndex]->SavePic();
    //    if(CameraIndex>0)
    //        UE_LOG(LogTemp, Warning, TEXT("!!!IS EQUAL:%s"), (Cameras[CameraIndex]->rt1== Cameras[CameraIndex-1]->rt1)? TEXT("true") : TEXT("false"));

    //    //APlayerController* PlayerController = GetWorld()->GetFirstPlayerController();
    //    //AActor* PreviousViewTarget = PlayerController->GetViewTarget();
    //    //// 切换视角到摄像机
    //    //PlayerController->SetViewTarget(CameraToUse);

    //    //// 设置截图分辨率
    //    ////PlayerController->GetLocalPlayer()->ViewportClient->Viewport->SetViewportSize(FIntPoint(Width, Height));

    //    //// 执行高分辨率截图
    //    //PlayerController->GetLocalPlayer()->ViewportClient->Viewport->TakeHighResScreenShot();

    //    //PlayerController->SetViewTarget(PreviousViewTarget);
    //}

    if (Cameras[CameraIndex])
    {
        //InitCapture();

        //FString FilePath = FPaths::ProjectContentDir() + RelativeDirectory;
        //FString FileName = Prefix + FString::FromInt(CameraIndex) + ".png";
        FString FilePath = FPaths::ProjectContentDir() + TEXT("/Pic/Pic") + FString::Printf(TEXT("%d"), CameraIndex) + TEXT("/CapturedImage") +
            FString::Printf(TEXT("%d"), current_timeslot) + TEXT(".png");

        //UE_LOG(LogTemp, Warning, TEXT("camera location:%s"), *Cameras[CameraIndex]->GetActorLocation().ToString());

        if (CamCapture)
        {
            UE_LOG(LogTemp, Warning, TEXT("camera location:%s"), *Cameras[CameraIndex]->GetActorLocation().ToString());
            SetActorLocation(Cameras[CameraIndex]->GetActorLocation());
            SetActorRotation(Cameras[CameraIndex]->GetActorRotation());
            FMinimalViewInfo viewInfo;
            Cam->GetCameraView(0, viewInfo);
            CamCapture->SetCameraView(viewInfo);
            RT->RenderTargetFormat = TargetFormat;
            CamCapture->bAlwaysPersistRenderingState = true;
            CamCapture->CaptureSource = SCS_FinalColorLDR;
            CamCapture->TextureTarget = RT;
            CamCapture->SetCameraView(viewInfo);
            CamCapture->CaptureScene();
            RT->UpdateResourceImmediate(false);
            //UKismetRenderingLibrary::ExportRenderTarget(this, RT, FilePath, FileName);

            FImageWriteOptions Options;
            Options.bAsync = true;//false;
            Options.bOverwriteFile = true;
            Options.Format = EDesiredImageFormat::PNG;
            Options.OnComplete.BindUFunction(this, "CompleteImage");
            Options.CompressionQuality = 0;

            //UImageWriteBlueprintLibrary::ExportToDisk(RT, FilePath + "/" + FileName, Options);
            UImageWriteBlueprintLibrary::ExportToDisk(RT, FilePath, Options);
            RT->UpdateResourceImmediate(true);
        }
    }
}

void AReplayCrowdSystem::CompleteImage()
{
    NumberOfImagesSaved++;
    /*if (Debug == true) {
        if (GEngine) {
            GEngine->AddOnScreenDebugMessage(-1, 10, FColor::Cyan, "Saved [" + FString::FromInt(NumberOfImagesSaved) + "]");
        }

        if (NumberOfImagesSaved == Cameras.Num()) {
            if (GEngine) {
                GEngine->AddOnScreenDebugMessage(-1, 10, FColor::Green, "All Saved");
            }
        }
    }*/
}

void AReplayCrowdSystem::InitCapture() {
    if (CamCapture == nullptr) {
        CamCapture = NewObject<USceneCaptureComponent2D>(this);
        CamCapture->RegisterComponent();
        CamCapture->bCaptureOnMovement = false;
        CamCapture->bCaptureEveryFrame = false;
        CamCapture->DetailMode = EDetailMode::DM_Epic;
    }
    if (RT == nullptr) {
        RT = UKismetRenderingLibrary::CreateRenderTarget2D(GetWorld(), RTSize.X, RTSize.Y);
    }
}

ReplayCrowdSystem.zip (5.1 KB)
Trajectory.csv (1.8 MB)

3dRaven · December 1, 2023, 10:52am

Tried to compile the project but it seems to be missing some key files. I narrowed it down to the following files.
missing

I was able to recreate the Trajectory info and tried recreating the classes as best I could.

Got the cameras to spawn on play but missing crowds.

edit: Ok getting the characters to spawn still needs work

ThanosSteve · December 1, 2023, 1:05pm

sorry for that. I’ll try to give you a full version later. Thanks a lot.

3dRaven · December 1, 2023, 1:38pm

Got them loaded and walking.

ThanosSteve · December 1, 2023, 1:54pm

that is great!! I now give you my project:
PopulationSystemFullPack.zip (10.8 MB)

ThanosSteve · December 1, 2023, 2:09pm

By the way, I want to take high res photos and now the pics’ size is too large(like 7MB). I want it decrease to about 200-250KB when save it. But how?
I use this code before and it did take about 220KB:

bool UMyBlueprintFunctionLibrary::SaveRenderTargetToFile(UTextureRenderTarget2D* rt, const FString& fileDestination)
{
    FTextureRenderTargetResource* rtResource = rt->GameThread_GetRenderTargetResource();
    //FTextureRenderTargetResource* rtResource = static_cast<FTextureRenderTargetResource*>(rt->GetResource());
    FReadSurfaceDataFlags readPixelFlags(RCM_UNorm);

    TArray<FColor> outBMP;
    outBMP.AddUninitialized(rt->GetSurfaceWidth() * rt->GetSurfaceHeight());
    rtResource->ReadPixels(outBMP, readPixelFlags);
    for (FColor& color : outBMP)
    {
        color.A = 255;
    }



    FIntPoint destSize(rt->GetSurfaceWidth(), rt->GetSurfaceHeight());
    TArray<uint8> CompressedBitmap;
    FImageUtils::CompressImageArray(destSize.X, destSize.Y, outBMP, CompressedBitmap);
    //FString FilePath = FPaths::ProjectContentDir() + TEXT("CapturedImage.png");
    bool imageSavedOk = FFileHelper::SaveArrayToFile(CompressedBitmap, *fileDestination);
	return imageSavedOk;
}

3dRaven · December 1, 2023, 2:23pm

You can use the same code. It works fine and as you mentioned saves lower file sizes.

ThanosSteve · December 1, 2023, 2:26pm

ok, thanks!! and did you notice that characters in the pics saved in my project do not play animation? I guess the camera takes photos at the wrong time or sth?

3dRaven · December 1, 2023, 2:29pm

It might be due to when you are doing the snapshots. Perhaps it needs to be after the skinning update.

Ok looking into your code you are destroying and recreating the characters per ReplayTimeslot. Wouldn’t a pooling system be better to not hammer the cpu / ram?

ThanosSteve · December 1, 2023, 2:37pm

That is a great suggestion, what is the meaning of a pooling system? Like I do not destory the characters and just change their transform ?

3dRaven · December 1, 2023, 2:39pm

Exactly. I’m doing a test now on the file. I’ll keep you updated.

ThanosSteve · December 1, 2023, 2:41pm

Thanks a lot for helping with my crowd simulator, I’ve learned a lot!

3dRaven · December 1, 2023, 2:48pm

After adding in the pooling system they animate fine. Just getting a GPU crash after a longer capture period.

SaveRenderTargetToFile might be not freeing up memory somewhere causing the VRAM to fill up.

ThanosSteve · December 1, 2023, 2:51pm

How to create a pooling system? there may be different number of characers at each timeslot, but the number is in range(like[90,100]).

3dRaven · December 1, 2023, 2:52pm

I’ve already created it, I will send you file once I track down the GPU error.

ThanosSteve · December 1, 2023, 2:58pm

Thanks a lot, and it seems you are very good at C++ and UE. How long did it take you to learn everything? Now i am doing this crowd simulator project because my supervisor and me are working on a UAV-view crowd dataset, and i found some interest in game development after I got to know Unreal Engine.

3dRaven · December 1, 2023, 3:02pm

I’ve been coding over 20 years, but Unreal c++ only since around 2015-2016.

Maybe

FTextureRenderTargetResource* rtResource = rt->GameThread_GetRenderTargetResource();

is not freeing it’s resources.

Changed

FImageUtils::CompressImageArray(destSize.X, destSize.Y, outBMP, CompressedBitmap);

to

FImageUtils::PNGCompressImageArray(destSize.X, destSize.Y, outBMP, CompressedBitmap);

as the previous function is deprecated.
and switched CompressedBitmap to TArray64<uint8>