SetHiddenInGame takes effect on the next playthrough

I’m adding a UStaticMeshComponent to my actor:

Roof = CreateDefaultSubobject<UStaticMeshComponent>(TEXT("RoofComponent"));
Roof->AttachToComponent(RootComponent, FAttachmentTransformRules::KeepWorldTransform, NAME_None);

Then at some point in my game I call the following function from a blueprint:

// .h
UFUNCTION(BlueprintCallable, Category = "Visibility")
void ToggleRoofVisibility(bool visibility);

// .cpp
void AStore::ToggleRoofVisibility(bool visibility)
{
	Roof->SetHiddenInGame(!visibility, true);
}

This doesn’t do anything at all. However the next time I run the game, the object visibility changes (So if toggled it off during the game, it will only appear hidden the next time I run the game).
Also I noticed that if I change the “Hidden In Game” property in the editor before running the game, it works. But then when I change it in runtime, it actually changes the value in the editor as well!

I feel like I’m missing something here…