CommonActivatableContainers have built-in widget pools for CommonActivatable widgets. OnInitialized is called only when widget is created, but not when it is taken from the pool. In Lyra owner of all UI elements is UPrimaryGameLayout widget, which is owned by UGameUILayout object, which is owned by UGameUIManagerSubsystem, which is owned by UGameInstance. In short, hierarchy is UGameInstance → UGameUIManagerSubsystem → UGameUILayout → UPrimaryGameLayout → UCommonActivatableContainers → UCommonActivatables. (all of this this is in CommonGame plugin, btw). Having UGameInstance as owner means objects are never deleted. So your CommonActivatableContainers exist until you close the game, and all pools and all widgets in those pools also exist. You MainMenu widget still exists after 2 hours of wandering inside your world, HUD exists if you go back to MainMenu, all pop-up windows (if you are using Modal stack in UILayout widget) exist until you close the game.
That is why OnInitialized is never called again.