So if I understand what you’re saying, you have actors that have a fixed location in an infinite world, so presumably an infinite number of actors.
PNG still works for this, but you must build the areas that are constrained first. So you start by building the player’s location, then you build the locations of nearest actors first. Finally you connect them using randomly generated structures so the player can get to all the actors.
When the player is moving around the world, you only want to build new area in the direction the player is moving. If you try to do more, your game will be too slow. You may also need to collect and store old regions to reduce memory useage as the play may never return there. If he does, you have to recreate the old geometry from the stored data.