I’d like to make a ton of floating lanterns with flames inside flying upward. New lantern actors will continuously spawn so this effect goes on forever. Is this something that Niagara could do, or is blueprints going be a better approach to tackle this?
It depends, as @EliasWick said it can be done either way. But I would suggest you to go for Niagara. It will allow you to set their bounds, keep it looping, and its as easy as just setting a mesh in Niagara emitter.
You can simply use 2 emitters in a single system. One that spawns lanterns, and one that spawns fire on lanterns using local space.