It is not quite as you describe - it is merging static meshes into one, so you have to begin with several SMs. However yes it does so the second part: generating a new mesh from the pieces, creating a new UV Map, and baking the texture atlas for all of them as one.
Essentially the idea is that you can optimize your game by rendering a single mesh with one material instead of several meshes each with multiple materials. E.g., if you have a table covered in plates and food, you could in theory arrange them how you want in your level and then merge all of them into a single Mesh with one texture set.
You can find more on Actor Merging in the documentation here: Merging Actors in Unreal Engine | Unreal Engine 5.1 Documentation
In my experience, simplify works best for non-Nanite SMs.