I wrap so much stuff with components. It should work great, versatile, modular, lightweight. Since 4.27 we can Add Components by Class dynamically. The POIs may not even need components until they are discovered - but that might be an overkill.
Depending on how things need to be represented, I’d experiment with extending the widget component and script the management inside. You get 2 in 1 - management and visualisation. If the scope is medium / large, it’s probably a good idea for the set / map to contain soft object refs.