Hey Timothy -
You would need to assign your Cubemaps directly in the Materials of the items themselves. You can setup a blueprint controlled volume system to change the cubemap that is used when an object is placed inside of certain volumes that you set up in your level. Here is what a basic material setup would look like:
Thank You
Eric Ketchum