How to SpaceShip physics?

I’m trying to make a inhabitable spacecraft that is not derived from vehicle class. Think of it as a moving platform.

It is not just one mesh but a collection of 6 different actors forming attached together as child actor components to form a simple box (I wanted walls destructible etc).

Now there are two ways to move it around one is Using SetLocation and other is using PhysX simulation.

The problem with the first one is if you jump while the box is moving you’ll go through the back wall since set location does not check collision unless you enable sweep. Even with sweep enabled you’ll only get hit for the root component of the main actor and collision with everything else will be ignored. FAIL

In the second way using PhysX all the meshes are welded together and moved as a single body using add impulse, However the problem with this approach is that the character is not simulating physics (obviously) so for the PhysX system treats the character as infinite mass body.
So if now you jump and hit the rear wall, it sends the spaceship box spinning crazy.FAIL

What should be proper approach for making a vehicle like object that you can also inhabit?