Changing the render device

Is it not possible to switch render devices after the engine loads?

Eg. this doesnt appear to work:

set Engine.Engine RenderDevice Engine.NullRenderDevice

Perhaps I am doing it wrong?