UE3 did support vertex lighting on instanced geometry and foliage mode.
I don’t really see why vertex lighting would be more incompatible than lightmaps when it comes to instancing. I mean if you don’t store vertex color for each instance you still have to store unique UVs for the lightmap right ?