Go to the lowest performance part of your scene. When you type the console command stat unit, what do each of the numbers roughly say?
Also, is this a generally bad performance over the entire duration of play? Or is it good sometimes but then gets sudden giant hitches or freezing?