Lower FPS on Steam, Varied network speeds.

I am testing an online multiplayer game through the Steam Dev AppID 480.
My FPS drops about 50-60 when running a packaged game.
My game has WMV videos that activate at certain times and the Frame/GPU speeds vary every time I start a new Server. Sometimes it’s amazing and really fast, other times it drops my FPS to 15 for a split second.

Got any advice for improving network speeds on my end?
I turned off Smooth Frame Rate and have Non-Standard Codecs and Low Latency Enabled on the WMV settings.