I would not advice you quitting, yes you can self-teach you everything, I have no doubt of that, but school/unit/masters etc are not only there for you to learn stuff they are also there to teach you how to learn. If you want to be a professional games programmer you will see that most of the offers require a computer science, math, physics degree or an equivalent, is not there because companies thing that you can only learn stuff from the uni but that you where able to achieve a hard goal. There is many competition out there and a way to filter candidates is using your education. You can argue with it, that is your right, but if I get 100 CVs I have to filter fast, and educational aspects are a very good candidate for it.
See your education as a goal, do it well and you will find many doors open instead of closed. Apart from your education you also have to do your own stuff, the competition is very high in the industry!
Cheers,
Moss