Well, if you've read the comic you probably know how bad the Governor was there: sadic, mercyless, just... bad. But in the TV Show? I really don't think of him as a "villain", I mean, yeah he has killed a bunch of people but no one that makes you feel "oh how much I hate him he's so evil, I want him dead", no, in the comic (SPOILERS) he raped Michonne, cutted Rick's hand, made Lilly kill Lori and Judy, and killed a bunch of other characters (Hershel and son, Alice, etc), so you wanted to watch him dead, he was a good villain, but in the show there is not much of that, he doesn't look really bad and when Michonne killed his daughter (brother's daughter in comic) it looked like Michonne was the bad one, I mean, he didn't made too much to her and she practically attacked for little to none reason compared to the comic, and he looked really sad about killing his daughter (already dead, tough) and aboout the fishbowls with heads, they gave him a good reason to have it (because by watching them he would have less fear of what's outside in the new world and therefore making him a better leader), so it looked pussy-er in that way. So, for me, he's not much of a villain, actually, I think Rick is much evil than him (Ok, maybe not that extreme) but he's definetively not what his comic counterpart made him such a good villain, and it's, in part because in the TV you can't do everything they made on the comic (raping, killing babies, etc), but still, he's not that bad.
So what do you think? you agree with me or think different than I do? Maybe they will develope his evil part later in the series? I really don't know.