Remember Dexter? If the comic is used as source material there are a group of ex-con's holed up in the Cafeteria of that prison. There's a sex offender that ends up slicing Andrea's face and there's a character named Dexter who is actually the first human Rick kills in the comic book in cold blood.
There's also a bearded guy who's name escapes me at the moment but he stays with the group until the Governor's massacre of the prison in issue #48.
Seeing how the creators of this series only follow the comic plotline loosely anyone want to venture a guess on who's in that prison right now?