Roger Waters wrote The Wall to describe the separation he was feeling from his audience. Pink is a mix between Syd Barret, and Roger, as Roger felt at the time that he was going to lose his mind like Syd did. The character of Pink created a Wall between him and society to protect himself from emotional pain. While the others, teachers, his mother, his wife helped him build this wall around himself, they also were hurt by the distance from him that it created. When the judge orders to tear down the wall, it's the wall of Pink's emotions, not his sanity. Also, if you listen very closely to Outside The Wall, David Gilmour, I think says, "Is This Where..." which creates the idea that Pink is continuously on a loop. He creates the wall to protect himself, then he tears it down, then he builds it back up again. The worms are simply a representation of the Waiting For The Worms motif. We're all just waiting to die, cowering behind our own walls.