I'm gonna have to totally disagree with you, Jim. Prisoner of Azkaban was done well in that it had to fit such a vast story into 2 hours. You really can't expect a director to fit a book as large as POA into 2 hours. When you make a movie 3 hours or longer, you risk losing your audience. Not many people are going to want to see 4 hours of Harry Potter. While Harry Potter obviously has a large core audience, the producers still want to tap into the larger mainstream market. In the end, this is about money, and what surprises me is that people didn't complain much about all of the stuff left out of the Lord of the Rings movies, but they always give POA a hard time.
Personally, I didn't even really like the first two movies much, because they were more like kids' movies. Prisoner of Azkaban is one of the few films I've seen that successfully incorporated both kid and adult elements into a film. Another thing that I think made POA better than the first two films is that it was far darker. The cinematography was a lot more complex and brooding than the first two movies, which I think reflects how the story itself is getting darker with each book. Chris Columbus simply isn't dark enough for Order of the Phoenix; why hire a Steven Speilberg, when you need a Stanley Kubrick?