For years, my knee-jerk reaction on the question favorite Harry Potter book has been Prisoner of Azkaban. But it's been so long since I've read it (haven't done a complete re-read in years) that the details have gotten fuzzy on why.
I remember now--this is when things start to come together, the tension starts to build, and Harry finally realizes he's not alone. I think this is the book that pushed the series to the next level.