I think what got me to think in that direction was after Jenny was murdered. He had someone he could vent his anger toward (Angelous). Buffy died after he murdered Ben. There was nothing left to fight. And if you look the group around Buffy's body they either broke down completely, like Spike, or they had someone to cling to. To me, Giles eyes spoke of a devastation beyond comprehension.

Of course it is all up to the interpretation of the writer. Which is why I love reading all of the different fics out there.