Thor fought him evenly without his hammer. Yet subsequently he has used his hammer and done no better. Doesn't make any sense does it?
The Hulk usually gets madder to compensate for the hammer. Thor's battle with Red Norvell in the 70's shows that Thor was taken out by his own hammer, if memory serves. The Red Norvell story was Thor's parallel to the Abomination story. A similiar antagonist beats the hero and takes off with the heroes woman in the end.
I'd like to see a Hulk/Thor battle where the Hulk has the axe of Terrax, just for a change of pace.
The Hulk/Thor clash in DEFENDERS seemed to ignore the Hulk's power to get stronger as he gets madder.