Easy explanation: Its indexing is faster and smarter
Case in point: tracking spider hits for a PR7 site with 1+ million pages:
Just compare those numbers. Yahoo! has over 5000 links to the index page – and yet it can’t even crawl 100 pages in a day?
That’s only half the story. How about the intelligence of the crawling (the following image is hits to the index page by the search engine spiders):
Yahoo! and MSN are obsessed with the frontpage – Google checks it out a few times a day.
Google’s spider isn’t just better – its far smarter.
Update: At the end of the day, Googlebot clocked in 68,162 hits, Yahoo 37 (5 to the index page), and MSN 4.
Update 2: Just broke the 100k mark.