On the 8th of June, Google announced the completion of a new web indexing system called Caffeine. Google Caffeine provides 50 percent fresher results for internet searches than it’s predecessor, and it’s the largest collection of internet content Google has offered. Google claims that whether it’s a news story, a blog or a forum post, you can now find links to relevant content much sooner after it is published than was ever possible before.

How Google Search works

When you search the internet using Google, you’re not searching the live web. Instead you’re searching through an index of the web created by Google which helps you find the exact information you need. (There is a great video on How Google works which will explain it much better than I have here!)

Why build Google Caffeine?

You might be wondering why Google would risk changing the current Google Search when it seemed to be working just fine? The main reason is that content on the web is being created at a far greater pace and it this does not just include textual information. With the more common additions of video, images, news and real-time updates, webpages are becoming richer and more complex. Add to this the public’s increasing expectation that Google Search will provide the latest relevant content and the expectation of publishers to be found the instant they publish information, the need for Google Caffeine was clear.

Welcome to the new search world of Google Caffeine

To keep up with the evolution of the web and to meet rising user expectations, Google has built Caffeine. The image below illustrates how Google’s old indexing system worked compared to the new and improved Google Caffeine indexing system:

Google’s old index had several layers. Each layer was refreshed at a different rate e.g. some layers were refreshed more often than others. There was a main layer that would update every couple of weeks that required Google to analyse the entire web. This meant there was a significant delay between when Google found a page and made it available to you.

With Google Caffeine, Google is able to analyse the web in small sections and update their search index on a continuous basis, world wide. As Google finds new pages, or new information on existing pages, they are able to add these straight to the index. That means the information that we are able to fine now is fresher than ever before—no matter when it was published around the world.

Caffeine allows Google to index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel from all around the world!

Is Google Caffeine Future Proof?

Only time will tell, but Google has built Caffeine with the future in mind. Not only does Google Caffeine provide fresher search results, it is also a robust foundation that makes it possible for Google to build an even faster and more comprehensive search engine that is able to scale with the growth of online information. Moving into the future, Google Caffeine will enable Google to deliver even more relevant search results to you.

Google’s old Index vs Google Caffeine!

Now there is a really cool site that allows you to see the comparison of Google’s old indexing versus Google Caffeine. There isn’t always a huge difference, but it is obvious that Google Caffeine is retuning much more relevant and fresher results overall. You are also able to see on this site that the new Google Caffeine is FAST!! Nearly every search you conduct will show you that Google Caffeine has shaved off about 20-50% or the search time from the old Google and it retrieves significantly more results. See this comparison of the search term “Barrack Obama”:

Old Google:

Google Caffeine:

In this example the Old Google returned results in 0.18 seconds and Google Caffeine returned results in 0.13 seconds. That is an improvement of 0.05 seconds (–28%) and the new Google Caffeine also returned about 3 times more results with 207 million results compared to only 67.8 million returned by the Old Google Index.

Not all changes are equal

Unfortunately not every change on every site will appear immediately. There will be factors that Google look at which will determine which sites to crawl more often. Google Caffeine will look at sites with a high page rank more frequently and also check news sites and blogs more often than other sites.

Conclusion

In my opinion, Google Caffeine has improved the relevancy, accuracy and timeliness of the search results. It is still early days yet so I will be keeping an eye out for any tweaks and updates in the months ahead.

Also, I would be very interested to hear from anyone that has been either positively or negatively affected by the release of Google Caffeine, please share your story.