Under the old index, when you did a search Google would scan the various layers of its index. These layers are prioritized by importance, so it would search one group of high priority sites and then work its way down to less prioritized groups of sites. Each layer was updated on a schedule and at different rates -- the primary one was refreshed every two weeks, for example. With Caffeine, Google drops the layered architecture and instead analyzes small portions of the web and updates the index on a continuous basis. This means that recently-published content is added much sooner than before.
According to Google, Caffeine is capable of adding hundreds of thousands of pages into the Google index per second, and hundreds of thousands of gigabytes of information per day. This is the biggest change to the search engine's methodology in four years, and one that reportedly increases their ability to scale up with the rapid growth of information online.