Does Lollipop Truly Deliver?
With all the benchmarks now out of the way, we're getting a good look at what Android 5.0 brings to the table from a performance and battery life perspective.
Two of the devices I tested – the Galaxy S5 and Moto G 2014 – saw performance gains on the CPU side when upgrading to Android 5.0. On average that improvement was around 5%, although individual results varied from no improvement to 10%+ gains.
I would expect that anything in the 5-10% improvement range represents a best case scenario for an Android 5.0 upgrade, and unless 64-bit support was suddenly enabled on a supporting device, I'd be surprised if overall gains would be larger than that.
Google actually claimed at the launch of Android 5.0 that some benchmarks and applications would see performance improvements up to 2x from switching from Davlik to ART. Across the benchmarks that I use, which I believe test fairly real-world use cases, improvements weren't nearly as high. In some specific cases you could see gains this high, like when a JIT compiler was particularly inefficient for the code base, though I'd expect these would be few and far between in everyday usage.
The Moto X was an interesting case in that it saw no gains on the CPU side. Performance was already quite good when I tested the device running Android 4.4, and it remains so with Android 5.0, performing slightly better than other flagships with similar hardware. Theoretically we should have seen extra performance from the switch to Android 5.0, though Motorola might have decided to tweak power management to keep performance flat. The device already gets quite hot under load, so carefully managing thermals might have been a priority for Motorola (and with good reason).
The LG G3 saw performance reduced across the board by ~4%. Although LG hasn't officially confirmed this, I suspect the reduction is due to a change in SoC thermal management aimed at providing better sustained performance. The G3 did thermally throttle under some conditions, and the effects of this seem to have been reduced after the update to Android 5.0. Although performance reductions are never ideal, if it's to achieve better sustained performance in real use cases (e.g. games), it's probably for the better.
As for GPU performance, unsurprisingly we saw no gains on any of the devices we tested. Android 5.0 wasn't designed for improving GPU efficiency, as ART focuses more on CPU-specific optimizations. While some GPU-limited applications saw slight jumps, this would be from CPU optimizations only.
Battery Life Results
Battery performance remains mostly the same after upgrading to Android 5.0. Unless you have the LG G3, which saw some unusual reductions in battery life, you should expect roughly the same life from your device during video playback and web browsing. Intermittent high performance workloads are the only areas you can expect to see any real gains; I recorded 20% gains on average in PCMark for Android's battery benchmark.
At the end of the day it's hard to say exactly what improvements you can expect from an Android 5.0 update as OEMs may choose to tweak many aspects of performance at the same time. However, looking purely at Android 5.0-related gains, a performance improvement of 5-10% on the CPU side - with better battery life in performance intensive tasks to go with it - would be a good result.
As developers begin to harness the new battery APIs in Android 5.0, and as we start to see more 64-bit capable devices on the market, I'd expect both performance and battery results to improve again purely from enhanced software. And even though gains are relatively small today, they're still pretty impressive considering nothing has been altered on the hardware side.
As for what the future holds, it's unclear how much more efficient you can make Android, and whether we'll see similar gains from a new major revision to the operating system. There are probably still tweaks to be done here and there, but the switch to ART with improved garbage collection will likely be one of the biggest performance-focused updates to Android for a few years, aside from continual improvements on the low end.
Most importantly, I hope we start to see more manufacturers roll out Android 5.0 as soon as possible. While four of the handsets I had on hand happened to receive Android 5.0 updates (after weeks of waiting) not all devices get this treatment, and especially products that aren't flagships. Choosing an OEM with a good update record, like Motorola, can be key to getting the latest software as quickly as possible, even on budget devices.