Forward-looking: Apple's push to make Siri more capable is starting to look less like a purely in-house effort and more like a concession to the realities of modern AI. To close the gap, the company is expected to split Siri's workload between on-device processing and the cloud, including Google's Gemini models.
Apple has spent years emphasizing the privacy benefits of keeping computation on-device. Its custom silicon, including the Neural Engine, has been steadily tuned for machine learning workloads. But even with those gains, phones remain limited by memory and processing ceilings. The largest AI models now operate at a scale that simply doesn't fit within those constraints.
Smaller models designed for local use can help, but they come with trade-offs. On-device systems typically run with only a few billion parameters and are often compressed using techniques like quantization to improve speed and efficiency. That makes them usable on a phone, but it also reduces accuracy and depth. In practice, they tend to feel less capable than their cloud-based counterparts, especially in open-ended conversations.
That gap is part of what Apple is now trying to bridge.
After striking a deal with Google, Apple reportedly began working on distilling Gemini's larger models into smaller versions that could run on the iPhone. Distillation allows a compact model to mimic the behavior of a much larger one, capturing useful patterns without the full computational load. It's a way to bring some level of advanced AI onto the device, even if it cannot match the original model's performance.
There are limits to how far that approach can go. Google itself doesn't attempt to run its full conversational Gemini experience locally on Android. Instead, those interactions are routed to the cloud, where far more powerful hardware can handle them.
Apple appears to be heading in a similar direction, even if it frames the experience differently.
According to The Information, more complex Siri requests will likely be processed off-device, potentially using Google's infrastructure. At the same time, Apple is working to maintain some control over how that data is handled. The company has reportedly partnered with Nvidia to use its Confidential Computing platform, which keeps data encrypted even while it is being processed on cloud GPUs.
That setup could allow Apple to continue emphasizing privacy, even as more user data leaves the device. Whether users notice the difference may come down to performance. Cloud-based AI systems, especially those running with added encryption layers, can introduce latency. In contrast, simpler on-device tasks should feel faster and more immediate.
Apple is unlikely to surface those distinctions directly. Like other companies building hybrid AI systems, it is expected to present the experience as seamless, with requests automatically routed based on what the system determines is most efficient.
Under the hood, though, the divide will remain.
The broader issue is not unique to Apple. Across the industry, there is a growing gap between what edge devices can handle and what cutting-edge AI models require. Even as mobile chips improve, the most advanced systems still depend on massive infrastructure – clusters of GPUs and specialized hardware far beyond what any phone can support.
Apple's evolving approach to Siri suggests that, for now, there is no clean way around that limitation. Keeping everything on-device may be ideal in theory, but in practice, delivering a competitive AI assistant increasingly means leaning on the cloud.
Apple's on-device AI dream is running into the hard limits of what a phone can do
