Why it matters: Advanced AI capabilities generally require massive cloud-hosted models with billions or even trillions of parameters. But Microsoft is challenging that with the Phi-3 Mini, a pint-sized AI powerhouse that can run on your phone or laptop while delivering performance rivaling some of the biggest language models out there.
Weighing in at just 3.8 billion parameters, Phi-3 Mini is the first of three compact new AI models Microsoft has in the works. It may be small, but Microsoft claims this little overachiever can punch way above its weight class, producing responses close to what you'd get from a model 10 times its size.
The tech giant plans to follow Mini up with Phi-3 Small (7 billion parameters) and Phi-3 Medium (14 billion) later on. But even the 3.8 billion-parameter Mini is shaping up to be a major player, according to Microsoft's numbers.
Those numbers show Phi-3 Mini holding its own against heavyweights like the 175+ billion parameter GPT-3.5 that powers the free ChatGPT, as well as French AI company Mistral's Mixtral 8x7B model. It's not bad at all for a model compact enough to run locally with no cloud connection required.
So how exactly is size measured when it comes to AI language models? It all comes down to parameters - the numerical values in a neural network that determine how it processes and generates text. More parameters generally mean a smarter understanding of your queries, but also increased computational demands. However, this isn't always the case, as OpenAI CEO Sam Altman explained.
While behemoth models like OpenAI's GPT-4 and Anthropic's Claude 3 Opus are rumored to pack several hundred billion parameters, Phi-3 Mini maxes out at just 3.8 billion. Yet Microsoft's researchers managed to get amazing results through an innovative approach to refining the training data itself.
By focusing the relatively tiny 3.8 billion parameter model on an extremely curated dataset of high-quality web content and synthetically generated material evolved from previous Phi models, they gave Phi-3 Mini outsize skills for its lean stature. It can handle up to 4,000 tokens of context at a time, with a special 128k token version also available.
"Because it's reading from textbook-like material, from quality documents that explain things very, very well, you make the task of the language model to read and understand this material much easier," explains Microsoft.
The implications could be huge. If tiny AI models like Phi-3 Mini really can deliver performance competitive with today's billion-plus parameter behemoths, we may be able to leave the energy-guzzling cloud AI farms behind for everyday tasks.
Microsoft has already made the model available to put through its paces on Azure cloud, as well as via open-source AI model hosts Hugging Face and Ollama.