In brief: Small language models are generally more compact and efficient than LLMs, as they are designed to run on local hardware or edge devices. Microsoft is now bringing yet another SLM to Windows 11, as users apparently need a few AI-powered hints to help them find specific OS settings and customize their PC experience.
Microsoft recently announced Mu, a new small language model designed to integrate with the Windows 11 UI experience. Mu will work alongside Phi Silica – the language model previously introduced on Copilot+ PCs – to power the AI agent feature in Windows Settings that the company is currently developing.
Like Phi Silica, Mu will run entirely on the system's neural processing unit, delivering responses at over 100 tokens per second, which should provide a smooth user experience in Settings.
Microsoft describes Mu as an efficient 330-million-parameter model optimized for small-scale deployments. It is based on a transformer encoder-decoder architecture, where the encoder converts user input into a fixed-length latent representation, and the decoder generates output tokens from that representation.
Microsoft designed Mu as a highly optimized "sibling" to Phi Silica, using several techniques to reduce the parameter count and improve efficiency. The SLM was trained using an unspecified number of A100 GPUs on the Azure Machine Learning platform, progressing through multiple phases and applying techniques that Redmond developed during work on the Phi models.
After fine-tuning the model, the developers found that Mu delivers performance nearly comparable to Phi-3.5-mini, despite being just one-tenth its size. Furthermore, Microsoft collaborated with silicon partners Intel, AMD, and Qualcomm to ensure that Mu is fully optimized for running on local NPU-powered machines.
What was the point of all these optimization efforts? According to Redmond, Mu will power the new "agentic" experience in the upcoming AI-enhanced Settings app. The AI and cloud giant aims to provide a new "chat" feature that lets Windows users ask settings-related questions in natural language and receive effective answers, even when they barely know what they're looking for.
Agentic AI in the Settings app is supposed to improve Windows' ease of use. Microsoft acknowledged that training an SLM capable of handling (and modifying) hundreds of system settings was a challenge, and the company says it welcomes user feedback on the new feature.
Personally, I never experienced any usability or discoverability issues with Settings during eight years of using Windows 10, yet Microsoft remains bullish about transforming Windows into an "AI-first" platform.
Meet Mu, the small language model in charge of Microsoft's Settings AI agent
”
