News
Microsoft Unveils Phi 3.5-Mini-Instruct Model in Latest AI Advancement
Microsoft has announced the release of its Phi-3.5-Mini-Instruct model, the latest addition to its Phi-3 model collection. The Phi series, known for its compact AI models designed to operate on smartphones without requiring an Internet connection, has been expanded with these new offerings.
The initial Phi 3 model debuted in April; Microsoft has since been working on enhancements to the technology. The company stated that the updated models have undergone significant improvements through a rigorous enhancement process. This process included supervised fine-tuning, proximal policy optimization, and direct preference optimization, all aimed at ensuring precise instruction adherence and robust safety measures.
The new Phi 3.5 lineup includes three models: the 3.82 billion parameter Phi-3.5-Mini-Instruct for basic and fast reasoning tasks, the 41.9 billion parameter Phi-3.5-MoE-instruct for more complex reasoning, and the 4.15 billion parameter Phi-3.5-Vision-Instruct, designed specifically for image and video analysis.
Maintaining the 128K token context length of its predecessor, the Phi-3.5-Mini-Instruct is well-suited for tasks requiring extensive context, such as long document summarization and information retrieval. The model was trained on 3.4 trillion tokens using 512 H100-80G GPUs over a span of 10 days, while the Vision Instruct model was trained on 500 billion tokens using 256 A100-80G GPUs over six days.
Microsoft highlighted that the Phi 3.5 models have shown strong performance in various third-party benchmark tests, outperforming competitors like Google’s Gemini 1.5 Flash, Meta’s Llama 3.1, and OpenAI’s GPT-4o in specific scenarios. However, the company acknowledged that the smaller models are not designed to compete with larger language models in every task.
"The model simply does not have the capacity to store too much factual knowledge, therefore, users may experience factual incorrectness," Microsoft stated. The company suggested that this limitation could be mitigated by augmenting the Phi-3.5 with a search engine, particularly in Retrieval-Augmented Generation (RAG) settings.
The Phi 3.5 Mini-Instruct model is now integrated into some of Microsoft’s own services and is available through Azure AI Studio, making it accessible to a broad user base. All three models are open-sourced under the MIT license, which stipulates that the software is provided "as-is," and Microsoft is not liable for any potential damages.
Microsoft expressed its commitment to ongoing improvement of the Phi-3 model family, welcoming feedback from the community to further refine these models.