Early this 12 months, Microsoft launched the Phi-3 household of small language fashions. Right this moment, Microsoft launched Phi-4, a 14B parameter state-of-the-art small language mannequin (SLM) that even beats OpenAI’s GPT-4 massive language mannequin in MATH and GPQA AI benchmarks.

Microsoft claims that Phi-4’s robust efficiency on math-related reasoning is because of the usage of high-quality artificial datasets, curation of high-quality natural knowledge, and post-training enhancements. Artificial knowledge for coaching was generated utilizing a number of methods, together with multi-agent prompting, self-revision workflows, and instruction reversal, and the generated artificial knowledge constitutes the majority of the coaching knowledge for Phi-4. Microsoft additionally used methods resembling rejection sampling to refine the mannequin’s outputs throughout the post-training course of.

Within the Phi-4 technical paper, Microsoft additionally addressed the issues across the leakage of benchmark take a look at units through the online. Microsoft has improved the information decontamination course of for Phi-4 to make sure no unfair affect on analysis outcomes. To verify this, Microsoft examined the Phi-4 mannequin on the November 2024 AMC-10 and AMC-12 math competitions, which occurred after Microsoft’s coaching knowledge was collected.

As you may discover within the picture beneath, Phi-4 outperforms each similar-size or open-weight fashions and in addition bigger frontier fashions, together with Gemini 1.5 Professional. By way of this take a look at, Microsoft claims that Phi-4’s top-tier efficiency on the MATH benchmark will not be because of overfitting or contamination.

Phi-4 additionally comes with weaknesses since it’s nonetheless basically restricted by its dimension. It can hallucinate round factual information, and it’s much less proficient at rigorously following detailed directions. For mannequin security analysis, the Phi-4 workforce labored with the unbiased AI Crimson Group (AIRT) at Microsoft to establish security and safety dangers posed by Phi-4 in each common and adversarial person situations.

Phi-4 is now accessible on Azure AI Foundry beneath a Microsoft Analysis License Settlement (MSRLA). Microsoft will even launch Phi-4 on Hugging Face subsequent week.