Phi-4
Phi-4 is a 14B-parameter open-weight language model from Microsoft Research, released on December 12, 2024 under the MIT licence. It takes text input and is intended mainly for English.
Released
December 12, 2024
Type
llm
License
open-weight
MIT
Context
16,000 tokens
Listed as 16K tokens on Microsoft’s model card.
Benchmarks
| MMLUMicrosoft’s figure, run with OpenAI’s simple-evals. | 84.8% |
| GPQAMicrosoft’s figure, run with OpenAI’s simple-evals. | 56.1% |
| MATHMicrosoft’s figure, run with OpenAI’s simple-evals. | 80.4% |
| HumanEvalMicrosoft’s figure, run with OpenAI’s simple-evals. | 82.6% |
Scores are the developer's own published figures for this model, under the test conditions noted. Benchmarks with no primary source are not listed.
Capabilities
Architecture
Parameters: 14B
Sources
Every figure on this page was checked against these pages on . A figure we could not source is left out.
Spotted something out of date? Tell us.
Links
In the news: Phi-4 and Microsoft Research
MarkTechPost
Microsoft Open-Sources Agensh: A Multi-Agent Harness for 1,024 Agents
Microsoft Research has open-sourced Agensh, a decentralized harness designed for coordinating large teams of coding agents. Unlike most agent frameworks that rely on a central orchestrator, Agensh allows each worker agent to independently select its own sub-task, share its findings, and merge code into a common Git repository. The Agensh multi-agent harness was tested with setups ranging from 1 to 1,024 agents. Each worker agent has an input limit of 272,000 tokens and an output limit of 128,000 tokens. The system runs on Linux with Docker Compose, defaulting to a single node with 8 workers, though the 1,024-agent configuration utilized 16 nodes, each running 64 agents. Workers are subject to a 64 GiB memory ceiling. Performance evaluations on the ProgramBench benchmark indicated that increasing the number of agents improved scores on all five of the hardest tasks and accelerated their completion. Specifically, a 79.59% pandoc pass rate was achieved with 32 agents on GPT-6.1-sol, while GPT-5.6-sol reached 55.06% with 1,024 agents. However, the mean score across the five tasks only reached 28.78% at 128 agents. The transition from 1 to 8 agents resulted in a modest 1.37-point increase. Agensh addresses the limitations of orchestrator-worker designs, where a lead agent handles all planning, assignment, and merging. Agensh enables workers to manage this coordination themselves through a concurrent and asynchronous 5-step loop. This loop involves gathering context, claiming a sub-task, taking action with tools, verifying results against acceptance criteria, and merging progress into a shared repository.
OpenAI
On the Navier–Stokes Millennium Prize Problem
An artificial intelligence system has generated a solution to the Navier-Stokes Millennium Prize Problem, a long-standing challenge in mathematics and physics. The AI's output includes a detailed writeup of the solution and a formal proof formalized in the Lean theorem prover. This development marks a significant moment in the application of AI to complex scientific and mathematical problems, potentially accelerating research and discovery in fields reliant on fluid dynamics. The Navier-Stokes equations describe the motion of viscous fluid substances, such as air, water, and blood. They are fundamental to understanding a vast array of phenomena, from weather patterns and ocean currents to the aerodynamics of aircraft and the flow of blood in arteries. Despite their importance, proving the existence and smoothness of solutions to these equations in three dimensions has remained an unsolved problem for over a century. The Clay Mathematics Institute designated it as one of seven Millennium Prize Problems in 2000, offering a $1 million prize for a correct solution. The AI's contribution, as detailed in its accompanying writeup, aims to address the existence and smoothness of solutions. Formalizing the proof in Lean is a critical step, as it allows for rigorous verification by human mathematicians. Lean is an interactive theorem prover and programming language developed by Leonardo de Moura at Microsoft Research, designed to assist in the formal verification of mathematical proofs and software. Its use signifies a commitment to mathematical rigor and allows the broader scientific community to scrutinize the AI's findings. While the AI has produced the solution and proof, its validity is now subject to review by experts in the field. The process of peer review for such a complex mathematical problem, especially one generated by an AI, is expected to be thorough and potentially lengthy. The implications of a confirmed AI-generated solution could be profound, not only for fluid dynamics but also for the broader landscape of AI-assisted scientific research. It could pave the way for AI to tackle other grand challenges in mathematics and physics, potentially leading to breakthroughs that have eluded human researchers for decades. The Clay Mathematics Institute will ultimately determine if the provided solution meets the criteria for the Millennium Prize.