Kimi K3 Open Weights Are Live: 2.8 Trillion Parameters and the New Open Source AI Frontier

Let me be direct with you. On Sunday, July 27, Beijing-based Moonshot AI released the full open weights for Kimi K3 — a 2.8 trillion parameter mixture-of-experts model that is, by a wide margin, the largest open-source AI model ever built. The weights went live on the same day the company promised them three weeks earlier, and the community response has been appropriately intense.

I have spent the past 48 hours digging through the architecture papers, running through the benchmark tables, and reading the early deployment reports. Here is what I found — and why this release matters far beyond the parameter count.

What Kimi K3 Actually Is

Kimi K3 is an autoregressive mixture-of-experts transformer with native vision capabilities and a 1,048,576-token context window. It activates 16 out of 896 experts per forward pass, meaning roughly 50 billion parameters are active per token while the remaining 2.75 trillion sit dormant until routed. This is the first open model to reach the 3-trillion-parameter class, surpassing the previous record holder, DeepSeek V4 Pro at 1.6 trillion, by roughly 75 percent.

The model is available today through the Kimi API at $3 per million input tokens and $15 per million output tokens. You can also use it through Kimi.com, the Kimi mobile app, and Kimi Code with a free Google account — no credit card required. But the headline event is the open weights release, which makes Kimi K3 the most capable model ever placed in the hands of the open-source community.

The Architecture: Three Innovations Worth Understanding

Moonshot did not just scale up. The company introduced three architectural innovations that solve real engineering problems at this scale.

Kimi Delta Attention (KDA) is a hybrid linear attention mechanism that alternates between cheap linear-attention layers and full-attention layers in a 3:1 ratio. Three linear layers handle local sequence structure efficiently; one full-attention layer preserves global information flow. The result is approximately 90 percent less KV cache memory consumption at 1-million-token contexts, with no measurable capability tradeoff against standard full-attention baselines.

Attention Residuals (AttnRes) replace standard residual connections with a selective retrieval mechanism. Instead of uniformly accumulating layer outputs across depth, each layer can reach back and pull high-value representations from earlier layers. Moonshot reports roughly 25 percent higher training efficiency at under 2 percent additional compute cost.

Stable LatentMoE manages the 896-expert pool with techniques like Quantile Balancing, which derives expert allocation directly from router-score quantiles, and Per-Head Muon, which optimizes each attention head independently. At 1.8 percent sparsity — only 16 experts active out of 896 — standard routing strategies break down. Moonshot's fixes make extreme sparsity actually work in production.

Together, these advances yield approximately 2.5 times better scaling efficiency compared to Kimi K2, meaning the same compute budget produces a significantly smarter model.

The model also uses quantization-aware training from the supervised fine-tuning stage onward, storing weights in MXFP4 format with MXFP8 activations. This is not post-training quantization — the model learns to compensate for precision loss during training. The 4-bit weight format means the full 2.8 trillion parameter model requires approximately 1.4 terabytes of weight storage, substantially less than the 5.6 terabytes that FP16 would demand. This brings self-hosting within reach of organizations with multi-node GPU clusters.

The Benchmark Picture: Where K3 Wins and Where It Does Not

Moonshot published its own benchmark results, and the picture is more nuanced than a simple third-place finish. On several key tests, Kimi K3 beats Claude Fable 5 and GPT-5.6 Sol outright.

SWE Marathon — a benchmark designed to measure sustained, long-horizon coding work — is where K3 shines brightest. It leads the field at 42.0, a 3-point gap over GPT-5.6 Sol and 7 points over Claude Fable 5. If your use case involves writing code across entire repositories for hours at a time, this is the benchmark to watch.

Program Bench: K3 scores 77.8, edging out Fable 5 at 76.8 and GPT-5.6 Sol at 77.6.

Terminal Bench 2.1: K3 scores 88.3, competitive with Fable 5 at 84.6 and GPT-5.6 Sol at 88.8.

FrontierSWE: K3 scores 81.2, which beats GPT-5.6 Sol at 71.3 by a wide margin but trails Fable 5 at 86.6.

DeepSWE: K3 at 67.5, Fable 5 at 70.0, GPT-5.6 Sol at 73.0 — the model takes third place here.

On general intelligence benchmarks, K3 scores 1,687 on GDPval-AA v2 against Fable 5's 1,815 and GPT-5.6 Sol's 1,747.8. It scores 93.5 on GPQA-Diamond, 81.6 on MMMU-Pro, and 94.3 on MathVision — all competitive numbers that place it firmly in the frontier tier.

The 4 Questions Nobody Has Answered

1. How reproducible are these benchmarks? Moonshot published its own numbers using Kimi Code, Codex, and Claude Code harnesses. Independent third-party reproductions are still ongoing, and until someone else can verify those SWE Marathon numbers on their own hardware, some skepticism is warranted.

2. What is the real cost of self-hosting? Moonshot recommends deploying Kimi K3 on supernode configurations with 64 or more accelerators. The MXFP4 format reduces weight storage to 1.4 TB, but that still means a minimum 8-node cluster with 8x 80GB GPUs each — and that is just for loading the weights. KV-cache and activations need their own space. The practical minimum deployment cost is in the hundreds of thousands of dollars.

3. Can the 896-expert MoE be efficiently served? Standard inference frameworks like vLLM, TensorRT-LLM, and SGLang need MoE-aware scheduling patches for 896-expert routing. Moonshot contributed a KDA-compatible prefix caching implementation to the vLLM community, but production readiness across the full inference stack is not guaranteed on day one.

4. What is the actual license? The weights are released as open weights, but the exact license terms were still TBD at the time of the July 27 release. For organizations that need legal certainty before deploying, this is a material open question.

What This Means: The Open Source Frontier Just Shifted

There is a story here that goes beyond the model itself. Moonshot AI's annual recurring revenue crossed $300 million as of June 2026, up from $100 million in March and $200 million in May. The company closed a $1 billion funding round this year at a $15 billion valuation. API revenue now makes up 70 percent of total revenue, with overseas paid users growing 400 percent.

This is not a research lab publishing papers. This is a company with real revenue, real infrastructure, and a real business model choosing to open-source its best technology. The contrast with OpenAI, which continues to signal open-weight releases without shipping them, and with Anthropic, which has never released open weights for its frontier models, could not be starker.

The practical capabilities are also underappreciated. During development, an early version of Kimi K3 handled the majority of the team's kernel optimization work. The model built MiniTriton, a compact compiler with its own tile-level IR layer, from scratch — beating Triton on certain workloads. It designed a chip for a nano model built on its own architecture in a single 48-hour autonomous run. These are not benchmark numbers. These are demonstrations of what sustained, long-context agentic coding looks like in practice.

What Comes Next

The open weights are live as of July 27. The community will now do what the community does best — probe the expert routing, test the Attention Residuals mechanism on smaller scales, and figure out whether LoRA or QLoRA interacts sensibly with an already-quantized MoE model.

Moonshot has promised low- and high-thinking-effort modes in subsequent updates, which will make the model more practical for everyday use. Inference partners are working on deployment infrastructure. The technical report with full architecture and training details is expected soon.

The race for open frontier intelligence is no longer theoretical. A 2.8-trillion-parameter model with competitive benchmarks, a 1-million-token context window, and open weights is sitting on Hugging Face right now. What the community builds with it will determine whether this release becomes a genuine inflection point or just another parameter count milestone.

I will be watching the self-hosting reports and independent benchmark reproductions closely. If you are running K3 on your own hardware, I want to hear about it.

— Allan Ali, Sylt.ing