Moonshot released Kimi K3 model weights and technical report
Source:https://mp.weixin.qq.com/s/tryHe81IyM6nr0fBPDz72g
Kimi K3 Open Day
Thank you for your patience.
Today is Kimi K3 Open Day. We are releasing the Kimi K3 model weights, the technical report, and the key infrastructure technologies that support Kimi K3 model training: MoonEP, FlashKDA, and AgentEnv. We hope this will accelerate the deployment and adoption of frontier intelligence and advance AGI research.
Kimi K3 Weights Released
Kimi K3 is our most capable model. It is a 2.8-trillion-parameter Mixture-of-Experts model with native visual understanding and support for a 1-million-token context window.
Kimi K3 has roughly three times as many parameters as Kimi K2.5. Scaling, however, is not merely the accumulation of parameters. Under constrained compute conditions, we achieved a 2.5× improvement in scaling efficiency through a set of technical innovations, including Kimi Delta Attention, Attention Residuals, and MoonEP. In compute-optimal terms, each unit of compute now produces roughly 2.5 times as much intelligence as before.
Everyone can now download and deploy the Kimi K3 model, whether for internal research and development or for integration into end-user products. For other use cases, please see the Kimi K3 License.
Download Kimi K3 model weights
Kimi K3 Technical Report Published
Alongside the Kimi K3 model weights, we are also making public our model training methods. The Kimi K3 technical report is now available.
The technical report covers the following details:
KDA + AttnRes: KDA and Gated MLA are mixed at a 3:1 ratio to enable efficient long-context modeling, while block-level attention residuals improve cross-layer information flow.
Stable LatentMoE: Each token activates 16 out of 896 routed experts. SiTU-GLU and Quantile Balancing maintain training stability under extremely high sparsity.
MoonViT-V2: The visual encoder is trained from scratch using next-token prediction, without contrastive pretraining. It reaches the baseline performance of SigLIP initialization while providing a more stable optimization process.
Post-training and evaluation: The report covers large-scale task synthesis across general reasoning, general agents, and coding agents; reinforcement learning infrastructure for million-token contexts; and full evaluation results from nearly 20 internal benchmarks.
These points are only an overview. Detailed discussions and ablation experiments are included in the technical report.
Download the Kimi K3 technical report
Key Kimi K3 Infrastructure Technologies Open-Sourced
Model capability depends on stable training systems at the infrastructure layer. Today, we introduce three infrastructure technologies that support Kimi K3 training: MoonEP, FlashKDA, and AgentEnv. They cover key links from high-performance communication and high-performance kernels to distributed RL environments. FlashKDA had already been open-sourced; MoonEP and AgentEnv are being officially open-sourced with this release.
MoonEP: MoonEP is a high-performance communication library designed for extremely large, fine-grained MoE models. It enables expert-parallel communication to maintain high efficiency even under imbalance.
FlashKDA: FlashKDA is our high-performance kernel implementation of Kimi Delta Attention. On NVIDIA H20, compared with the flash-linear-attention baseline, its prefill speed is 1.72–2.22× faster, and it can be used directly as a replacement backend for flash-linear-attention.
AgentEnv: AgentEnv is a sandbox system developed jointly with KVCache.ai for running agent environments at scale. It provides high-fidelity, strongly isolated sandboxes for Kimi K3 post-training, with flexible support for fast snapshotting, restoration, and forking. It is designed for massively parallel agent workflows and training tasks.
These three technologies are key to Kimi K3’s training efficiency and stability. We are opening them up in the hope that they can also help you train next-generation models.
Why We Choose Openness
We firmly believe in the value of open-weight models. They lower the barrier to accessing intelligence, promote innovation, and give users greater control over their data, privacy protection, and ownership. Over the past few weeks, we have been pleased to hear support from many members of the AI community and from technology leaders who share our vision.
We believe that for a technology with far-reaching impact like AGI, a broad and open ecosystem is the most suitable foundation. We will continue to contribute to that ecosystem.
Quick Start
GitHub Repository Content
Repository Overview
MoonshotAI/Kimi-K3 is titled “Open Frontier Intelligence.” The repository is public and includes an assets directory, LICENSE, README.md, and k3_tech_report.pdf. The repository page links to the Kimi chat site, Moonshot AI homepage, Hugging Face organization, Twitter/X account, Discord, ModelScope organization, the Kimi K3 license, the technical blog, and the full report.
1. Model Introduction
Kimi K3 is an open-weight, native multimodal agentic model and Moonshot AI’s most capable model to date. It is a 2.8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window. It is described by the repository as the world’s first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.
Key Features
New Architecture: Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), and scales up MoE sparsity with a Stable LatentMoE framework that activates 16 out of 896 experts, yielding an approximate 2.5× improvement in overall scaling efficiency over Kimi K2.
Long-Horizon Coding: Operating with minimal human oversight, Kimi K3 sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools, from GPU kernel optimization and compiler development to vision-in-the-loop game development, CAD, and chip design.
Agentic Knowledge Work: Kimi K3 advances end-to-end knowledge work, producing deep research with interactive visualizations, widgets and dashboards, motion design, and video editing, powered by its native multimodal architecture.
Native Multimodality & Long Context: Kimi K3 understands text, images, and video within the same model, and supports a 1-million-token context window.
Open Frontier Weights: Moonshot AI releases the full Kimi K3 model weights under the Kimi K3 License, making frontier intelligence openly available for research, deployment, and further innovation.
2. Model Summary
3. Evaluation Results
Benchmark | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol | Claude Opus 4.8 | GPT-5.5 | GLM-5.2 <tr><td>GPQA Diamond</td><td>93.5</td><td>92.6</td><td>94.1</td><td>91.0</td><td>93.5</td><td>91.2</td></tr> <tr><td>CritPt</td><td>23.4</td><td>28.6</td><td>32.3</td><td>20.9</td><td>27.1</td><td>20.9</td></tr> <tr><td>AA-LCR</td><td>74.7</td><td>70.0</td><td>73.7</td><td>67.7</td><td>74.3</td><td>71.3</td></tr> <tr><td>HLE-Full</td><td>43.5 / 56.0</td><td>53.3 / 63.0</td><td>44.5 / 58.0</td><td>49.8 / 57.9</td><td>41.4 / 52.2</td><td>—</td></tr> <tr><td>DeepSWE</td><td>67.5</td><td>70.0</td><td>73.0</td><td>59.0</td><td>67.0</td><td>46.2</td></tr> <tr><td>ProgramBench</td><td>77.8</td><td>76.8</td><td>77.6</td><td>71.9</td><td>70.8</td><td>63.7</td></tr> <tr><td>Terminal-Bench 2.1</td><td>88.3</td><td>88.0</td><td>88.8</td><td>84.6</td><td>83.4</td><td>82.7</td></tr> <tr><td>FrontierSWE</td><td>81.2</td><td>86.6</td><td>71.3</td><td>66.7</td><td>64.9</td><td>67.3</td></tr> <tr><td>SWE-Marathon</td><td>42.0</td><td>35.0</td><td>39.0</td><td>40.0</td><td>14.0</td><td>13.0</td></tr> <tr><td>PostTrainBench</td><td>36.6</td><td>41.4</td><td>34.6</td><td>34.1</td><td>28.4</td><td>34.3</td></tr> <tr><td>MLS-Bench-Lite</td><td>48.3</td><td>49.9</td><td>46.2</td><td>42.8</td><td>35.5</td><td>40.4</td></tr> <tr><td>SciCode</td><td>58.7</td><td>60.2</td><td>56.1</td><td>53.5</td><td>56.1</td><td>50.5</td></tr> <tr><td>Kimi Code Bench 2.0</td><td>72.9</td><td>76.9</td><td>64.8</td><td>71.7</td><td>69.0</td><td>64.2</td></tr> <tr><td>BrowseComp</td><td>91.2</td><td>88.0</td><td>90.4</td><td>84.3</td><td>84.4</td><td>—</td></tr> <tr><td>DeepSearchQA F1</td><td>95.0</td><td>94.2</td><td>—</td><td>93.1</td><td>—</td><td>—</td></tr> <tr><td>ResearchRubrics</td><td>76.2</td><td>—</td><td>73.8</td><td>73.5</td><td>64.0</td><td>71.1</td></tr> <tr><td>GDPval-AA v2 Elo</td><td>1686</td><td>1747</td><td>1736</td><td>1593</td><td>1491</td><td>1510</td></tr> <tr><td>Toolathlon-Verified</td><td>76.5</td><td>77.9</td><td>74.9</td><td>76.2</td><td>73.5</td><td>59.9</td></tr> <tr><td>MCPMark-Verified</td><td>94.5</td><td>87.4</td><td>92.9</td><td>76.4</td><td>92.9</td><td>—</td></tr> <tr><td>MCP-Atlas</td><td>84.2</td><td>84.7</td><td>83.6</td><td>83.6</td><td>82.8</td><td>82.6</td></tr> <tr><td>AutomationBench</td><td>30.8</td><td>29.1</td><td>29.7</td><td>27.2</td><td>22.7</td><td>12.9</td></tr> <tr><td>JobBench</td><td>54.3</td><td>57.4</td><td>45.4</td><td>48.4</td><td>38.3</td><td>43.4</td></tr> <tr><td>AA-Briefcase Elo</td><td>1548</td><td>1583</td><td>1495</td><td>1354</td><td>1158</td><td>1260</td></tr> <tr><td>Agents' Last Exam</td><td>28.3</td><td>25.7†</td><td>29.6</td><td>27.0</td><td>26.6</td><td>20.4</td></tr> <tr><td>APEX-Agents</td><td>41.0</td><td>43.3</td><td>39.9</td><td>39.4</td><td>38.5</td><td>35.6</td></tr> <tr><td>OfficeQA Pro</td><td>63.3</td><td>69.9</td><td>63.2</td><td>63.9</td><td>60.9</td><td>41.4</td></tr> <tr><td>SpreadsheetBench 2</td><td>34.8</td><td>34.7</td><td>32.4</td><td>31.6</td><td>29.1</td><td>28.1</td></tr> <tr><td>OSWorld-Verified</td><td>84.8</td><td>85.0</td><td>83.0</td><td>83.4</td><td>79.0</td><td>—</td></tr> <tr><td>OSWorld 2.0</td><td>58.3</td><td>66.1</td><td>62.6</td><td>55.7</td><td>49.5</td><td>—</td></tr> <tr><td>SaaS-Bench</td><td>60.1</td><td>—</td><td>61.4</td><td>56.1</td><td>43.8</td><td>—</td></tr> <tr><td>τ³-Banking</td><td>33.4</td><td>26.8</td><td>33.0</td><td>27.6</td><td>31.3</td><td>26.8</td></tr> <tr><td>Harvey Lab-AA</td><td>94.6</td><td>93.6</td><td>87.2</td><td>91.1</td><td>86.3</td><td>91.0</td></tr> <tr><td>CorpFin v2</td><td>71.6</td><td>71.8</td><td>64.4</td><td>66.7</td><td>68.4</td><td>66.1</td></tr> <tr><td>Finance Agent v2</td><td>54.4</td><td>56.3</td><td>53.8</td><td>53.9</td><td>51.8</td><td>49.7</td></tr> <tr><td>Legal Research Bench</td><td>44.2</td><td>49.5</td><td>48.1</td><td>43.8</td><td>40.4</td><td>31.3</td></tr> <tr><td>WorldVQA ForceAnswer</td><td>51.0</td><td>56.7</td><td>41.8</td><td>39.1</td><td>38.5</td><td>—</td></tr> <tr><td>OmniDocBench</td><td>91.1</td><td>89.8</td><td>85.8</td><td>87.9</td><td>89.4</td><td>—</td></tr> <tr><td>PerceptionBench</td><td>58.5</td><td>57.2</td><td>59.7</td><td>47.2</td><td>55.8</td><td>—</td></tr> <tr><td>Video-MME w. sub</td><td>90.0</td><td>—</td><td>89.5</td><td>86.0</td><td>89.3</td><td>—</td></tr> <tr><td>MMVU</td><td>82.1</td><td>—</td><td>81.2</td><td>79.2</td><td>81.7</td><td>—</td></tr> <tr><td>BabyVision w/ python</td><td>85.7</td><td>90.5</td><td>88.9</td><td>81.2</td><td>83.6</td><td>—</td></tr> <tr><td>MMMU-Pro</td><td>81.6 / 83.4</td><td>81.2 / 86.5</td><td>83.0 / 84.6</td><td>78.9 / 82.7</td><td>81.2 / 83.2</td><td>—</td></tr> <tr><td>CharXiv RQ</td><td>84.8 / 91.3</td><td>88.9 / 93.5</td><td>84.6 / 89.1</td><td>80.5 / 89.9</td><td>84.1 / 89.0</td><td>—</td></tr> <tr><td>MathVision</td><td>94.3 / 97.8</td><td>94.8 / 98.6</td><td>95.8 / 97.8</td><td>86.7 / 97.1</td><td>92.2 / 96.8</td><td>—</td></tr> <tr><td>ZeroBench pass@5</td><td>23.0 / 41.0</td><td>23.0 / 46.0</td><td>17.0 / 35.0</td><td>17.0 / 34.0</td><td>22.0 / 41.0</td><td>—</td></tr>
The repository footnotes state that all Kimi K3 results are obtained with reasoning effort set to max and temperature set to 1.0. For single-step tasks such as GPQA Diamond, HLE-Full, and vision benchmarks without tools, top-p is set to 0.95. For agentic tasks, top-p is set to 1.0. For HLE-Full, MMMU-Pro, CharXiv (RQ), MathVision, and ZeroBench, each cell reports scores without and with tool augmentation in that order.
Evaluation Footnotes
Reasoning and knowledge benchmark scores for CritPt and AA-LCR are cited from Artificial Analysis as of July 23, 2026.
For coding benchmarks, DeepSWE is evaluated with the Kimi Code harness. GLM-5.2 is taken from the GLM-5.2 release blog, while remaining DeepSWE scores are from the official DeepSWE leaderboard; Kimi K3 attains 67.3 with the mini-SWE-agent harness. Terminal-Bench 2.1 reports the best score across harnesses for other models. ProgramBench uses the Kimi Code harness for Kimi K3 and Vals AI for other non-GLM scores. SWE-Marathon is based on an H20-calibrated branch of official tasks as of July 9, 2026. FrontierSWE dominance scores are recomputed from raw scores using the official evaluation script and are current as of July 16, 2026. PostTrainBench scores for selected models are adopted from the official results, while Kimi K3, Claude Fable 5, and GPT-5.6 Sol are evaluated with Harbor at maximum reasoning effort over three runs on H20 GPUs.
For agentic benchmarks, OfficeQA Pro provides each test case with the entire PDF corpus as images and no machine-readable text. MCP-Atlas uses the 500-task public subset with a 100-turn limit and Gemini 3.1 Pro as the judge. AutomationBench uses the 600-task public subset. BrowseComp adopts a context-compaction strategy triggered at 300K tokens; with the full 1M-token context window and no context management, Kimi K3 scores 90.4. Several benchmark scores are cited from Artificial Analysis, APEX-Agents, Vals AI, and Agents’ Last Exam leaderboards as of July 23, 2026.
For multimodal benchmarks, all scores except ZeroBench are averaged over three runs. ZeroBench follows the official setting and is run five times. MMMU-Pro preserves the original input order and prepends images to the text input. PerceptionBench is described as an in-house benchmark focused on atomic visual perception capabilities.
4. Native MXFP4 Quantization
Kimi K3 applies quantization-aware training from the SFT stage onward, using MXFP4 weights with MXFP8 activations for broad hardware compatibility.
5. Deployment
Kimi K3’s API can be accessed at platform.kimi.ai by selecting kimi-k3. Moonshot AI provides OpenAI-compatible and Anthropic-compatible APIs. The repository recommends the following inference engines:
6. Model Usage
Kimi K3 always has thinking enabled and returns reasoning_content. Thinking effort is configured through the top-level reasoning_effort request field, which supports low, high, and max; the default is max.
Kimi K3 was trained in preserved thinking history mode. For multi-turn conversations and tool calls, Kimi K3 requires the complete assistant message returned by the API to be passed back into messages unchanged, including reasoning_content and tool_calls, not only content.
import openai
def chat_with_preserved_thinking(client: openai.OpenAI, model_name: str):
messages = [
{
"role": "user",
"content": "Tell me three random numbers."
},
{
"role": "assistant",
"reasoning_content": "I'll start by listing five numbers: 473, 921, 235, 215, 222, and I'll tell you the first three.",
"content": "473, 921, 235"
},
{
"role": "user",
"content": "What are the other two numbers you have in mind?"
}
]
response = client.chat.completions.create(
model=model_name,
messages=messages,
stream=False,
max_tokens=4096,
reasoning_effort="max",
)
# the assistant should mention 215 and 222 that appear in the prior reasoning content
print(f"response: {response.choices[0].message.reasoning}")
return response.choices[0].message.content
For full guides and examples covering vision input, structured output, partial mode, tool choice, dynamic tool loading, and context caching, see the Kimi K3 Quickstart and Thinking Effort.
Coding Agent Framework
Kimi K3 works best with Kimi Code CLI as its agent framework. The repository invites users to run Kimi Code in the terminal and select Kimi K3 using the /model command.
7. License
Both the code repository and the model weights are released under the Kimi K3 License.
8. Contact
For questions, the repository lists support@moonshot.ai.