moonshotai
Kimi-K3
Kimi K3 is a Moonshot AI multimodal model for text, images and video. It uses Kimi Delta Attention and Attention Residuals; the card reports a million-token-class context window.
Specifications
- Source-reported parameters
- Unknown
- Architectures
- Unknown
- Layers
- 93
- KV heads
- 96
- Head dimension
- Unknown
- config.json context
- 1048576
- Native context in model card
- Unknown
- Extended context
- Unknown
- Weight dtype
- Unknown
- File formats
- Unknown
- Declared languages
- Unknown
- Estimation method
- Not validated