Moonshot released the Kimi K3.1 Pro weights, as promised by November 4. It is an MoE (a model in which only some experts run for each token) with 2.8 trillion parameters, of which 104 billion are active. There are 896 experts, with 16 active per token plus 2 shared experts. Of the 93 layers, 69 use KDA linear attention and 24 use sparse Gated DSA. These are the attention mechanisms the model uses to connect the current token to the preceding text. The cont