This AI does less work per token. It can still eat your memory.

A mixture-of-experts model divides part of its network into alternatives and uses a router to select which ones process a token. The original Mixtral research provides a concrete example: each relevant layer has eight feed-forward experts, with two selected at each step. The selection can change as the sequence continues, so the same pair does not handle every word.

The paper describes forty-seven billion total parameters but around thirteen billion active during inference. Those figures answer different questions. Total parameters describe the complete model, including experts that might be selected later. Active parameters describe the work involved in processing a particular token. Shared parts of the network also explain why multiplying the number of experts by the size in the product name does not produce a precise model total. Sparse routing is a compute strategy, not a claim that the unselected weights have ceased to exist.

Hugging Face’s engineering explanation highlights the deployment consequence: conventional serving generally keeps all experts in memory, even though each token uses only some of them. A model can therefore behave more like a smaller dense network in computation while requiring memory closer to its full stored size. The comparison depends on the architecture and serving method, rather than the active count alone.

Other costs also remain, including routing and communication when experts are spread across accelerators. A sparse model is consequently not automatically faster on every device or at every batch size. For a local deployment, check the downloadable weight size, numerical precision, supported runtime and working memory required for the intended context. For a hosted API, measure completed-task cost and latency rather than inferring them from parameter marketing. An active-parameter figure is useful, but it describes one part of inference. It should not be used as a substitute for the hardware requirements of the complete system.