Keep storage and computation in separate columns.
A fully resident deployment must make the checkpoint's experts available even when a particular token does not use all of them. As a concrete publisher example, Gemma 4 26B-A4B lists approximately 25.2B backbone parameters and 3.8B active parameters, alongside a vision encoder. Those numbers answer different questions. When reading a model name, check whether embeddings, encoders, and auxiliary prediction heads are included in the stated total.
Work an example before shopping for hardware.
Imagine a simplified MoE with 100 billion total parameters and 10 billion active per token. At uniform 16-bit precision, the raw total weight storage is 200 GB. Multiplying only the active count would produce 20 GB and undercount storage by a factor of ten. Neither number includes serving allocations. This is an arithmetic teaching example, not an estimate for an actual checkpoint or evidence that any particular GPU arrangement works.
Offloading changes the experiment.
If weights are kept outside GPU memory and moved when needed, the deployment is no longer the same fully resident setup. Record where the weights live and the transfers involved. For our future tests, an offloaded configuration should have its own row, workload, and latency measurements. We would not combine its memory number with throughput obtained from an all-GPU run. That separation makes the result useful to someone trying to reproduce it.
Compare sparse and dense models on the same task.
Start with a quality target and a representative request set. Then compare successful throughput, tail latency, resident memory, and hardware cost under matching load. Do not assume that similar active counts make two architectures interchangeable. A useful shortlist can include both, but the ranking must come from measured behavior. Until those runs exist, BenchGrid keeps minimum and recommended GPU fields pending and links to the publisher's specifications.
Put the context to work.
Explore the models, compare the fundamentals, and start with a clearer picture.
Compare models