SlimWise prunes MoE experts only during decoding, nearly doubling throughput

Mixture-of-Experts inference often stalls on expert traffic during batched decoding, which limits latency gains. SlimWise addresses this by pruning experts solely in the decode phase while reusing the full-model KV cache built during prefill. On Qwen3.6-35B-A3B, the approach raises decode throughput by up to 1.81 times at 50 percent expert pruning, with only minimal accuracy loss. Because the cache format is unchanged, the runtime can switch weight masks between phases without costly data conversion, and the gains were shown on two MoE backbones and three pruning criteria. Scalability to larger models and more aggressive pruning ratios remains untested, and the lightweight distillation step adds a modest training requirement.