KTransformers Trades GPU Memory for CPU RAM, Raising NUMA Complexity Questions

KTransformers is being evaluated as a way to run large mixture-of-experts models by keeping routed experts in CPU memory and computing them on the CPU, while GPUs handle attention and other intensive work. The reviewers argue this could cut costs versus buying enough GPUs to hold every expert, though the benefit looks clearer for low-concurrency workloads than for latency-sensitive public serving. They did not run a GPU inference benchmark, lacking the required hardware and model weights, so performance figures cited are published measurements rather than their own results. The review separates hardware execution optimizations, NUMA-aware placement, and behavior-changing techniques like reducing activated experts, arguing they should not be merged into a single speedup figure. The team plans to reproduce the historical native setup and the newer SGLang path separately to keep benchmarks reproducible.