Confidential computing's performance cost varies sharply across CPUs and GPUs

Confidential computing is increasingly being adopted because some customers will not send data to models they cannot independently verify. Encrypting guest memory and attesting the software stack adds costs that differ widely by workload: CPU-only confidential VMs carry only a few percent overhead. Attestation round trips, protected I/O, and especially GPU confidential mode with its PCIe encryption and bounce buffers reduce throughput substantially. The author estimates roughly a 22% premium for H100 confidential mode, and up to 38% for confidential GPU SKUs that also cost more. The author concludes that confidentiality currently justifies its cost mainly for regulated workloads, and suggests splitting workloads so only tensor operations run on the confidential GPU.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in