gpu
5 posts — newest first.
-
One VM, one GPU, 100 customers: what it actually takes to sell access to an open-weight model
An open-weight model serving HTTP takes twenty minutes. Making it something 100 customers can share takes a control plane. The whole path, with numbers.
-
Round-robin is malpractice for LLM traffic: what the Inference Gateway actually fixes
The Gateway API Inference Extension is GA. Why a normal Kubernetes Service balances model servers badly, and how to tell if you need an InferencePool.
-
GPU utilization is a lying metric: the unit economics of self-hosted inference
You can run a GPU at 100% utilization and waste most of it. The cost model that predicts your inference bill, and the five levers in payback order.
-
NAS vs SAN for GPU workloads — what changed when AI showed up
File vs block was the old NAS-vs-SAN question. GPU training rewrote it. How the calculus shifts when storage has to keep an H100 cluster fed.
-
The Memory Hierarchy: Why Data Locality Beats Clock Speed
Each memory level is 10–100× slower than the one above. Cache lines, locality, and why 'keep data near compute' is the biggest lever in LLM inference.