vllm
4 posts — newest first.
-
One VM, one GPU, 100 customers: what it actually takes to sell access to an open-weight model
An open-weight model serving HTTP takes twenty minutes. Making it something 100 customers can share takes a control plane. The whole path, with numbers.
-
Round-robin is malpractice for LLM traffic: what the Inference Gateway actually fixes
The Gateway API Inference Extension is GA. Why a normal Kubernetes Service balances model servers badly, and how to tell if you need an InferencePool.
-
GPU utilization is a lying metric: the unit economics of self-hosted inference
You can run a GPU at 100% utilization and waste most of it. The cost model that predicts your inference bill, and the five levers in payback order.
-
Making Claude Code Work with Locally Deployed Models
Claude Code can point at Ollama, vLLM, LM Studio, or any private endpoint. When that's the right call, exactly how to configure it, and what you give up.