load-balancing
2 posts — newest first.
-
Load Balancing and Health Checks: The Three States Your Checks Don't Have
Balancing decides where load goes; health checks decide where it stops. Most outages here come from a check that can't tell busy apart from broken.
-
Round-robin is malpractice for LLM traffic: what the Inference Gateway actually fixes
The Gateway API Inference Extension is GA. Why a normal Kubernetes Service balances model servers badly, and how to tell if you need an InferencePool.