Round-Robin vs Least Connections
Your application handles two types of requests:
- Fast API calls that complete in ~5 ms
- Video transcoding jobs that take ~30 seconds
Both types hit the same pool of servers. Would round-robin or least-connections be more appropriate? Explain why.
Least-connections is more appropriate here.
With round-robin, the load balancer distributes requests equally by count, ignoring how long each request ties up a worker. During a burst of transcoding jobs, some servers will accumulate multiple long-running jobs while others sit mostly idle, because round-robin sends the next fast request to whichever server is "next" regardless of its current load.
Least-connections routes each new request to the server currently handling the fewest active requests. Servers buried under transcoding jobs accumulate fewer new requests, while idle servers absorb more. This produces a more even actual load distribution when request durations vary widely.
General rule: use round-robin for homogeneous, short-lived requests; use least-connections when request duration varies significantly.