Least Outstanding Requests

Least Outstanding Requests steering allows you to route traffic to pools that currently have the lowest number of outstanding requests.

This steering policy selects a pool by taking into consideration random_steering weights, as well as each pool’s number of in-flight requests. Pools with more pending requests are weighted proportionately less in relation to others.

Least Outstanding Requests steering is best to use if your pools are easily overwhelmed by a spike in concurrent requests. This steering method lends itself to applications that value server health above latency, geographic alignment, or other metrics. It takes into account the pool’s health status, adaptive routing, and session affinity.

Configure via the API

Load Balancers

{
  "steering_policy": "least_outstanding_requests"
}

Refer to the API documentation for more information on the load balancer configuration.

Note

Least Outstanding Requests steering can also be configured on a pool as a local traffic steering policy, taking into account outstanding request counts and weights for endpoints within the pool.

Limitations

Least Outstanding Requests steering can be configured for DNS-only load balancers, but is only supported in a no-operation form. For DNS-only load balancers, all pool outstanding request counts are considered to be zero, meaning traffic is served solely based on random_steering weights.

Although it is configurable, it is not recommended to use Least Outstanding Requests steering for DNS-only load balancers due to its partial support.