Least Outstanding Requests

Least Outstanding Requests steering allows you to route traffic to endpoints that currently have the lowest number of outstanding requests.

This steering policy selects an endpoint by taking into consideration endpoint weights, as well as each endpoint’s number of in-flight requests. Endpoints with more pending requests are weighted proportionately less in relation to others.

Least Outstanding Requests steering is best to use if your endpoints are easily overwhelmed by a spike in concurrent requests. It supports adaptive routing and session affinity.

Configure via the API

Pools

{
  "origin_steering": {
    "policy": "least_outstanding_requests"
  }
}

Refer to the API documentation for more information on the pool configuration.

Note

Least Outstanding Requests steering can also be configured on a load balancer as a global traffic steering policy, taking into account outstanding request counts and random_steering weights for pools on the load balancer.

Limitations

Least Outstanding Requests steering can be configured for pools that are part of DNS-only load balancers, but is only supported in a no-operation form. When endpoint steering logic is applied for a pool on a DNS-only load balancer, all endpoint outstanding request counts are considered to be zero, meaning traffic is served solely based on endpoint weights.

Although it is configurable, it is not recommended to associate pools that use Least Outstanding Requests steering with DNS-only load balancers due to its partial support.