Configuring criteria-based load-balancing algorithms

You can configure alternate load-balancing algorithms that determine how they function according to request or connection criteria.

These algorithms allow you to balance your load across a server topology depending on the types of operations received or the client issuing the request. They are called criteria-based load-balancing algorithms and are configured using at least one connection criteria or request criteria.

For example, you can configure criteria-based load-balancing algorithms to accomplish the following:

Route write operations to a single server from a set of replicated servers to prevent replication conflicts while load balancing all other operations across the full set of servers.
Route all operations from a specific client to a single server in a set of replicated servers, eliminating errors that arise from replication latency and load balancing operations from other clients across the full set of servers. This configuration is useful for certain provisioning applications that need to write and then immediately read the same data.

When a request is received, the proxying request processor first iterates through all of the criteria-based load-balancing algorithms in the order in which they are listed to determine whether the request matches the associated criteria. If there is a match, then the criteria-based load-balancing algorithm is selected. If there is not a match, then the default load-balancing algorithm is used.

Preferring failover LBA for write operations

You can configure the PingDirectoryProxy server to use criteria-based load-balancing algorithms to balance providing a consistent view of directory server data for applications that require it with taking advantage of all servers in a topology for handling read-only operations, such as search and bind. The flexible configuration model supports a wide range of criteria for choosing which load-balancing algorithm to use for each operation.

In most PingDirectoryProxy server deployments, you should use a failover load-balancing algorithm for at least add, delete, and modifyDN operations

Each Proxying Request Processor configured in the PingDirectoryProxy server uses a load-balancing algorithm to choose which PingDirectoryProxy server to use for a particular operation.

The load-balancing algorithm takes several factors into account when choosing a server:

The availability of the directory servers
The location of the directory servers

By default, load-balancing algorithms prefer directory servers in the same location as the PingDirectoryProxy server.
If the PingDirectoryProxy server is degraded for any reason, such as having a local database (DB) index being rebuilt
The result of configured health checks

For instance, a server with a small replication backlog can be preferred over one with a larger backlog.
Recent operation routing history

How these factors are used depends on the specific load-balancing algorithm. The two most commonly used load-balancing algorithms are:

The failover load-balancing algorithm
The fewest operations load-balancing algorithm

These two algorithms are similar when determining which directory servers are the possible candidates for a specific operation. The algorithms use the same criteria to determine server availability and health, and by default they prefer directory servers in the same location as the PingDirectoryProxy server. However, they differ in the criteria they use to choose between available servers. The following sections describe the algorithms in more detail.

Choosing between available servers

The failover load-balancing algorithm sends all operations to a single server until it is unavailable, and then sends all operations to the next preferred server, and so forth. This algorithm provides the most consistent view of the topology to clients because all clients, at least those in the same location as the PingDirectoryProxy server,, see the same, up-to-date view of the data, but it leaves unused capacity in the failover instances because most topologies include multiple directory server replicas within each data center.

The fewest operations load-balancing algorithm efficiently distributes traffic among multiple servers by choosing to send each operation to the server that has the fewest number of outstanding operations, which is the server from the PingDirectoryProxy server’s point of view that is the least busy.

The fewest operations load-balancing algorithm routes traffic to the least loaded server, which, in a lightly-loaded environment, can result in an imbalance because the first server in the list of configured servers is more likely to receive a request.

This algorithm naturally routes to servers that are more responsive as well as limits the impact of servers that have become unreachable. However, this implies that consecutive operations that depend on each other can be routed to different directory servers, which can cause issues for some types of clients:

If two entries are added in quick succession where the first entry is the parent of the second in the LDAP hierarchy, the addition of the child entry could fail if that operation is routed to a different directory server instance than the first add operation, and this happens within the replication latency.
Some clients add or modify an entry and then immediately read the entry back from the server, expecting to see the updates reflected in the entry.

In these situations, you should configure the PingDirectoryProxy server to route dependent requests to the same server.

The server affinity feature

The server affinity feature achieves this in some environments but not in all because the affinity is tracked independently by each PingDirectoryProxy server instance, and some clients send requests to multiple proxies. It is common for a client to not connect to the PingDirectoryProxy servers directly but instead to connect through a network load balancer, which in turn opens connections to the PingDirectoryProxy servers. Each individual client connection is established to a single PingDirectoryProxy server so that operations on that connection are routed to the same PingDirectoryProxy server, and server affinity configured within the PingDirectoryProxy server ensures those operations are routed to the same directory servers.

However, many clients establish a pool of connections that are reused across operations, and within this pool, connections are established through the load balancer to different PingDirectoryProxy servers. Dependent operations sent on different connections could then be routed to different PingDirectoryProxy servers, and then on to different PingDirectoryProxy servers.

For more information about the server affinity feature, see Configuring Server Affinity.

A failover load-balancing algorithm addresses this issue by routing all requests to a single server, but that leaves unused search capacity on the other instances. A criteria-based load-balancing algorithm enables the proxy to route certain types of requests (or requests from certain clients) using a different load-balancing algorithm than the default. For instance, all write operations (add, delete, modify, and modifyDN ) could be routed using a failover load-balancing algorithm while all other operations (bind, search, and compare) use a fewest operations load-balancing algorithm.

If there are clients that are particularly sensitive to reading entries immediately after modifying them, additional connection criteria can be specified to all operations from those clients using the failover load-balancing algorithm.

Routing all write requests to a single server in a location instead of evenly across servers does not limit the overall throughput of the system because all servers ultimately have to process all write operations either from the client directly or through replication.

Replication conflicts

Another benefit of using the failover load-balancing algorithm for write operations is reducing replication conflicts. The PingDirectory server follows the traditional LDAP replication model of eventual consistency. This provides high availability for handling write traffic even in the presence of network partitions, but it can lead to replication conflicts. Replication conflicts involving modify operations can be automatically resolved, leaving the servers in a consistent state where each attribute on each entry reflects the most recent update to that attribute. However, conflicts involving add, delete, and modifyDN operations cannot always be resolved automatically and might require manual involvement from an administrator. By routing all write operations (or at least add, delete, and modifyDN operations) to a single server, replication conflicts can be avoided.

When using a failover load-balancing algorithm, consider the following:

When using the failover load-balancing algorithm in a configuration with multiple locations, the load-balancing algorithm fails over between local instances before failing over to servers in a remote location. The list of servers in the backend-server configuration property of the load-balancing algorithm should be ordered such that preferred local servers should appear before failover local servers, but the relative order of servers in different locations is unimportant as the preferred-failover-location of the PingDirectoryProxy server’s configuration is used to decide which remote location to fail over to. It is also advisable that the order of local servers match the gateway-priority configuration settings of the replication server configuration object on the directory server instances. This can reduce the WAN replication delay because the PingDirectoryProxy server will then prefer to send writes to the directory server with the WAN Gateway role, avoiding an extra hop to the remote locations.
For PingDirectoryProxy server configurations that include multiple Proxying Request Processors, including Entry-Balancing environments, each Proxying Request Processor should be updated to include its own criteria-based load-balancing algorithm.

Routing operations to a single server

About this task

The following example shows how to extend a PingDirectoryProxy server’s configuration to use a criteria-based load-balancing algorithm to route all write requests to a single server using a failover load-balancing algorithm. This example can be extended to support alternate criteria and more complex topologies using multiple locations or entry balancing.

This example uses a basic deployment of a PingDirectoryProxy server fronting three directory servers: ds1.example.com, ds2.example.com, and ds3.example.com.

Steps

To create a location, run dsconfig with the create-location option.

Example:
```
dsconfig create-location --location-name Austin
```
To update the failover location for your server, run dsconfig with the set-location-prop option.

Example:
```
dsconfig set-location-prop --location-name Austin
```
To set the location as a global configuration property, run dsconfig with the set-global-configuration-prop option.

Example:
```
dsconfig set-global-configuration-prop --set location:Austin
```

To set up the health checks for each external server, run dsconfig with the create-ldap-health-check option.

Example:

dsconfig create-ldap-health-check \
--check-name ds1.example.com:389_dc_example_dc_com-search-health-check \
--type search --set enabled:true --set base-dn:dc=example,dc=com

dsconfig create-ldap-health-check \
--check-name ds2.example.com:389_dc_example_dc_com-search-health-check \
--type search --set enabled:true --set base-dn:dc=example,dc=com

dsconfig create-ldap-health-check \
--check-name ds3.example.com:389_dc_example_dc_com-search-health-check \
--type search --set enabled:true --set base-dn:dc=example,dc=com

To create the external servers, run dsconfig with the create-external-server option.

Example:

dsconfig create-external-server --server-name ds1.example.com:389 \
--type Ping Identity-ds \
--set server-host-name:ds1.example.com --set server-port:389 \
--set location:Austin --set "bind-dn:cn=Proxy User,cn=Root DNs,cn=config" \
--set password:AADoPkhx22qpiBQJ7T0X4wH7 \
--set health-check:ds1.example.com:389_dc_example_dc_com-search-health-check

dsconfig create-external-server --server-name ds2.example.com:389 \
--type Ping Identity-ds \
--set server-host-name:ds2.example.com --set server-port:389 \
--set location:Austin --set "bind-dn:cn=Proxy User,cn=Root DNs,cn=config" \
--set password:AAAoVqVYsEavey80T0QfR60I \
--set health-check:ds2.example.com:389_dc_example_dc_com-search-health-check

dsconfig create-external-server --server-name ds3.example.com:389 \
--type Ping Identity-ds \
--set server-host-name:ds3.example.com --set server-port:389 \
--set location:Austin --set "bind-dn:cn=Proxy User,cn=Root DNs,cn=config" \
--set password:AADOkveb0TtYR9xpkVrNgMtF \
--set health-check:ds3.example.com:389_dc_example_dc_com-search-health-check

To create a load-balancing algorithm, run dsconfig with the create-load-balancing-algorithm option.

Example:

This example creates a load-balancing algorithm for dc=example,dc=com.

dsconfig create-load-balancing-algorithm \
--algorithm-name dc_example_dc_com-fewest-operations \
--type fewest-operations --set  enabled:true \
--set backend-server:ds1.example.com:389 \
--set backend-server:ds2.example.com:389 \
--set backend-server:ds3.example.com:389

To create a request processor, run dsconfig with the create-request-processor option.

Example:

This example creates a request processor for dc=example,dc=com.

dsconfig create-request-processor \
--processor-name dc_example_dc_com-req-processor \
--type proxying \
--set load-balancing-algorithm:dc_example_dc_com-fewest-operations

To create a subtree view, run dsconfig with the create-subtree-view option.

Example:

This example creates a subtree view for dc=example,dc=com.

dsconfig create-subtree-view \
--view-name dc_example_dc_com-view \
--set base-dn:dc=example,dc=com \
--set request-processor:dc_example_dc_com-req-processor

To update the client connection policy, run dsconfig with the set-client-connection-policy-prop option.

Example:

This example updates the client connection policy for dc=example,dc=com.
```
dsconfig set-client-connection-policy-prop \
--policy-name default \
--add subtree-view:dc_example_dc_com-view
```

To create a new request criteria object to match all write operations, run dsconfig with the create-request-criteria option.

Example:

dsconfig create-request-criteria \
--criteria-name any-write \
--type simple --set "description:All Write Operations" \
--set operation-type:add --set operation-type:delete \
--set operation-type:modify --set operation-type:modify-dn

To create a new failover load-balancing algorithm listing the servers that should be included, run dsconfig with the create-load-balancing-algorithm option.

Example:

In this example, the order that the servers are listed in is the failover order between servers.
```
dsconfig create-load-balancing-algorithm \
--algorithm-name dc_example_dc_com-failover \
--type failover --set enabled:true \
--set backend-server:ds1.example.com:389 \
--set backend-server:ds2.example.com:389 \
--set backend-server:ds3.example.com:389
```

To tie the request criteria and the failover load-balancing algorithm together into a criteria-based load-balancing algorithm, run dsconfig with the create-criteria-based-load-balancing-algorithm option.

Example:

dsconfig create-criteria-based-load-balancing-algorithm \
--algorithm-name dc_example_dc_com-write-traffic-lba \
--set "description:Failover LBA For All Write Traffic" \
--set request-criteria:any-write \
--set load-balancing-algorithm:dc_example_dc_com-failover

To update the proxying request processor to use the criteria-based load-balancing algorithm, run dsconfig with the set-request-processor-prop option.

Example:

dsconfig set-request-processor-prop \
--processor-name dc_example_dc_com-req-processor \
--set criteria-based-load-balancing-algorithm:dc_example_dc_com-write-traffic-lba

Result

The PingDirectoryProxy server routes all write operations to ds1.example.com as long as it’s available and then to ds2.example.com if it’s not. The PingDirectoryProxy server routes other types of operations, such as searches and binds, to all three servers using the fewest operations load-balancing algorithm.

Routing operations from a single client to a specific set of servers

About this task

To create a type of server affinity where all operations from a single client are routed to a specific set of servers, follow a similar process as in the previous task, but configure connection criteria instead of request criteria. These connection criteria identify clients that could be impacted by replication latency. These clients use the failover load-balancing algorithm rather than the default fewest operations load-balancing algorithm.

For example, an administrative tool includes a delete user function. If the application immediately re-queries the directory for an updated list of users, the just-deleted entry must not be included.

To configure a criteria-based load balancing algorithm:

Steps

Create the new failover load-balancing algorithm using dsconfig with the create-load-balancing-algorithm option.

The failover load-balancing algorithm should list the same set of servers as the existing fewest operation load-balancing algorithm.

Example:
```
dsconfig create-load-balancing-algorithm \
  --algorithm-name client_one_routing_algorithm \
  --type failover --set enabled:true \
  --set backend-server:east1.example.com:389 \
  --set backend-server:east2.example.com:389
```
To route operations from a single client to a single server in a set of replicated servers, create connection criteria using dsconfig with the create-connection-criteria option.

Example:
```
dsconfig create-connection-criteria \
  --criteria-name "Client One"  --type simple \
  --set included-user-base-dn:cn=Client One,ou=Apps,dc=example,dc=com
```
Configure a criteria-based load balancing algorithm and assign it to the proxying request processor, usingdsconfig with the create-criteria-based-load-balancing-algorithm option.

Use the load balancing algorithm and connection criteria created in steps 1-2.

Example:
```
dsconfig create-criteria-based-load-balancing-algorithm \
  --algorithm-name dc_example_dc_com-client-operations \
  --set load-balancing-algorithm:dc_example_dc_com-failover \
  --set "request-criteria:Client One Requests"
```

Assign the new criteria-based load balancing algorithm to the proxying request processor using dsconfig with the set-request-processor-prop option.

Example:

dsconfig set-request-processor-prop \
  --processor-name dc_example_dc_com-req-processor \
  --add criteria-based-load-balancing-agorithm:dc_example_dc_com-client-operations