PingDirectory

About the replication protocol

The topic covers the replication, topology, protocol process, and background information.

Replication communication uses a proprietary binary protocol that implements on top of the TCP/IP protocol using SSL encryption. Some protocol messages are used for administrative purposes, such as WAN Gateway server negotiation or flow control, some carry updates to replicated data, and others are directed to all servers for monitoring requests.

In a replicated topology, each participating PingDirectory server connects to every other server through the replication server port to monitor health. Servers that share the same location setting are connected to rapidly replicate changes, and the WAN Gateway servers are all interconnected to replicate changes across locations.

The PingDirectory servers keep connections open as long as possible to reduce the communication latency when exchanging messages. Heartbeat messages transmit on a regular basis to detect a network failure or an unresponsive server as early as possible. Heartbeat messages also prevent idle connections from being closed by firewalls.

Replication Communication Flow

An illustration of the replication communication workflow between the Clients, Identity Data Store A, replication server A, Identity Data Store B, and replication server B. The steps of the replication communication process are explained in the following text.

  1. The client sends a MODIFY request to PingDirectory server A.

  2. PingDirectory server A assigns a unique change number to the operation. Conflict resolution is executed to see if the MODIFY request is in conflict with the existing attribute types or values in the existing entry. The change number is assigned before the server backend is updated to preserve the arrival order of client requests. Historical data in the target entry is updated to reflect the change.

    Historical data is only updated for ADD and MODIFY operations.

  3. The PingDirectory server applies the modifications in the corresponding backend.

  4. If the MODIFY operation successfully completes, then the PingDirectory server submits the update to its embedded replication server. The replication server is a component within the PingDirectory server process responsible for propagating updates to and from the replication topology. The PingDirectory server itself only communicates with a single replication server, whereas the replication server component is connected to all other replication servers. If the PingDirectory server process exits unexpectedly and some updates haven’t been passed to the replication server, the backend has the ability to recover the last 50,000 recent changes that were processed at this server, guaranteeing that these changes can be replicated when the server starts up. The figure above also shows that replication protocol is used not just between replication servers but also between the PingDirectory server and the replication server.

  5. The response is sent to the client. In this example, a successful response is assumed.

  6. The replication server records the update in its own Changelog backend, such as backend ID of replicationChanges and on disk with the path to changelogDb under the server root. The Replication Changelog backend keeps track of updates at each and every PingDirectory server in the replication topology. When a server joins the replication topology after being disconnected for some reason, updates from the Replication Changelog backend are re-sent to this server. Old records from the Replication Changelog backend are purged, which, by default, removes records older than 24 hours. If the backend doesn’t contain all of the records that another PingDirectory server needed when rejoining the replication topology, then the replicated data set in the PingDirectory server must be re-initialized. In this case, the server enters lockdown mode and sends an administrative alert.

  7. The replication server submits the update to the replication server component in PingDirectory server B. If there were more PingDirectory servers in this example, the replication server would submit the update to all the other replication servers in the same location.

  8. Just like in Step 6, the replication server component receiving an update inserts the change into its Replication Changelog backend.

  9. The update forwards to the Replica in PingDirectory server B. Conflict resolution executes to see if the MODIFY request is in conflict with the existing attribute types or values in the existing entry.

  10. The PingDirectory server applies the modification in the corresponding backend. The Recent Changes database doesn’t update because only updates that originate at this server record in the Recent Changes database.