Comparing the data in two PingDirectory servers

The PingDirectory server provides an ldap-diff tool to compare the data on two LDAP servers to determine any differences that they might contain.

The differences are identified by first issuing a subtree search on both servers under the base distinguished name (DN) using the default search filter (objectclass=*) to retrieve the DNs of all entries in each server. When the tool finds an entry that is on both servers, it retrieves the entry from each server and compares all of its attributes. The tool writes any differences it finds to an LDIF file in a format that could be used to modify the content of the source server so that it matches the content of the target server. Any non-synchronized entries can be compared again for a configurable number of times with an optional pause between each attempt to account for replication delays.

You can control the specific entries to be compared with the --searchFilter option. In addition, only a subset of attributes can be compared by listing those attributes as trailing arguments of the command. You can also exclude specific attributes by prepending a ^ character to the attribute. On Windows operating systems, excluded attributes must be quoted, such as "^attrToExclude". The @objectClassName notation is used to compare only attributes that are defined for a given objectclass.

The ldap-diff tool can be used on servers actively being modified by checking differing entries multiple times without reporting false positives caused by replication delays. By default, it re-checks each entry twice, pausing two seconds between checks. You can configure these settings with the --numPasses and --secondsBetweenPass options. If the utility cannot make a clean comparison on an entry, it lists any exceptions in comments in the output file.

The PingDirectory server user specified for performing the searches must be privileged enough to see all of the entries being compared and to issue a long-running, unindexed search. For the PingDirectory server, the out-of-the-box cn=Directory Manager user has these privileges, but you can assign the necessary privileges by setting the following attributes in the user entry.

ds-cfg-default-root-privilege-name: unindexed-search
ds-cfg-default-root-privilege-name: bypass-acl
ds-rlim-size-limit: 0
ds-rlim-time-limit: 0
ds-rlim-idle-time-limit: 0
ds-rlim-lookthrough-limit: 0

The ldap-diff tool tries to make efficient use of memory, but it must store the DNs of all entries in memory. For PingDirectory servers that contain hundreds of millions of entries, the tool might require a few gigabytes of memory. If the progress of the tool slows dramatically, it might be running low on memory. You can customize the memory used by the ldap-diff tool by editing the ldap-diff.java-args setting in the config/java.properties file and running the dsjavaproperties command.

If you do not want to use a subtree search filter, you can use an input file of DNs for the source, target, or both. The format of the file can accept various syntaxes for each DN.

dn: cn=this is the first dn
dn: cn=this is the second dn and it is wrapped cn=this is the third dn
# The following DN is base-64 encoded dn::
Y249ZG9uJ3QgeW91IGhhdmUgYmV0dGVyIHRoaW5ncyB0byBkbyB0aGFuIHNlZSB3aGF0IHRoaXMgc2F5cw==
# There was a blank line above dn: cn=this is the final entry.

Do not manually update the servers when the tool identifies differences between two servers involved in replication. First contact your authorized support provider for explicit confirmation because manual updates to the servers risk introducing additional replication conflicts.

Comparing two PingDirectory servers using ldap-diff

Steps

Use ldap-diff to compare the entries in two PingDirectory server instances.

Ignore the userpassword attribute because of the one-way password hash used for the password storage scheme.

Example:

$ bin/ldap-diff --outputLDIF difference.ldif \
  --sourceHost server1.example.com --sourcePort 1389 \
  --sourceBindDN "cn=Directory Manager" --sourceBindPassword secret1 \
  --targetHost server2.example.com --targetPort 2389 \
  --targetBindDN "cn=Directory Manager" --targetBindPassword secret2 \
  --baseDN dc=example,dc=com --searchFilter "(objectclass=*)" "^userpassword"

Open the output file in a text editor to view any differences.

The file is set up so that you can re-apply the changes without any modification to the file contents. The file shows any deletes, modifies, and adds from the perspective of the source server as the authoritative source.

Example:

# This file contains the differences between two LDAP servers.
#
# The format of this file is the LDIF changes needed to bring server
# ldap://server1.example.com:1389 in sync with server
# ldap://server2.example.com:2389.
#
# These differences were computed by first issuing an LDAP search at both
# servers under base DN dc=example,dc=com using search filter (objectclass=*)
# and search scope SUB to first retrieve the DNs of all entries. And then each
# entry was retrieved from each server and attributes: [^userpassword] were
# compared. # # Any entries that were out-of-sync were compared a total of 3 times
# waiting a minimum of 2 seconds between each attempt to account for replication
# delays.
#
# Comparison started at [24/Feb/2010:10:34:20 -0600]
# The following entries were present only on ldap://server2.example.com:2389 and
# need to be deleted. This entry existed only on ldap://server1.example.com:1389
# Note: this entry might be incomplete. It only includes attributes:
# [^userpassword]dn: uid=user.200,ou=People,dc=example,dc=com
# objectClass: person
# objectClass: inetOrgPerson
... (more attributes not shown) ...
# st: DC
dn: uid=user.200,ou=people,dc=example,dc=com
changetype: delete

# The following entries were present on both servers but were out of sync.

dn: uid=user.199,ou=people,dc=example,dc=com
changetype: modify
add: mobile
mobile: +1 300 848 9999
-
delete: mobile
mobile: +1 009 471 1808

# The following entries were missing on ldap://server2.example.com:2389 and need
# to be added. This entry existed only on ldap://server2.example.com:2389
# Note: this entry might be incomplete. It only includes attributes:

# [^userpassword]
dn: uid=user.13,ou=People,dc=example,dc=com
changetype: add
objectClass: person
objectClass: inetOrgPerson
... (more attributes not shown) ...
# Comparison completed at [24/Feb/2010:10:34:25 -0600]

Comparing configuration entries using config-diff

Steps

Use config-diff to compare PingDirectory server configurations and produce a dsconfig batch file needed to bring the source inline with the target.

Example:

The following example compares the current configurations of server1 and server2. The changes necessary to bring server1’s configuration inline with server2 are written to the console. The same credentials are used to connect to both servers.
```
$ bin/config-diff --sourceHost server1 --sourceBindDN "cn=Directory Manager" \
  --sourceBindPassword password --targetHost server2
```
For more information about runtime options, see Available command-line tools or the config-diff tool help.

Comparing entries using source and target DN files

Steps

To compare the entries in two PingDirectory server instances, use ldap-diff.

In the following example, the utility uses a single distinguished name (DN) input file for the source and target servers, so that no search filter is used. Ignore the userpassword attribute because of the password storage scheme that uses a one-way hashing algorithm.

Example:

$ bin/ldap-diff --outputLDIF difference.ldif \
  --sourceHost server1.example.com --sourcePort 1389 \
  --sourceBindDN "cn=Directory Manager" --sourceBindPassword secret1 \
  --targetHost server2.example.com --targetPort 2389 \
  --targetBindDN "cn=Directory Manager" --targetBindPassword secret2 \
  --baseDN "dc=example,dc=com" --sourceDNsFile input-file.ldif \
  --targetDNsFile input-file.ldif "^userpassword"

Comparing PingDirectory servers for missing entries only using ldap-diff

Steps

To compare two PingDirectory servers and return only those entries that are missing on one of the servers, use ldap-diff with the --missingOnly option.

This can significantly reduce the runtime for this utility.

Example:

$ bin/ldap-diff --outputLDIF difference.ldif \
  --sourceHost server1.example.com --sourcePort 1389 \
  --sourceBindDN "cn=Directory Manager" --sourceBindPassword secret1 \
  --targetHost server2.example.com --targetPort 2389 \
  --targetBindDN "cn=Directory Manager" --targetBindPassword secret2 \
  --baseDN dc=example,dc=com --searchFilter "(objectclass=*)" "^userpassword" \
  --missingOnly