Changelog for notifications

Some applications require notification when directory data updates occur. For example, an application might need to sync directory data with another database, or the application might need to kick off other processing when certain updates occur. DS replication servers provide an external changelog that replications can use to read changes. This mechanism is more scalable and robust than LDAP persistent searches.

By default, changelog database files are found under the opendj/changelogDb directory.

Do not compress, tamper with, or otherwise alter changelog database files directly, unless specifically instructed to do so by a qualified ForgeRock technical support engineer.

External changes to changelog database files can render them unusable by the server.

Enable the external changelog

DS servers that have a replication server port and an LDAP port publish an external changelog over LDAP:

Ports Configured Examples Notes

Ports Configured	Examples	Notes
LDAP or LDAPS port	`1389`, `1636`	LDAP client applications use the LDAP or LDAPS port to read changelog data. A standalone replication server may not have an LDAP or LDAPS port configured.
Replication port	`8989`	Servers with replication ports maintain a changelog for their own use. The changelog is exposed over LDAP under the base DN, `cn=changelog`. Standalone directory servers do not maintain a changelog by default.

LDAP or LDAPS port

1389, 1636

LDAP client applications use the LDAP or LDAPS port to read changelog data.

A standalone replication server may not have an LDAP or LDAPS port configured.

Replication port

8989

Servers with replication ports maintain a changelog for their own use. The changelog is exposed over LDAP under the base DN, cn=changelog.

Standalone directory servers do not maintain a changelog by default.

Make sure an LDAP or LDAPS port is configured so that LDAP client applications can read the changelog.

Depending on the server configuration, enable the changelog one of the following ways:

For servers with replication ports, make sure replication is properly configured.

The following example shows how the directory superuser can read the changelog:

$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN cn=changelog \
 --searchScope base \
 "(&)"

dn: cn=changelog
objectclass: top
objectclass: container
cn: changelog

For standalone directory servers without replication ports, you can manually enable the changelog. These steps cause the server to begin maintaining a replication changelog, which consumes disk space:

Update the default replication synchronization configuration entry as in the following example:

$ dsconfig \
 set-synchronization-provider-prop \
 --provider-name "Multimaster Synchronization" \
 --set bootstrap-replication-server:localhost:8989 \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Create a replication server configuration entry as in the following example:

$ dsconfig \
 create-replication-server \
 --provider-name "Multimaster Synchronization" \
 --set replication-port:8989 \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Create a replication domain configuration entry as in the following example:

$ dsconfig \
 create-replication-domain \
 --provider-name "Multimaster Synchronization" \
 --domain-name "Example.com" \
 --set base-dn:dc=example,dc=com \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Make sure the user who needs to read changelog data has the changelog-read privilege, and has access to read entries under cn=changelog.

For details, see Let a User Read the Changelog.

Encrypt changelog data

DS servers do not encrypt changelog data on disk by default. Any user with system access to read directory files can potentially read external changelog data.

In addition to preventing read access by other users, you can configure confidentiality for changelog data. When confidentiality is enabled, the server encrypts changelog records before storing them on disk. The server decrypts changelog records before returning them to client applications.

Encrypting stored directory data does not prevent it from being sent over the network in the clear.

Use secure connections to protect data sent over the network.

DS servers encrypt data using symmetric keys. Servers store symmetric keys, encrypted with the shared master key, with the data they encrypt. As long as servers have the same shared master key, any server can recover symmetric keys needed to decrypt data.

Symmetric keys for (deprecated) reversible password storage schemes are the exception to this rule. When you configure a reversible password storage scheme, enable the adminRoot backend, and configure a replication domain for cn=admin data.

Encrypting and decrypting data require cryptographic processing that reduces throughput, and extra space for larger encrypted values. Tests with default settings show that the cost of enabling confidentiality can be quite modest. Your results can vary based on the host system hardware, the JVM, and the settings for cipher-transformation and cipher-key-length. Make sure you test your deployment to qualify the impact of confidentiality before changing settings in production.

Follow these steps to enable confidentiality:

Before you enable confidentiality on a replication server for the changelog data, first enable confidentiality for data stored in directory backends.

For details, see Data encryption.

Enable changelog confidentiality with the default encryption settings:

$ dsconfig \
 set-replication-server-prop \
 --provider-name "Multimaster Synchronization" \
 --set confidentiality-enabled:true \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Encryption applies to the entire changelog regardless of the confidentiality settings for each domain.

After confidentiality is enabled, new changelog records are encrypted. DS servers do not rewrite old records in encrypted form.

If necessary, adjust additional confidentiality settings.

Use the same cipher suite for changelog confidentiality and data confidentiality.

The default settings for confidentiality are cipher-transformation: AES/GCM/NoPadding, and cipher-key-length: 128. This means the algorithm is the Advanced Encryption Standard (AES), and the cipher mode is Galois/Counter Mode (GCM). The syntax for the cipher-transformation is algorithm/mode/padding. You must specify the algorithm, mode, and padding. When the algorithm does not require a mode, use NONE. When the algorithm does not require padding, use NoPadding.

Let a user read the changelog

For a user to read the changelog, the user must have access to read, search, and compare changelog attributes, might have access to use the control to read the external changelog, and must have the changelog-read privilege.

Give the user access to read and search the changelog.

The following example adds two global ACIs. The first ACI gives My App read access to root DSE attributes that hold information about the changelog. The second ACI gives My App read access to the changelog data:

$ dsconfig \
 set-access-control-handler-prop \
 --add global-aci:"(target=\"ldap:///\")\
(targetattr=\"changeLog||firstChangeNumber||lastChangeNumber||lastExternalChangeLogCookie\")\
(version 3.0; acl \"My App can access changelog attributes on root DSE\"; \
allow (read,search,compare) \
userdn=\"ldap:///cn=My App,ou=Apps,dc=example,dc=com\";)" \
 --add global-aci:"(target=\"ldap:///cn=changelog\")(targetattr=\"*||+\")\
(version 3.0; acl \"My App can access cn=changelog\"; \
allow (read,search,compare) \
userdn=\"ldap:///cn=My App,ou=Apps,dc=example,dc=com\";)" \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

The IDM liveSync feature requires access to the root DSE attributes changeLog, firstChangeNumber, and lastChangeNumber.

Give the user access to use the public changelog exchange control.

The following example adds a global ACI to give My App access to use the control:

$ dsconfig \
 set-access-control-handler-prop \
 --add global-aci:"(targetcontrol=\"Ecl\")\
(version 3.0; acl \"My App control access\"; \
allow (read) \
userdn=\"ldap:///cn=My App,ou=Apps,dc=example,dc=com\";)" \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Give the user the changelog-read privilege:

$ ldapmodify \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password << EOF
dn: cn=My App,ou=Apps,dc=example,dc=com
changetype: modify
add: ds-privilege-name
ds-privilege-name: changelog-read
EOF

Check that the user can read the changelog:

$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN cn=changelog \
 --control "ecl:false" \
 "(&)" \
 changes changeLogCookie targetDN

Include unchanged attributes

The changes returned from a search on the external changelog include only what was actually changed. If you have applications that need additional attributes published with every changelog entry, regardless of whether the attribute itself has changed, specify those with the ecl-include and ecl-include-for-deletes properties:

Set the attributes to include for all update operations:

$ dsconfig \
 set-replication-domain-prop \
 --provider-name "Multimaster Synchronization" \
 --domain-name dc=example,dc=com \
 --set ecl-include:"@person" \
 --set ecl-include:entryUUID \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

The entryUUID can be useful, for example, to ensure integrity when entries are renamed.

Set the attributes to include for deletes:

$ dsconfig \
 set-replication-domain-prop \
 --provider-name "Multimaster Synchronization" \
 --domain-name dc=example,dc=com \
 --add ecl-include-for-deletes:"*" \
 --add ecl-include-for-deletes:"+" \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

With the default configuration, the changelog records only the DN of the deleted entry.

Exclude a domain

Exclude domains to prevent applications that read the external changelog from having to process update notifications for entries that are not relevant to them:

Change the replication server configuration to exclude the domain by base DN.

The following example prevents the changelog from sending notifications about Example.com entries:

$ dsconfig \
 set-replication-server-prop \
 --provider-name "Multimaster Synchronization" \
 --set changelog-enabled-excluded-domains:dc=example,dc=com \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Align draft change numbers

The external changelog can be used by applications that follow the Internet-Draft: Definition of an Object Class to Hold LDAP Change Records, and cannot process changelog cookies. Only default change number indexing is required to get the objects specified for this format, but there are steps you must perform to align change numbers across servers.

Change numbers described in the Internet-Draft are simple numbers, not cookies. When changelog numbers are aligned, applications can fail over from one server to another when necessary.

If you do not align the change numbers, each server keeps its own count. The same change numbers can refer to different changes on different servers.

For example, if you manually initialize a server from another, the last change numbers are likely to differ. The following example shows different last change numbers for two such servers:

$ ldapsearch \
 --hostname existing-ds.example.com \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN "" \
 --searchScope base \
 "(&)" lastChangeNumber

dn:
lastChangeNumber: 285924

Result Code:  0 (Success)

$ ldapsearch \
 --hostname new-ds.example.com \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN "" \
 --searchScope base \
 "(&)" lastChangeNumber

dn:
lastChangeNumber: 198643

Result Code:  0 (Success)

Follow these steps to align the change numbers with those of an existing server:

Make sure that the new server has the same replication configuration as the existing server.

The change number calculations must be the same on both servers. Specifically, both servers must replicate data under the same base DNs. If the base DN configurations differ, the change numbers cannot be aligned.
If you must start the new servers’s change numbering from a specific change, determine the changeNumber to use.

The changeNumber must be from a change that has not yet been purged following the replication purge delay (default: 3 days).

Reset the change number on the new server to the change number from the existing server.

The following example does not specify the change number to use. By default, the new server uses the last change number from the existing server:

$ dsrepl \
 reset-change-number \
 --sourceHostname existing-ds.example.com \
 --sourcePort 4444 \
 --sourceBindDn uid=admin \
 --sourceBindPasswordPassword password \
 --targetHostname new-ds.example.com \
 --targetPort 4444 \
 --targetBindDn uid=admin \
 --targetBindPasswordPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

The new server’s changelog now starts with the last change number from the existing server. Earlier change numbers are no longer present in the new servers’s changelog.

Disable change number indexing

By default, a replication server indexes change numbers for replicated user data. This allows applications to get update notifications by change number. Indexing change numbers requires additional CPU, disk I/O, and storage. Disable it if none of your applications require change number-based browsing:

Adjust the replication server settings appropriately for your deployment:
1. If applications need change notifications, but use changelog cookies rather than change numbers, set changelog-enabled:enabled-cookie-mode-only on each server.
  
  The replication server no longer indexes change numbers, and so has less work to do.
2. If no applications need notifications, set changelog-enabled:disabled on each server after enabling replication.
3. If applications need notifications for some domains but not for others, set changelog-enabled-excluded-domains on each server after enabling replication.

Directory Services 7.2.5