Overview of the LDAP changelog

The PingDirectory server provides a client-accessible LDAP changelog, based on the Changelog Internet Draft Specification, for the purpose of allowing other LDAP clients to retrieve changes made to the server in standard LDAP format.

The LDAP changelog is used by external software to maintain application compatibility between client services. For example, you can install the PingDirectory server that monitors the LDAP changelog for any updates that occur on a source PingDirectory server and synchronizes these changes to a target directory information tree (DIT) or database server.

The PingDirectory server provides an additional feature in that the LDAP changelog supports virtual attributes.

The LDAP changelog should not be confused with the Replication changelog. The main distinction is as follows:

The LDAP changelog (the external changelog that clients can access) physically resides at <server-root>/db/changelog.
The Replication changelog backend (the changelog that replication servers use) physically resides at <server-root>/changelogDB.

Key changelog features

As of 3.2, the PingDirectory server supports two new changelog backend properties that allow access control filtering and sensitive attribute evaluation for targeted entries.

External client applications can change the contents of attributes seen in the targeted entry based on the access control rules applied to the associated base DN.

Attribute Description

Attribute	Description
`apply-access-controls-to-changelog-entry-contents`	Indicates whether the contents of changelog entry attributes, such as `changes`, `deletedEntryAttrs`, `ds-changelog-entry-key-attr-values`, `ds-changelog-before-values`, and `ds-changelog-after-values`, are subject to access control and sensitive attribute evaluation to limit data that LDAP clients can see. The client must have the access control permissions to read changelog entries to retrieve them in any form. If this feature is enabled and the client does not have permission to read an entry at all, or if that client does not have permission to see any attributes that were targeted by the change, then the associated changelog entries targeted by those operations are suppressed. If a client does not have permission to see certain attributes within the target entry, then references to those attributes in the changelog entry are also suppressed. This property only applies to standard LDAP searches of the `cn=changelog` branch.
`report-excluded-changelog-attributes`	Indicates whether to include additional information about any attributes that might have been removed because of access control filtering. This property only applies to content removed as a result of processing performed by the `apply-access-controls-to-changelog-entry-contents` property. Possible values are: none Indicates that changelog entries should not include any information about attributes that have been removed. `attribute-counts` Indicates that changelog entries should include a count of user and operational attributes that have been removed. If any user attribute information was excluded from a changelog entry, the number of the excluded user attributes are reported in the `ds-changelog-num-excluded-user-attributes` attribute of the changelog entry. If any operational attribute information was excluded from a changelog entry, then the number of the excluded operational attributes are reported in the `ds-changelog-num-excluded-operational-attributes` attribute of the changelog entry. Both the `ds-changelog-num-excluded-user-attributes` and `ds-changelog-num-excluded-operational-attributes` are operational and must be explicitly requested by clients or all operational attributes requested using `+` to be returned. `attribute-names` Indicates that changelog entries should include the names of user and operational attributes that have been removed. If any user attribute information was excluded from a changelog entry, then the names of the excluded user attributes are reported in the `ds-changelog-excluded-user-attributes` attribute of the changelog entry. If any operational attribute information was excluded from a changelog entry, then the names of the excluded operational attributes are reported in the `ds-change-log-excluded-operational-attribute` attribute of the changelog entry. Both the `ds-changelog-excluded-user-attribute` and `ds-changelog-excluded-operational-attribute` attributes are operational and must be explicitly requested by clients or all operational attributes requested using `+` to be returned.

apply-access-controls-to-changelog-entry-contents

Indicates whether the contents of changelog entry attributes, such as changes, deletedEntryAttrs, ds-changelog-entry-key-attr-values, ds-changelog-before-values, and ds-changelog-after-values, are subject to access control and sensitive attribute evaluation to limit data that LDAP clients can see. The client must have the access control permissions to read changelog entries to retrieve them in any form. If this feature is enabled and the client does not have permission to read an entry at all, or if that client does not have permission to see any attributes that were targeted by the change, then the associated changelog entries targeted by those operations are suppressed. If a client does not have permission to see certain attributes within the target entry, then references to those attributes in the changelog entry are also suppressed. This property only applies to standard LDAP searches of the cn=changelog branch.

report-excluded-changelog-attributes

Indicates whether to include additional information about any attributes that might have been removed because of access control filtering. This property only applies to content removed as a result of processing performed by the apply-access-controls-to-changelog-entry-contents property. Possible values are:

none: Indicates that changelog entries should not include any information about attributes that have been removed.
attribute-counts: Indicates that changelog entries should include a count of user and operational attributes that have been removed. If any user attribute information was excluded from a changelog entry, the number of the excluded user attributes are reported in the ds-changelog-num-excluded-user-attributes attribute of the changelog entry. If any operational attribute information was excluded from a changelog entry, then the number of the excluded operational attributes are reported in the ds-changelog-num-excluded-operational-attributes attribute of the changelog entry. Both the ds-changelog-num-excluded-user-attributes and ds-changelog-num-excluded-operational-attributes are operational and must be explicitly requested by clients or all operational attributes requested using + to be returned.
attribute-names: Indicates that changelog entries should include the names of user and operational attributes that have been removed. If any user attribute information was excluded from a changelog entry, then the names of the excluded user attributes are reported in the ds-changelog-excluded-user-attributes attribute of the changelog entry. If any operational attribute information was excluded from a changelog entry, then the names of the excluded operational attributes are reported in the ds-change-log-excluded-operational-attribute attribute of the changelog entry. Both the ds-changelog-excluded-user-attribute and ds-changelog-excluded-operational-attribute attributes are operational and must be explicitly requested by clients or all operational attributes requested using + to be returned.

Enabling access control filtering in the LDAP changelog

Use the dsconfig tool to enable the properties to the changelog backend to set up access control to the LDAP changelog.

About this task

Only admin users with the bypass-acl privilege can read the changelog.

Steps

To allow LDAP clients to undergo access control filtering using standard LDAP searches of the cn=changelog backend, enable the apply-access-control-to-changelog-entry-contents property.

Access control filtering is applied regardless of the value of the apply-access-controls-to-changelog-entry-contents setting when the changelog backend is servicing requests from a PingDirectory server that has the filter-changes-by-user Sync Pipe property set.

Example:
```
$ bin/dsconfig set-backend-prop --backend-name "changelog" \
  --set "apply-access-controls-to-changelog-entry-contents:true"
```
To include a count of users that have been removed through access control filtering, set the report-excluded-changelog-attributes property.

The count appears in the ds-changelog-num-excluded-user-attributes attribute for users and in the ds-changelog-num-excluded-operational-attributes attribute for operational attributes.

Example:
```
 $ bin/dsconfig set-backend-prop --backend-name "changelog" \
  --set "report-excluded-changelog-attributes:attribute-counts"
```

Useful changelog features

The PingDirectory server provides two changelog configuration properties: changelog-max-before-after-values and changelog-include-key-attribute.

changelog-max-before-after-values

Setting this property to a non-zero value causes all of the old values and all of the new values, up to the specified maximum, for each changed attribute to be stored in the changelog entry. The values are stored in the ds-change-log-before-values and ds-changelog-after-values attributes on the changelog entry. These attributes are not present by default.

The changelog-max-before-after-values property can be expensive for attributes with hundreds or thousands of values, such as a group entry.

If any attribute has more than the maximum number of values, their names and number of before and after values are stored in the ds-changelog-attr-exceeded-max-values-count attribute on the changelog entry. This is a multi-valued attribute with the following format.:

attr=attributeName,beforeCount=100,afterCount=101

attributeName is the name of the attribute, and the beforeCount and afterCount are the total number of values for that attribute before and after the change, respectively. This attribute indicates that you must reset the changelog-max-before-after-values property to a higher value. When this attribute is set, an alert is generated.

If the number of values for an attribute exceeds the maximum value set by the changelog-max-before-after-values property, then those values are not stored.

changelog-include-key-attribute

This property is used for correlation attributes that need to be synchronized across servers, such as uid. It causes the current (after-change) value of the specified attributes to be recorded in the ds-changelog-entry-key-attr-values attribute on the changelog entry. This applies for all change types. On a DELETE operation, the values are from the entry before they were deleted.

The key values are recorded on every change and override the settings configured in changelog-include-attribute, changelog-exclude-attribute, changelog-deleted-entry-include-attribute, or changelog-deleted-entry-exclude-attribute.

Example of the changelog features

After the changelog-max-before-after-values property is set, the before and after values of any change attribute are recorded in the LDAP changelog.

For example, consider a simple entry with two multi-valued mail attributes.

dn: uid=test,dc=example,dc=com
objectclass: inetorgperson
cn: test user
sn: user
description: oldDescription
mail: test@yahoo.com
mail: test@gmail.com

Then, apply the following changes to the entry.

dn: uid=test,dc=example,dc=com
changetype: modify
add: mail
mail: test@hotmail.com
-
delete: mail
mail: test@yahoo.com
-
replace: description
description: newDescription

The resulting changelog would record the following attribute values.

dn: changeNumber=1,cn=changelog
objectClass: top
objectClass: changeLogEntry
targetDN: uid=test,dc=example,dc=com
changeType: modify
changes::
YWRkOiBtYWlsCm1haWw6IHRlc3RAaG90bWFpbC5jb20KLQpkZWxldGU6IG1haWwKbWFpbDogdGVzdEB5YWh
vby5jb20KLQpyZXBsYWNlOiBkZXNjcmlwdGlvbgpkZXNjcmlwdGlvbjogbmV3RGVzY3JpcHRpb24KLQpyZX
BsYWNlOiBtb2RpZmllcnNOYW1lCm1vZGlmaWVyc05hbWU6IGNuPURpcmVjdG9yeSBNYW5hZ2VyLGNuPVJvb
3QgRE5zLGNuPWNvbmZpZwotCnJlcGxhY2U6IGRzLXVwZGF0ZS10aW1lCmRzLXVwZGF0ZS10aW1lOjogQUFB
QkxxQitIaTQ9Ci0KAA==
ds-changelog-before-values:: ZGVzY3JpcHRpb246IG9sZERlc2NyaXB0aW9uCm1haWw6IHRlc3RAeW
Fob28uY29tCm1haWw6IHRlc3RAZ21haWwuY29tCmRzLXVwZGF0ZS10aW1lOjogQUFBQkxxQjdaZ1E9Cm1vZ
GlmaWVyc05hbWU6IGNuPURpcmVjdG 9yeSBNYW5hZ2VyLGNuPVJvb3QgRE5zLGNuPWNvbmZpZwo=
ds-changelog-after-values:: ZGVzY3JpcHRpb246IG5ld0Rlc2NyaXB0aW9uCm1haWw6IHRlc3RAZ21
haWwuY29tCm1haWw6IHRlc3RAaG90bWFpbC5jb20KZHMtdXBkYXRlLXRpbWU6OiBBQUFCTHFCK0hpND0KbW
9kaWZpZXJzTmFtZTogY249RGlyZWN0b3J5IE1hbmFnZXIsY249Um9vdCBETnMsY249Y29uZmlnCg==
ds-changelog-entry-key-attr-values:: dWlkOiB0ZXN0Cg==
changenumber: 1

Run the bin/base64 decode -d command line tool to view the decoded value for the changes, ds-changelog-before-values, and ds-changelog-after-values attributes.

After base64 decoding, the changes attribute displays the following.

add: mail
mail: test@hotmail.com
-
delete: mail
mail: test@yahoo.com
-
replace: description
description: newDescription
-
replace: modifiersName
modifiersName: cn=Directory Manager,cn=Root DNs,cn=config
-
replace: modifyTimestamp
modifyTimestamp: 20131010020345.546Z
-

After base64 decoding, the ds-changelog-before-values attribute displays the following.

description: oldDescription
mail: test@yahoo.com
mail: test@gmail.com
modifyTimestamp: 20131010020345.546Z
modifiersName: cn=Directory Manager,cn=Root DNs,cn=config

After base64 decoding, the ds-changelog-after-values attribute displays the following.

description: newDescription
mail: test@gmail.com
mail: test@hotmail.com
modifyTimestamp: 20131010020345.546Z
modifiersName: cn=Directory Manager,cn=Root DNs,cn=config

PingDirectory