General tips on indexes

Important critical indexes

The PingDirectory server has several built-in indexes on the local DB backend that are critical to internal server processing and should never be removed.

aci, ds-entry-unique-id, objectClass

You can define a composite index with a filter pattern of "(objectClass=?)" and use it in place of the objectClass equality attribute index. The composite index is more performant in cases where many entries have the same object class values.

In most cases, you should avoid setting the index entry limit too high for the objectClass index. In a typical directory of users and groups, you should set the index entry limit for indexes covering the objectClass type higher than the total number of groups in the server but lower than the number of users.

Built-in indexes for efficient queries

The PingDirectory server has built-in indexes on the local DB backend. Internal processing of the server relies on the aci, ds-soft-delete-from-dn, ds-soft-delete-timestamp, entryUUID, member, objectClass, and uniqueMember indexes, which must not be removed. You can remove the mail and uid indexes, but these attributes are referenced from the Password Modify Extended Operation and their removal causes problems with components, such as the Exact Match Identity Mapper. If you remove the mail or uid indexes, additional configuration changes might be necessary to ensure that the server starts properly. You can safely remove the cn, givenName, mail, sn, and telephoneNumber indexes if clients do not query on these attributes. This reduces the size of the database both on disk and in memory.

Using composite indexes for performance

When exploded attribute index keys exceed the index entry limit, background cleanup processing can sometimes significantly impact server performance. To avoid this performance impact, in cases where you expect index keys will match a large number of entries, use composite indexes instead of attribute indexes.

Composite indexes don’t have the performance and scalability issues that attribute indexes have when working with large ID sets, and they don’t require the same expensive cleanup processing for keys that exceed the index entry limit.

For example, you can use a composite index with a filter pattern of "(givenName=?)":

To replace an attribute index on the givenName attribute for the equality and ordering index types
To process substring searches that contain at least a subInitial (that is, "starts with") component

You can also use a composite index with a filter pattern of "(cn=?)" to replace a substring index on the cn attribute.

Online rebuilds

When an online index rebuild is in progress, the data in that backend is available and writable although the index being rebuilt is not used. Searches that attempt to use that attribute might be unindexed.

Index rebuild administrative alert

The PingDirectory server generates an administrative alert when the rebuild process begins and ends. It has a degraded-alert-type of index-rebuild-in-progress so that a PingDirectoryProxy server can avoid using the PingDirectory server while the rebuild is in progress.

System indexes cannot be rebuilt

The contents of the backend must be exported and re-imported in order to rebuild system indexes. For more information, see the list of System indexes.

Indexing certain attributes

Ensure that the following recommendations are used when setting up the indexes:

Do not use equality and substring indexes for attributes that contain binary data.
Avoid approximate indexes for attributes containing numbers, such as telephone numbers.

Unindexed searches

Unindexed attributes result in longer search times as the database itself has to be searched instead of the database index file. Only users with the unindexed-search privilege are allowed to carry out unindexed searches. In general, you should prevent applications from performing unindexed searches so that searches that are not indexed would be rejected rather than tying up a worker thread. Ways to achieve this include the following:

Make sure that only the absolute minimum set of users have the unindexed-search privilege. This privilege can be used without any other restrictions.
To allow unindexed searches with some control, the Permit Unindexed Search request control can be used with the unindexed-search-with-control privilege. With this privilege, a user is only permitted to request an unindexed search if the search request includes the Permit Unindexed Search request control. The unindexed-search privilege allows a client to request an unindexed search without this control.
You can use the Reject Unindexed Search request control to explicitly indicate that a client does not want the server to process an unindexed search request, regardless of privileges. For information about these controls, see the LDAP SDK. These capabilities are also available with the ldapsearch tool.
Make sure that allow-unindexed-searches property is set to false in all client connection policies, in which unindexed searches should never be necessary. If the client connection policy should allow undindexed searches, set the allow-unindexed-searches-with-control property to true. If allow-unindexed-searches is false but, allow-unindexed-searches-with-control is true, the policy only permits an unindexed search if the request includes the Permit Unindexed Search request control. For more information, see the LDAP SDK and the ldapsearch tool.
Set a nonzero value for the maximum-concurrent-unindexed-searches global configuration property to ensure that if unindexed searches are allowed, only a limited number of them are active at any given time. Administrators can configure the maximum number of concurrent unindexed searches by setting a property under Global Configuration.

To change the maximum number of concurrent unindexed searches, use the dsconfig tool to set a value for the number. A default value of "0" represents no limit on the number of concurrent unindexed searches.
```
$ bin/dsconfig set-global-configuration-prop \
  --set maximum-concurrent-unindexed-searches:2
```

Index entry limit

The PingDirectory server specifies an index entry limit property. This property defines the maximum number of entries that are allowed to match a given index key before it is no longer maintained by the server. If the index keys have reached this limit (default value is 4000), then you must rebuild the indexes using the rebuild-index tool. If an index entry limit value is set for the local DB backend, it overrides the value set for the overall JRE backend index entry limit configuration, such as 4000.

To change the default index entry limit, use the dsconfig tool, as in the following example.

$ bin/dsconfig set-local-db-index-prop --backend-name userRoot \
  --index-name cn --set index-entry-limit:5000

Rebuild index vs. full import

Expect a limited amount of database growth because of the existence of old data when running rebuild-index versus doing a full import of your database.