Composite indexes

A PingDirectory server composite index can generate from multiple pieces of information, such as a combination of multiple filter components or a combination of filter components and a base distinguished name (DN), or from a single piece of information.

Because composite indexes are much more efficient than exploded indexes when dealing with large numbers of identical keys, use composite indexes for data sets where large numbers of entries (several thousand or more) share the same value for an indexed attribute.

To improve the performance of searches in directories with a large number of entries, especially with a large number of non-leaf entries, use equality composite indexes to combine a mandatory equality filter pattern with an optional base DN pattern.

Equality composite indexes offer two advantages over existing equality attribute indexes in the these types of deployments, a base DN pattern and index pages.

Base DN pattern

If a directory environment has many branches, but searches are often done within specific individual branches, use the base DN pattern to make search processing more efficient. The server only needs to search entries within a target branch.

For example, if the directory contains an "ou=Customers,dc=example,dc=com" branch with separate branches below that for sets of customers, such as "ou=ACME,ou=Customers,dc=example,dc=com" and "ou=SHOPCO,ou=Customers,dc=example,dc=com", you can define a composite index with a filter pattern of "(sn=?)" and a base DN pattern of "ou=?,ou=Customers,dc=example,dc=com". Then use a search with a filter of "(sn=Smith)" and a base DN of "ou=ACME,ou=Customers,dc=example,dc=com" to narrow the search to the Smiths in the ACME branch.

Index Pages

If many entries have the same value for a specific attribute, you can use composite indexes to break large ID sets up across multiple pages, unlike a traditional attribute index.

Using the previous example, if a search of the directory returns 50,000 Smiths, the results can be served in blocks of 5,000 IDs. An attribute index returns either:

One Smith record whose value is a block that contains all 50,000 of the matching entry IDs when the index isn’t exploded.

The exploded form is efficient for writing, for example, adding or removing a Smith involves just that one entry ID, but it is expensive for searching because it takes 50,000 reads to get all entry IDs for all of the Smiths.

50,000 Smith records that each have a value of the matching entry ID when the index is exploded.

The non-exploded form is efficient for searching because all of the entry IDs return in a single read, but it is expensive for writing. If a Smith must be added or removed, the entire block of 50,000 entry IDs must be rewritten.

Composite indexes break up the block of entry IDs across multiple pages, a page size of up to 5000. If the directory contains 50,000 Smiths, instead of having to choose between one block of 50,000 IDs or 50,000 blocks of one ID, the composite index returns 10 blocks of 5,000 IDs. This improves the efficiency of a read or write across many entries.

There is little performance overhead to the paging mechanism. Use an equality composite index for an attribute that has a lot of entries that have the same value, such as givenName or sn. Do not use a equality composite index for an attribute with very few entries with the same value, such as id or mail. For attributes in which all of the values match a small number of entries, use an equality attribute index.

Rebuilding indexes

When rebuilding specific non-attribute indexes, it is important to include any appropriate prefixes or postfixes with the index name to avoid errors.

To rebuild a composite index, use the rebuild-index tool. The index name must be preceded by composite.

bin/rebuild-index --index composite.thisIndexName

For the exact name to use in rebuilding the index, run the dbtest tool with the list-database-containers option and --backendID property.

bin/dbtest list-database-containers --backendID {backendID}

Define the following properties when configuring a composite index.

Composite Index Properties Description

Composite Index Properties	Description
`index-filter-pattern` (required)	Specifies a single-valued filter property used to identify a portion of the index criteria. You can only specify this at the time that the index definition is created.
`index-base-dn-pattern` (optional)	Specifies a single-valued DN property that indicates that the index is scoped to a specific subtree or subtree pattern. You can only specify this at the time the index definition is created.

index-filter-pattern (required)

Specifies a single-valued filter property used to identify a portion of the index criteria. You can only specify this at the time that the index definition is created.

index-base-dn-pattern (optional)

Specifies a single-valued DN property that indicates that the index is scoped to a specific subtree or subtree pattern. You can only specify this at the time the index definition is created.