Directory Services 7.4.3

DS software

A directory service provides LDAP and HTTP access to distributed, shared directory data. A deployed directory service consists of one or more components. Each component plays a particular role in your directory service. Before you design your deployment, you need to be familiar with the roles that each component can play:

Directory service component diagram

  • Directory servers maintain and serve requests for directory data.

    Directory servers use data replication to ensure their data sets eventually converge everywhere. This documentation refers to a replicated directory server as a replica.

  • (Optional) Directory proxy servers forward LDAP requests to directory servers and return responses to client applications.

  • (Optional) Replication servers transmit data replication messages among replicas.

    Some advanced deployments use standalone replication servers. These servers only broker replication change messages and do not store directory data. In most deployments, each directory server acts as a replication server as well.

  • (Optional) DSML gateways intermediate between DSML client applications and an LDAP directory.

  • (Optional) HDAP gateways intermediate between RESTful HTTP client applications and LDAP directories.

  • LDAP client tools and server administration tools serve to test and configure servers.

Directory servers

Directory servers have the following characteristics.

Roles

Directory servers provide access to their copy of the distributed directory database. A directory server usually functions as the repository of identities for users, applications, and things. They respond to requests from client applications directly or indirectly through directory proxy servers. This includes the following:

  • LDAP requests for authentication, reads, and updates.

    An LDAP client application authenticates with the directory server, and then performs one or more operations before either re-authenticating to reuse the connection or ending the session and closing the connection.

  • HTTP read and update requests, often including credentials for authentication.

    An HTTP request translates to one or more internal LDAP requests.

  • Administrative requests, such as requests to modify the server configuration or to perform a task such as backup or LDIF export.

In deployments with multiple replicas, directory servers replay replicated operations. Expect each replica to replay every successful update to any replica.

Data

In addition to the libraries and tools delivered with the server distribution, a directory server has the following persistent state information and local data:

User data

Directory servers store user data. The directory server stores the data in local storage, such as an internal disk or an attached disk array. The storage must keep pace with throughput for update operations.

The amount of user data depends entirely on the deployment, ranging from a few LDAP entries to more than a billion. The amount of user data grows or shrinks depending on the pattern of update operations.

The directory server stores user data in a backend database. For details, refer to Data storage.

Metadata for replication

To avoid single points of failure, almost all real-world deployments depend on replication. Each directory server is a replica of other directory servers, meaning it holds an eventually consistent copy of the data on the other replicas.

When serving a request to update directory data, the directory server modifies its data and makes a request to a replication server. The replication server is usually but not always part of the same Java process as the directory server. The replication server ensures all other replicas update their data to eventually reflect the current state of the data.

To tolerate network partitions, the directory service supports concurrent update operations on different replicas. Concurrent updates potentially cause conflicts, but directory servers can resolve most conflicts automatically. To resolve conflicts, a directory server stores historical metadata alongside user data, trading space for resilience. For details, refer to About replication.

The directory server purges this historical metadata after a configurable interval. The volume of historical metadata depends on the total number of updates made to the directory service since the purge interval.

Server configuration

Each server has configuration data in its config directory. This includes the server configuration in LDIF and JSON files, LDAP schema definitions in LDIF files, keystores, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command applies necessary changes to the configuration data.

Log files

The server writes multiple log files by default, including error and access logs.

The server writes a message to the current access log for each operation. For high-volume directory services, log file storage must keep pace with the requests to record access to the service.

Log file retention and rotation policies prevent log file data from filling the disk. For details, refer to Logging. As a result of default retention policies, messages can eventually be lost unless you copy old files to another system for permanent storage.

Backup files

When you export directory data to LDIF or create a backup, the directory server writes the files to the specified directory. If you never purge or move these files, they can eventually fill the disk.

For details, refer to Import and export, and Backup and restore.

System resources

When deciding how to deploy a directory server, think of it as a copy of the database. A large, high-performance, distributed database serving lots of applications requires more system resources than a small database serving one, simple application.

A directory server requires the following system resources:

  • Sufficient RAM to cache frequently used data.

    For best read performance, cache the entire directory data set in memory.

  • Sufficient CPU to perform any required calculations.

    Authentication operations generally use more CPU than other operations. In particular, password storage schemes like PBKDF2 are designed to consume CPU resources. Calculations for transport layer security can use CPU as well, particularly if many client requests are over short-lived HTTPS connections.

  • Sufficient fast disk access to serve client applications, to replay replication operations, and to log access and errors.

    The underlying disk subsystem must serve enough input/output operations per second (IOPS) to avoid becoming a bottleneck when performing these operations. A small database that serves few client operations and changes relatively infrequently requires fewer IOPS than a large database sustaining many updates and serving many clients.

    Plan additional capacity for any backup or LDIF files stored on local partitions.

  • Sufficiently fast network access to serve client applications and relay replication traffic.

    When considering network requirements, keep the following points in mind:

    • Each LDAP search request can return multiple response messages.

    • Each request to update directory data results in corresponding replication traffic. The operation must be communicated to replication servers and replayed on each other directory server.

    • Once established, and unlike most HTTP connections, LDAP connections remain open until the client closes the connection, or until the server idles the connection. This is particularly true for applications using persistent searches, which by design are intended to be permanent.

Replication servers

A replication server is usually but not always part of the same Java process as a directory server.

Replication servers have the following characteristics.

Roles

Replication servers provide the following services:

  • Receive and transmit change messages between replicas.

    Each replica is connected to one replication server at a time.

  • Maintain information about all other replication servers and directory servers in the deployment that replicate the same data.

    Change messages travel from a connected directory server to the replication server. The replication server transmits the message to connected replicas and to the other replication servers, which in turn transmit the message to their connected replicas. This hub-and-spoke communication model means directory services can be composed of many individual servers.

  • Respond to administrative requests.

  • Respond to requests for monitoring information.

In all deployments using replication, the replication service provides the foundation of directory service availability. This is as important to the directory service as a naming service is for a network.

To avoid a single point of failure, always install two or more replication servers. For example, install at least two directory servers operating as replication servers as well.

Data

In addition to the libraries and tools delivered with the server distribution, a replication server has the following persistent state information and local data:

Change data

When serving a request to update directory data, a directory server, described in Directory servers, modifies its data and makes a request to a replication server. The replication server makes sure all other replicas update their data to eventually reflect the current state of the data.

The replication protocol is proprietary. Replication servers expose a public record of changes in a change log, allowing other applications to keep up to date with changes to user data. Replication servers store changes in change log files. For details, refer to Changelog for notifications.

The replication server purges this historical metadata after a configurable interval. The volume of historical metadata depends on the updates made to the directory service since the purge interval.

Server configuration

Each server has configuration data in its config directory. This includes the server configuration in LDIF and JSON files, LDAP schema definitions in LDIF files, keystores, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command applies necessary changes to the configuration data.

Log files

The server writes multiple log files by default, including error and access logs.

Log file retention and rotation policies prevent log file data from filling the disk. For details, refer to Logging. This means, however, that messages are eventually lost unless you move old files to another system for permanent storage.

System resources

When deploying a replication server, keep its foundational role in mind. Directory servers communicate with other replicas through replication servers. Directory proxy servers rely on replication servers to find directory servers.

A replication server requires the following system resources:

  • Sufficient fast disk access to log and read change messages, and to update access and error logs.

    The underlying disk subsystem must serve enough IOPS to avoid becoming a bottleneck when performing these operations.

  • Sufficiently fast network access to receive and transmit change messages for multiple replicas and for each other replication server.

Directory proxy servers

Some deployments use directory proxy servers.

Directory proxy servers have the following characteristics.

Roles

Directory proxy servers provide the following services:

  • Balance load of requests to LDAP directory servers.

  • Receive and transmit LDAP client requests to LDAP directory servers.

  • Receive and transmit LDAP directory server responses to LDAP client applications.

  • Respond to administrative requests.

  • Respond to requests for monitoring information.

A directory proxy server can hide the underlying directory service architecture from client applications, enabling you to build a single point of directory service access.

A directory proxy server can discover directory servers through a replication server. This capability depends on the replication server configuration. If you use the proxy server with third-party directory service components, then you must manually maintain the network locations for directory servers.

A directory proxy server provides LDAP access to remote LDAP directory servers. If you want to offer HTTP access to remote LDAP directory servers, use a gateway instead. For details, refer to HDAP gateway.

Data

In addition to the libraries and tools delivered with the server distribution, a directory proxy server has the following persistent state information and local data:

Server configuration

Each server has configuration data in its config directory. This includes the server configuration in LDIF and JSON files, LDAP schema definitions in LDIF files, keystores, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command applies necessary changes to the configuration data.

Log files

The server writes multiple log files by default, including error and access logs.

Log file retention and rotation policies prevent log file data from filling the disk. For details, refer to Logging. This means, however, that messages are eventually lost unless you move old files to another system for permanent storage.

System resources

A directory proxy server decodes incoming and encodes outgoing requests and responses. When you deploy directory proxy servers, the volume of decoding and encoding means you might need as many proxy servers as directory servers.

A directory proxy server requires the following system resources:

  • Sufficient fast disk access to update access and error logs.

    The underlying disk subsystem must serve enough IOPS to avoid becoming a bottleneck when performing these operations.

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding and encoding consume CPU resources.

  • Sufficient RAM to maintain active connections.

Command-line tools

When you install a server, its files include tools for setup, upgrade, configuration, and maintenance, and LDAP command-line tools for sending LDAP requests and measuring directory service performance.

For details, refer to Server commands.

DSML gateway

The interface stability of this feature is Deprecated.

The standalone DSML gateway web application has the following characteristics.

You can install this component independently of directory services. For details, refer to Install a DSML gateway.

Roles

DSML gateways provide the following services:

  • Receive HTTP DSML requests from client applications, and transmit them as LDAP requests to a directory service.

  • Receive LDAP responses from a directory service, and transmit them as HTTP DSML responses to client applications.

A DSML gateway runs in a Java web application container. It is limited to one host:port combination for the LDAP directory service.

Data

A DSML gateway maintains only its own service configuration, recorded in the web application WEB-INF/web.xml file. It depends on the host web application container for other services, such as logging.

System resources

A DSML gateway requires the following system resources:

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding, encoding, and transformation all consume CPU resources.

    Calculations to secure network connections also consume CPU resources.

  • Sufficient RAM to maintain active connections.

HDAP gateway

The standalone HDAP gateway web application has the following characteristics. REST refers to the representational state transfer architectural style. RESTful requests use the HTTP protocol.

The interface stability for HDAP is Technology Preview. Technology previews offer access to new technology ForgeRock doesn’t support yet. They may be functionally incomplete and subject to change without notice. For details, refer to Interface stability.

REST to LDAP remains supported as documented for DS 7.3. The interface stability for REST to LDAP is Deprecated in favor of HDAP for future applications.

You can install this component independently of directory services. For details, refer to Install an HDAP gateway.

Roles

HDAP gateways provide the following services:

  • Receive HTTP requests from client applications, and transmit them as LDAP requests to a directory service.

  • Receive LDAP responses from a directory service, and transmit them as HTTP responses to client applications.

An HDAP gateway runs in a Java web application container. You can configure the gateway to contact multiple LDAP directory servers.

Data

An HDAP gateway maintains only its own service configuration files. It depends on the host web application container for other services, such as logging.

System resources

An HDAP gateway requires the following system resources:

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding, encoding, and transformation all consume CPU resources.

    Calculations to secure network connections also consume CPU resources.

  • Sufficient RAM to maintain active connections.