PingDirectory suite of products 10.0.0.0 (December 2023) - PingDirectory - 10.0

PingDirectory

• Historically, LDAP servers favor data integrity over resiliency. However, given the growth in customer topologies, there is a strong requirement for maintaining production server uptimes to meet customer expectations. In this environment, servers can be removed from the topology frequently, and if the server is down longer than the configured replication purge delay, problems could arise once the server is brought back online. In this release, a new feature allows you to configure the level of availability when encountering this issue during topology management.
• Static groups, which are the simplest and most commonly used type of group, explicitly list the DNs of group members. Server performance when adding or removing members from a static group depends partially on the group size itself, but we have identified a number of further inefficiencies in how the server handles static group membership changes. This release includes changes to improve performance when updating static groups.

  This release also introduces a new group type: inverted static groups. As with traditional static groups, inverted static group membership is explicitly defined rather than automatically determined. However, instead of storing the entire list of members in the group entry, each user entry lists the set of inverted static groups in which that user is a member. Inverted static groups with a large number of members can be more efficient to maintain than traditional static groups, because the change needed to add or remove a user only requires updating the user entry, which is not affected by the number of members in the group. The server also provides an optional plugin that allows an inverted static group to be updated as if it were a traditional static group, intercepting attempts to alter the membership attribute in the group entry itself and making the corresponding changes in user entries instead.

• PingDirectory allows clients to interact with the server using a REST API over HTTP as an alternative to LDAP. Recent updates to the Directory REST API, including the addition of support for controls and select extended operations, have improved feature parity between the REST-based and LDAP-based interfaces, creating a more robust experience for developers using the REST API.

  While it is possible to authorize individual requests using either HTTP basic authentication (using the DN and password of the target user) or with an OAuth 2 access token obtained through another service, the Directory REST API didn’t provide a fine-grained way of verifying user credentials. This release introduces a new authenticate endpoint, which provides a way for Directory REST API clients to verify user credentials. This enables you to better differentiate authentication failures from authorization failures, and to obtain an access token to use in authorizing subsequent requests as a specific user. Users can be identified with either a DN or a username, and the credentials may include a static password on its own or in conjunction with a delivered one-time password, a time-based one-time password, or a one-time password generated by a YubiKey device.

• PingDirectory has always offered support for defining deprecated password storage schemes. If a user successfully authenticates and provides the server their clear-text password, and if their password is currently encoded with an undesirable scheme, the server can automatically re-encode their password using a more desirable scheme. This release expands on this functionality by making it possible to re-encode passwords if the configuration of the underlying scheme has changed in a way that affects the scheme’s stored representation.

  For example, if a user’s password is encoded using the PBKDF2 scheme, the server can now automatically re-encode the password if their stored password uses a digest algorithm, iteration count, salt length, or derived key length that doesn’t match the current configuration of that scheme. PingDirectory has also long supported the Pwned Passwords service, rejecting attempts to set passwords that are known to have been compromised. In the past, interaction with the Pwned Passwords service used a hard-coded timeout of 30 seconds in case the service became unreachable or unresponsive. You can now customize that timeout.

• PingDirectory uses the Berkeley DB Java Edition to store its data, and this database library offers support for caching some or all of the data in memory for faster access. PingDirectory also allows administrators to configure separate backends to hold different portions of the DIT. Previously, the server maintained a separate database cache for each backend, requiring the administrator to adjust the percentage of the JVM’s memory that each backend is allowed to consume. This release now enables you to share a common database cache across all backends. Although this capability is disabled by default, it can simplify the server configuration by only requiring administrators to specify the total percentage of JVM memory to use for caching, without needing to configure caching separately for each backend.
• Amazon’s Simple Storage Service (S3) is a popular cloud-based data storage service that can be used as a convenient off-site backup mechanism. In the past, some PingDirectory server administrators have chosen to manually copy certain types of files (for example, LDIF exports or rotated log files) to an S3 bucket as an additional layer of safety in their disaster recovery strategy. This release introduces direct support for using the S3 service as a way of backing up LDIF exports and log files.

  This release offers support for post-LDIF-export task processors. This enables you to automatically perform additional processing after successfully completing an LDIF export, including exports created as part of a recurring task. We have included an implementation that can copy the resulting export file to a specified S3 bucket for safekeeping, and it can automatically remove older export files from that bucket based on the number or age of files in that bucket. It is also possible to use the Server SDK to develop custom post-LDIF-export task processor implementations to perform other kinds of processing after an export completes.

  This release offers a new log file rotation listener that can automatically copy log files to a specified S3 bucket as soon as they have been rotated out of place. This support is available for most types of log files that the server can generate, and it also supports automatic retention based on the number or age of the files in the bucket. The server now includes a new amazon-s3-client command-line tool that can be used to manually interact with the S3 service. This tool can be used to manually manage buckets and files contained in the S3 service, including uploading files to or downloading files from a specified bucket.

• This release includes changes to dramatically improve performance when creating a backup, restoring a backup, or performing online replica initialization.

Fixed a security issue that could potentially affect customers using Delegated Admin. Customers are advised to apply a maintenance patch or upgrade to the latest supported version of the PingDirectory server. The Delegated Admin application is unaffected and does not require updating. Additional details are provided in SECADV039 (requires sign-on).

Added a new amazon-s3-client command-line tool that can be used to interact with the Amazon AWS Simple Storage Service (S3) service. This tool enables you to list, create, and delete buckets, as well as list, upload, download, and delete files in a specified bucket. This may be useful in deployments where the server is configured to automatically copy rotated log files or exported LDIF files to the S3 service.

Added support for access log field request control in Directory REST API requests.

Added an /authenticate endpoint to the Directory REST API that enables users to generate an access token by providing combinations of valid credentials, depending on the authentication type specified in the HTTP request body. The supported authentication types are:

• password
• passwordPlusTOTP
• passwordPlusDeliveredOTP
• passwordPlusYubiKeyOTP

For more information on the /authenticate endpoint, see Managing the Directory REST API.

Added five new Directory REST API endpoints to support the new /authenticate endpoint. These endpoints enable users to interact with supporting services that facilitate the creation, delivery, and revocation of one-time passwords (OTP) and time-based one-time passwords (TOTP), which are required to perform authentication operations with the API. These endpoints include:

• /directory/v1/{dn}/generateTOTPSharedSecret
• /directory/v1/{dn}/revokeTOTPSharedSecret
• /directory/v1/deliverOneTimePassword
• /directory/v1/{dn}/registerYubiKeyOTPDevice
• /directory/v1/{dn}/derigesterYubiKeyOTPDevice

For more information on these endpoints, see Managing the Directory REST API.

Updated the bcrypt password storage scheme to include support for the 2b variant in addition to the existing 2y, 2a, and 2x variants.

Added support for post-LDIF-export task processors to use in performing custom processing whenever an LDIF export task (including those invoked as part of a recurring task) successfully completes the export.

These processors include an Upload to S3 processor, which can be used to upload the resulting LDIF file to a specified Amazon S3 bucket. You can also use the Server SDK to create custom post-LDIF-export task processors. For more information, see Performing post-LDIF-export task processing.

Added support for inverted static groups, which operate like traditional static groups in that membership is explicitly specified rather than dynamically determined, but where membership information is stored in user entries rather than in the group entry. For groups with a large number of members, inverted static groups may exhibit substantially better performance than traditional static groups.

Although it is not enabled by default, the server also provides a new plugin that makes it possible for clients to interact with inverted static groups in much the same way as they interact with traditional static groups. The plugin will intercept attempts to add or remove member DNs in the group entry itself and will instead cause the corresponding changes to be applied in the member entries. It also provides limited support for interacting with group members in the group entry for search and compare operations as if the member DNs actually existed in the group entries. For more information, see Using inverted static groups.

Added a split-ldif tool that can be used to split an LDIF file into multiple segments, with each having a subset of the entries below a specified base DN, and entries at or above that base DN will be included in all sets. This is primarily intended for splitting a large data set for use in entry balancing, and it offers several algorithms for dividing the entries between segments.

Added a new HTTP Connection configuration property to enable SNI hostname checks, which are now disabled by default.

Added the include-all-remote-servers-state-in-monitor-message configuration property to control whether replication monitor messages include information about remote servers. By default, the property is set to true so that information about remote servers is sent. Setting the property to false may be helpful in large topologies because the size of monitor messages scales with the number of servers.

Added a new log file rotation listener that can be used to upload newly rotated log files to a specified Amazon S3 bucket. The listener can remove previously updated log files based on the specified number or age of files to retain.

Added the ability to share a single database cache across all local DB backends. This is an alternative to the default behavior in which each local DB backend maintains its own independent database cache, and it can simplify cache sizing in deployments with multiple local DB backends. This behavior is controlled by two new global configuration properties:

• use-shared-database-cache-across-all-local-db-backends: Indicates whether to use a shared database cache. If this property is set to true, then all local DB backends will use a shared database cache, and you must set the property to specify the size of that shared cache. If the property is set to false (the default value), then each local DB backend will maintain its own independent database cache with a size specified by the db-cache-percent property configuration property for that backend.
• shared-local-db-backend-database-cache-percent: Specifies the percentage of the total JVM heap size that will be used for the shared database cache. This property will only be used if the use-shared-database-cache-across-all-local-db-backends property is set to true, in which case the server will ignore the db-cache-percent property in the backend configuration.

If a shared database cache is enabled, the server will expose a Shared Local DB Backend Database Cache monitor entry with information about that shared cache, including how much of the cache is consumed by each of the backends.

Added the re-encode-passwords-on-scheme-config-change property to the password policy configuration to indicate if the server should automatically re-encode passwords that are encoded with settings that don't match the scheme's current configuration. If a user authenticates with a mechanism that provides their password unencoded, and if the password stored in their entry is encoded with settings that don't match the current configuration for the associated password storage scheme, then the server now automatically re-encodes their password with the default password storage scheme(s) using the current settings. The following password storage schemes support this functionality: AES256, ARGON2, ARGON2D, ARGON2I, ARGON2ID, BCRYPT, PBKDF2, SCRYPT, SSHA, SSHA256, SSHA384, and SSHA512.

You can also implement this capability for custom password storage schemes developed with the Server SDK.

The ds-pwp-state-json virtual attribute provider has also been updated with a new has-password-encoded-with-non-current-settings field whose value indicates if the user's password is encoded with settings that don't match the current configuration, and a new non-current-password-storage-scheme-settings-explanations field that can provide additional details on how the password encoding differs from the current configuration.

Added a --re-encrypt argument to the encrypt-file tool to read the contents of an existing encrypted file and re-encrypt it with a different encryption settings definition or user-supplied passphrase. If the file is currently encrypted with a user-supplied passphrase, then the --prompt-for-current-passphrase or --current-passphrase-file argument should be used to supply the current encryption passphrase. If the file is currently encrypted with an encryption settings definition, then that definition will automatically be obtained from the encryption settings database.

Added a --find-encrypted-files argument to the encrypt-file tool to identify encrypted files in a specified location on the filesystem. By default, the tool will search for files that are encrypted with any encryption settings definition or a user-supplied passphrase, but it can be used in conjunction with the --encryption-settings-id argument to only identify files that are encrypted with the specified definition.

These new arguments can be useful when migrating away from a former encryption settings definition, particularly if the former definition will eventually be removed from the encryption settings database. If a definition is removed from the encryption settings database, any files encrypted with that definition will no longer be accessible.

Added a replication-missing-changes-policy configuration property for both replication servers and replication domains to control how replication handles missing changes. This property can be used to avoid missing changes lockdown in cases where such lockdown is not beneficial to the server.

When the missing changes policy is modified, connections are restarted so that the missing changes state can be reevaluated. Lockdown mode is not cleared, but may be cleared by running the leave-lockdown-mode tool.

Added support for an access log field request control to specify field names and values that should be included in the access log message for the associated operation.

Added support for a generate access token request control that can be included in a bind request to indicate that the server should generate and return an access token in the bind response. That access token may be used in conjunction with the OAUTHBEARER SASL mechanism to authorize subsequent connections by that client. This can be useful in cases where the initial authentication should be performed in a manner that involves single-use credentials like a time-based one time password, a delivered one-time password, or a one-time password generated by a YubiKey device, but the client wishes to establish multiple connections in which the initial credentials cannot be replayed.

Upgraded Jetty version to 10.0.17.

Removed support for Java 8 in the PingDirectory server. For more information, see System requirements. For information on upgrading from a PingDirectory instance installed with Java 8, see Java considerations when upgrading to version 10.0.

Removed support for the deprecated remove-defunct-server and cleanup-local-server dsreplication subcommands. To remove a defunct server from the topology, use the remove-defunct-server command-line tool. To clean up topology references on a server, run remove-defunct-server --performLocalCleanup.

PingDataMetrics was previously deprecated and has been removed from this release. For more information about support for versions of PingDirectory containing PingDataMetrics, see Ping Identity's End-of-Life Policy (sign on required).

To monitor and provide statistics for your PingDirectory suite of products, see Monitoring PingDirectory metrics with Splunk and Monitoring server metrics with Prometheus.

Updated the server to allow configuration of connect and response timeouts when communicating with external HTTP services, such as CyberArk Conjur and HashiCorp Vault instances, the Pwned Passwords service, and YubiKey OTP validation servers.

To improve server performance and prevent invalid block type errors, java.util.zip will now be used instead of com.jcraft.jzlib for zip compression.

The replication generation ID, a value used by replication to determine if replicas are compatible and can be replicated, will now be calculated in a way that is independent of the disk order in which the entries are stored. This is helpful when entries are imported into new servers instead of being initialized.

To increase password security when using the Directory REST API, we improved the sanitization of password-related data in API responses.

Improved server upgrade times by streamlining the post-upgrade stability checks.

To help avoid excessive memory pressure on a server running multiple processes, we reduced the JVM memory requirements for the export-ldif and backup command-line tools.

To help you manage your backup and restore times, the backup tool now displays a warning when you run it with the --compress flag on an encrypted backend.

dsreplication commands that produce an error are now archived to avoid being overwritten. In addition, the dsreplication command now logs subcommands in separate files.

Significantly improved the performance times of backup, restore, and online replica initialization processes.

Improved performance when making updates to static groups.

For Active Directory Sync sources, when setting the startpoint to end-of-changelog, extraneous data is no longer sent from the Active Directory server to the Sync server. With this update, setting the startpoint should be faster, particularly for slow networks.

Fixed an issue where an AggregatePTAhandler's subhandlers sometimes did not properly initialize on startup and threw a NullPointerException.

Fixed an issue where the server did not properly log the alternative authorization DN for multi-update extended operations that used proxied authorization.

Fixed an issue where running dsjavaproperties --initialize would append duplicate arguments to common.java-args in the java.properties file.

Fixed an issue where a cn=config does not exist error message would appear in the error logs after navigating to the status page of the administrative console.

Fixed an issue where running manage-profile generate-profile on an instance that had been upgraded from an earlier version would result in a profile that contained commands that were part of the upgrade, and could not be used to set up new installations.

Fixed an issue where the Dictionary password validator would sometimes incorrectly handle dictionary words contained as password substrings.

Fixed an issue where the Changelog Password Encryption plugin would not work properly in a replicated environment if a password was changed with a Password Modify extended operation.

Fixed an issue where an ldapsearch for rootDSE did not exclude the baseDNs that were specified in a client connection policy.

Fixed an issue where help text incorrectly suggested using the --force flag if unable to connect to the server properly when running dsreplication initialize.

Fixed an issue that could prevent the server from properly updating a user's password history for a password change if the request included the password update behavior request control, indicating that password history violations should be ignored. This control is designed to prevent the server from rejecting an attempt to change a user's password if the new password is already in the history, but it incorrectly caused the server to skip all password history processing for the update.

Fixed an issue that could cause the server to add two copies of the current password into the password history when setting a new password with the password modify extended operation. This did not affect password changes with a regular LDAP modify operation.

Fixed an issue where the server could incorrectly allow a user to set an empty password in cases where none of the configured password validators would have rejected an empty password.

Fixed a regression that was introduced in the 9.3.0.0 release to allow additional values for the allow-pre-encoded-passwords property in the password policy configuration. This issue only affects password policies stored outside of the server configuration in local DB backends, and only those policies that include the ds-cfg-allow-pre-encoded-passwords attribute.

This fix enables the server to recognize and properly interpret compacted values for the ds-cfg-allow-pre-encoded-passwords attribute when parsing a password policy definition contained in a local DB backend. When the password policy entry is retrieved, the attribute may still appear to have a corrupt value, as the value that is actually stored in the entry would still represent the compacted token rather than the logically equivalent Boolean value. Replacing the value of the ds-cfg-allow-pre-encoded-passwords attribute in affected entries with the appropriate value is the best way to address this issue.

Fixed an issue that could prevent replace modifications for attribute types with subordinate types from being properly applied.

Fixed an issue where the Configuration API treated SCIM patch operations with empty arrays as invalid. Now, the API resets configuration attributes for replace operations with an empty array and ignores add operations with an empty array.

Fixed an issue that could allow the server to continue processing a search operation for longer than the allowed time limit. The server would not check the time limit in the course of index processing to identify potential matching entries, and in certain cases where the server had to iterate across a very large number of index keys, the allowed time limit could be exceeded in that portion of the processing.

Fixed an issue where encryption settings were not initialized before initializing password policy components when running remove-defunct-server against servers configured with an AES256 password storage scheme.

Fixed an issue that caused the server to incorrectly include client certificate messages in the expensive operations log.

Fixed an issue where the absence of the request-handler-per-connection configuration property for LDAP Connection Handler objects resulted in a single request handler being unable to acknowledge incoming client requests for long-running TLS negotiations.

Fixed the check-replication-domains tool so that the --serverRoot argument is no longer required, and it defaults to the server's root directory.

Fixed a possible NullPointerException replication error that occurred when missing changes were found for a replica, but that replica did not exist on all servers.

Fixed an issue that could prevent an unsuccessful bind attempt from being properly counted toward account lockout for a user. If the user's account had been temporarily locked as a result of too many failed authentication attempts, and if the first bind attempt after that temporary lockout period had elapsed was also unsuccessful, then the act of clearing the elapsed temporary lockout prevented the new failure from being properly recorded.

Fixed a NullPointerException error where an alert or alarm was raised and one or more of the alert handlers was not configured. This most commonly happened when the server was being stopped.

We fixed an issue where, when using the create-sync-pipe-config command, the correlated attributes for Generic JDBC sync pipe destinations were a single string value. The attributes are now correctly split by commas.

Fixed an issue where syncing to an Active Directory sync destination could result in the destination rejecting operations if a DN map was not configured on the sync class, and if the operations included modifications to the unicodePwd attribute.

Fixed an issue with synchronizing the enabled attribute of a user in a PingOne destination. This issue only occurred when attempting to enable or disable a user in PingOne from the source server.

To create an attribute mapping that will modify the enablement status of a user in PingOne, use the dsconfig tool to create a constructed attribute mapping of the following form. This will ensure that the enabled attribute will always have a well-defined value, even if the source attribute is not present on an entry in the source server.

dsconfig create-attribute-mapping --type constructed --map-name mapName --mapping-name enabled --set conditional-value-pattern:'(sourceAttribute=*) : {sourceAttribute}' --set conditional-value-pattern:'(!(sourceAttribute=*)) : true'

Fixed an issue where a NullPointerException would be thrown when adding a sync server to a topology of two or more existing sync servers using manage-topology add-server.

Fixed an issue where Proxy would not accurately report the availability of backends added through automatic backend discovery.