Upgrade considerations

Important:

If you plan to upgrade servers using a mixed-version environment where one version is earlier than 7.0 and some of the servers are still using the admin backend while others have been updated to the topology registry, do not attempt to make size changes to the topology. You cannot remove any existing servers (using dsreplication disable) or add new servers (using dsreplication enable) when in this transitional state of partially-updated servers. When a topology has been completely migrated to a 7.0 or later version with the topology registry, changes to the topology size are allowed, even in mixed-version environments (for example, mixed 7.3 and 8.3).

For other upgrade considerations, see Upgrade overview and considerations.

Critical Fixes

This release of the PingDirectory Server addresses critical issues from earlier versions. Update all affected servers appropriately.

  • Fixed an issue that could cause the server to report an "Unable to decode a blacklist key" error while trying to open a local DB backend after an unclean shutdown.

    • Fixed in: 8.0.0.0
    • Introduced in: 7.2.0.0
    • Support identifiers: DS-40788

What's New

These are new features for this release of the PingDirectory Server:

  • Improved end user experience with several feature additions to user e-mail notifications. Now administrators can configure the server to send multi-part emails, including plain text and HTML. Also, in addition to changes to account state, such as when a user account is locked, administrators can configure notifications to send whenever any configured profile attributes are changed. For example, this can be used for confirmations of self-service account changes, or as notifications for changes made by administrators.
  • Better insight to server health and the performance of connected applications. Administrators can now push metrics to application insight and monitoring applications such as Splunk using two new methods. A new StatsD monitoring endpoint pushes metrics to StatsD-compatible services. Also, the periodic stats logger has been updated to use JSON-format log files, greatly simplifying the use of log forwarding tools like Splunk's Universal Forwarder or Elastic's FileBeats.
  • Several password-related improvements. A new password validator has been added that rejects passwords known to have been breached or stolen, as collected by the haveibeenpwned.com service. Also, administrators can take advantage of a new Argon2 algorithm and password storage scheme, as well as new options to the PBKDF2 algorithm and password storage scheme.
  • New SCIMv2 REST API to create, read, update, and delete (CRUD) users and other resources using JSON over HTTP. This complements the Directory REST API introduced in 7.2. While the Directory REST API is intended for custom application development, the SCIMv2 REST API is more intended for integrations with third party software and services. With SCIMv2, administrators configure the data conventions used in PingDirectory Server – resource types that map to directory object classes, their attributes, and their locations within the Directory Information Tree (DIT).
  • A new tool to easily update the security certificates used by the server. The replace-certificate tool allows administrators to replace the certificates presented by the server for external client connections, both HTTPS and LDAPS. If required, the replace-certificate also allows the administrator to replace the certificates used for inter-server communication, like replication.

Known Issues/Workarounds

The following are known issues in this version of the PingDirectory Server:

  • The following are suggested solutions for problems with slow DNS:
    • Maintain a connection pool in the client app rather than opening new connections for each bind.
    • Add appropriate records, including PTR records, to DNS.
    • Add options timeout:1 in the /etc/resolv.conf file and/or options single-request.
    • If IPv6 requests specifically are causing issues, add -Djava.net.preferIPv4Stack=true to the start-server.java-args line in PingDirectory Server’s config/java.properties file, run bin/dsjavaproperties, and restart the server to stop the issuance of IPv6 PTR requests.
  • SCIM2 PATCH operations require ACI read permissions for any required attributes in the resource's SCIM schema, even if the operation itself does not modify the required attribute itself. Otherwise, PATCH operations will fail with the error message "Applying patch ops results in an invalid resource". The following is an example of an ACI to grant read permission to the required attributes for pass-through SCIM resource types of objectClass inetOrgPerson for a particular oauth scope:
    • aci: (targetfilter="(objectClass=inetOrgPerson)")(targetattr="objectClass||uid||cn||givenName||sn")(version 3.0; acl "Allow read access to inetOrgPerson required attributes with OAuth scope inetOrgPersonScope"; allow (read) oauthscope="inetOrgPersonScope";)
  • Some server tools, such as dsreplication, collect-support-data, and rebuild-index, will fail with errors if they are run with an encrypted tools.properties file.
    • Workaround: Add the --noPropertiesFile argument to the server tools to prevent them from pulling information from the encrypted file.
  • The working directory value used by exec tasks is not implemented for recurring exec tasks.
  • Deploying the Admin Console to an external container using JDK 11 requires downloading the following dependencies and making them available at runtime (for example, by copying them to the WEB-INF/lib directory of the exploded WAR file).
    • groupId:jakarta.xml.bind, artifactId:jakarta.xml.bind-api, version:2.3.2
    • groupId:org.glassfish.jaxb, artifactId:jaxb-runtime, version:2.3.2
    Workaround: Deploy the Console in an external container using JDK 8.

Resolved Issues

The following issues have been resolved with this release of the PingDirectory Server:

Ticket ID Description
DS-17278 Added a cn=Server Status Timeline,cn=monitor monitor entry to track a history of the local server's last 100 status changes and their timestamps. Updated the LDAP external server monitor to include attributes tracking health check state changes for external servers. The new attributes include the number of times a health check transition has occurred, timestamps of the most recent transitions, and messages associated with the most recent transitions.
DS-37042 Added an account status notification handler that can send multi-part email messages (including a plain-text part, HTML part, and optional attachments) in response to notable events affecting user accounts.
DS-37745 Added support for an oauthscope access control bind rule, which can be used to grant or deny access control permissions based on a set of scopes associated with an OAuth token.
DS-37859 Added the --exportImportLocalDBData option to the revert-update tool. This option will perform an export and re-import of any local db backends if there are changes that require it between the two versions. The new option does not handle the LDAP changelog and replication changelog.
DS-37881 The PingFederate Access Token Validator will now refresh its cached value of the PingFederate server's token introspection endpoint. A new attribute, endpoint-cache-refresh, has been added to the PingFederate Access Token Validator, which will determine how often this refresh occurs.
DS-37955 To support multiple trace loggers, each trace logger now has its own resource key, which is shown in the Resource column in the output of status. This key allows multiple alarms, due to sensitive message types for multiple trace loggers.
DS-38053 The JWT Access Token Validator no longer requires a restart after a change to one of its signing certificates.
DS-38243 Added a password validator that can use the Pwned Passwords service (https://haveibeenpwned.com/) to reject passwords that are known to have been compromised in data breaches.
DS-38371 Added a password storage scheme that uses the Argon2i password hashing algorithm. This scheme requires the free and open source Bouncy Castle library cryptographic, which is not included with the server. This library must be obtained from https://bouncycastle.org/ and placed in the server lib directory before the storage scheme may be used.
DS-38528 Added support for a generate password request control that can be included in an add request to request that the server generate a password for the new entry. The generated password will be provided to the client in a corresponding response control.
DS-38560 Updated manage-profile replace-profile to apply configuration changes directly, when possible. If the new server profile used by replace-profile has changed only the dsconfig batch files from the original profile, then only the dsconfig files are applied. If no changes are detected between profiles, replace-profile takes no action. If changes other than dsconfig are detected, the full replace-profile process is followed.
DS-38699 Avoid sending replication messages for LDAP operations where the required associated replication information is missing.
DS-38777 Added support for updating the server version during manage-profile replace-profile. The server must have been originally set up with a server profile.
DS-38832 Fixed an issue that could cause the server to leak a small amount of memory each time it failed to establish an LDAP connection to another server.
DS-38863 Updated the manage-profile setup subcommand to set a server's cluster name to match its instance name by default. This prevents servers in the same replication topology from being in the same cluster, reducing the risk of unintentionally overwriting parts of an existing server's configuration in a DevOps environment. The --useDefaultClusterName argument can be used to leave the cluster name unchanged.
DS-38867

Updated the PBKDF2 password storage scheme to add support for variants that use the 256-bit, 384-bit, and 512-bit SHA-2 digest algorithms. At present, the SHA-1 variant remains the default to preserve backward compatibility with older versions.

Also, in accordance with the recommendations in NIST SP 800-63B, we have increased the default iteration count from 4096 to 10,000, and the default salt length from 64 bits to 128 bits.

DS-38869 Updated the remove-defunct-server tool's --ignoreOnline option. When using --ignoreOnline in a mixed-version environment, all servers must support the option.
DS-39099 Fixed an issue where error log notifications were generated regardless of whether the account status notification handler has the property enabled or disabled.
DS-39176, DS-39308

Updated the Groovy scripting language version to 2.5.7. For a list of changes, visit groovy-lang.org and view the Groovy 2.5 release notes.

As of this release, only the core Groovy runtime and the groovy-json module are bundled with the server. To deploy a Groovy-scripted Server SDK extension that requires a Groovy module not bundled with the server, such as groovy-xml or groovy-sql, download the appropriate jar file from groovy-lang.org and place it in the server's lib/extensions directory.

DS-39253 Added a replace-certificate tool, which can help an administrator replace the listener or inter-server certificate for a server instance.
DS-39325
Removed the legacy product-specific scripts for starting and stopping the server. These include:
  • start-ds and stop-ds for PingDirectory Server
  • start-proxy and stop-proxy for PingDirectoryProxy Server
  • start-sync and stop-sync for Data Sync Server
  • start-metrics-engine and stop-metrics-engine for Data Metrics Server

These legacy scripts had been deprecated for several releases in favor of the more general start-server and stop-server scripts, and they displayed a warning message about their pending removal if they were invoked.

If you still have dependencies on these legacy product-specific scripts, you will need to update them to reference the general start-server and stop-server scripts instead. If it is not feasible to update these references immediately, you may create symbolic links that use the legacy script names and point at the start-server and stop-server scripts.

DS-39347 Fixed an issue where Delegated Admin would not work properly if the name of the REST Resource Type was not the same as the resource endpoint.
DS-39373 Preserve the privileges that are explicitly set on the admin user when migrating from the admin backend to the topology registry.
DS-39518 Fixed an issue in which escaped characters in schema extensions may not be handled properly. If used in attribute type constraints (such as X-VALUE-REGEX), this could cause unexpected or incorrect behavior.
DS-39525, DS-39526
Delegated Admin enhancements for constructed attributes.
  • Allow a required attribute to be read-only if it is constructed.
  • Add a configured list of "Update Constructed Attributes" on the REST resource type, similar to the "Post Create Constructed Attributes", so that constructed attributes can be updated when dependent attributes change.
  • Handle constructed attributes which reference other constructed attributes.
DS-39589 Fixed an issue that could cause access log messages for bind and StartTLS results to report the client connection policy that had previously been assigned to the connection rather than the new policy that is in place as a result of the operation.
DS-39592 HTTP External Servers have a new attribute, ssl-cert-nickname, which defines the alias of a specific certificate within their keystore to be used as a client certificate.
DS-39603 Fixed an issue where Server SDK extensions could not be configured by dsconfig batch files in the manage-profile tool.
DS-39626, DS-40357 The trace log publisher will now record an access token's scopes after the token is successfully validated.
DS-39654 Added support for the --topologyFilePath argument to the manage-topology add-server subcommand.
DS-39671 Updated the manage-topology add-server subcommand to require being run from the older server in a mixed-version environment.
DS-39693 Fixed an issue where Delegated Admin search results were truncated and invalid upon encountering a Directory entry containing a Boolean or Integer syntax attribute whose values were invalid because they did not conform to the appropriate syntax. With this fix, the offending values are omitted from the results and a warning message is logged to the server errors log.
DS-39715 Updated the Server SDK to add support for sending email messages.
DS-39762 Added support for a "generate password" extended operation that can be used to request that the server generate one or more passwords that may be suggested as possible values when creating a new user or changing the password for an existing user.
DS-39787 Fix verify-index errors on multibyte UTF-8 strings.
DS-39796 Added support for two new account status notification types. The account-created notification type can be used to generate a notification whenever a new entry is created in an add request that matches a given set of criteria. The account-updated notification type can be used to generate a notification whenever an entry is updated in a modify or modify DN request that matches a given set of criteria.
DS-39797
Updated the account status notification handler framework to make several new properties available for use in constructing notification messages. These properties provide information about the password policy configuration and the user's password policy state, including:
  • The time the notification was generated
  • The DN of the account with which the notification is associated
  • The time the account password was last changed
  • Whether the account is in a usable state and information about any issues that might affect its usability
  • Whether the account is administratively disabled
  • Whether the account is expired or not yet active
  • Information about any account lockout that may be in effect
  • Information about password expiration
  • Whether a new password was generated for a self-change or administrative reset
DS-39857 Added the StatsD monitoring endpoint. When the Stats Collector Plugin is enabled, this endpoint sends metric data from the server in StatsD format to the configured destination.
DS-39872 Updated Summarize Access Log Rotation Listener and Copy Log File Rotation Listener to run on a background thread.
DS-39873 Fixed an issue that allowed replicated subtree deletes to cause OutOfMemory errors on replicas. Also fixed a related issue that would cause the replication log to fill up with mild errors.
DS-39882 Fixed an issue that could prevent the simple request criteria from properly evaluating a target entry filter or group membership for a modify DN operation after the change had already been applied to the backend.
DS-39908 Added a new JVM-default trust manager provider that can be used to automatically trust any certificate signed by an authority included in the JVM's default set of trusted issuers. Also, updated other trust manager providers to offer an option to use the JVM-default trust addition to the trust that they normally provide.
DS-40114 Added a new cn=Status Health Summary,cn=monitor monitor entry that provides a summary of the server's current assessment of its health. This simplifies monitoring with third party tools that support retrieving monitoring data over JMX. The Periodic Stats Logger has also been updated to allow some of this monitoring information to be logged. No new information is logged by default.
DS-40154 Fixed an issue where the restore tool was not restoring all dependencies of an incremental backup.
DS-40177 Fixed an issue that could interfere with the server's ability to purge an earlier, full backup when the following conditions are satisfied: Retention is used. Multiple levels of incremental backups depend on the full backup."
DS-40210 Fixed an issue that prevented uniqueMembers from adding themselves to a groupOfUniqueNames.
DS-40249 Fix an issue where an LDAP search across entry-balanced server sets sometimes returned 0 (success) even though all servers in one of the sets failed with a timeout. The search should return 52 (unavailable) in this situation.
DS-40317 Created a new function that takes index objects rather than the counts so that we can avoid adding to a long.max_value, creating a long overflow.
DS-40354 Fixed a problem with config-diff when writing properties that span multiple lines using the --prettyPrint argument.
DS-40366 Fixed an issue where the server was attempting to connect by an IP address rather than a hostname when DNS lookup was successful.
DS-40377 Added support for logging to a JSON file in the Periodic Stats Logger Plugin.
DS-40409 Updated the ldif-diff tool so that it provides a --stripTrailingSpaces option that can cause the LDIF parser to strip off illegal trailing spaces rather than reject the associated entry or change record.
DS-40517 Added metrics for status summary, replication database, and LDAP changelog to the Stats Collector Plugin.
DS-40543 Updated manage-profile replace-profile to copy the tool log file to the server being updated.
DS-40556 Added support for specifying a working directory for exec tasks.
DS-40561 Fixed an issue that prevented assured replication from working for requests received via SCIM or the REST API.
DS-40674 Added the --addMissingRdnAttributes argument to manage-profile setup. This argument can be used when including LDIF files in the server profile. It will automatically add any missing RDN values to the set of entry attributes when they are not already present.
DS-40730 Updated the encrypt-file tool to prevent using the same path for both the input file and the output file.
DS-40744 Fixed an issue with the interaction between deprecated password storage schemes and forced password reset. If a user's password is reset by an administrator using a password storage scheme that is subsequently configured as deprecated, the act of re-encoding the password with the new default scheme would have incorrectly cleared the password reset flag.
DS-40771 Added a --duration argument to collect-support-data. When used, only the log files covering the specified duration before the current time will be collected.
DS-40788 Fixed an issue that could cause the server to report an "Unable to decode a blacklist key" error while trying to open a local DB backend after an unclean shutdown.
DS-40799 Fixed an issue in which an account that had been temporarily locked after too many failed authentication attempts could become re-locked with fewer than the expected number of subsequent failed attempts after the previous lockout period had elapsed.