Upgrade Considerations

Important considerations for upgrading to this version of PingDirectoryProxy Server

  • This release introduces significant changes to the way servers in a topology are configured with information about each other. Once a server has been upgraded from a pre-7.0 version to 7.0 or later, reverting to the previous version is not supported. Before beginning the upgrade process, make sure you have read and understood the Administration Guide's chapter "Upgrading the Server".

  • SCIM 2 error responses, including Config API error responses, now represent the "status" field as a JSON string rather than as a number. Clients written to expect the earlier version format will need to be updated. In particular, clients written using the SCIM 2 SDK for Java should upgrade to version 2.2.0 or higher.

  • The Administrative Console now uses server information found in the topology registry to populate its server selection control. If the Console is used to manage a legacy server that does not use the topology registry, then the server selection control will not be populated. To manage a different server, the administrator will need to log out of the Console and provide the other server's connection details from the login page.

What's New

These are new features for this release of PingDirectoryProxy Server

  • Simplified management tasks related to configuring servers in a large cluster topology or in an automated deployment. Most notably, servers can now be added to a cluster while other servers are offline.

  • Added management features for SSL/TLS certificates. The default certificates used in inter-server replication can be replaced; validation of client certificates for HTTPS-based services like the SCIM REST API can be configured; and you can reload from the trust store for HTTPS client certificates without restarting the server or the HTTP-based services.

  • Added support for these operating system versions: Ubuntu LTS 16.04, CentOS 7.4, RedHat Linux 7.4, SUSE Enterprise 12 SP3

Known Issues/Workarounds

The following are known issues in the current version of PingDirectoryProxy Server:

  • Simultaneous cloning multiple PingDirectoryProxy Servers, PingDataSync Servers, and PingDataGovernance Servers from another server of the same type is not currently possible. To create server instances that are identical to a master server, cloning must be performed one at a time.

Resolved Issues

The following issues have been resolved with this release of PingDirectoryProxy Server:

Ticket ID Description
DS-426

Added the ability to generate administrative alert notifications when a task starts running, when it completes successfully, or when it fails to complete successfully. Also added the ability to send an email message to a specified set of users when a task starts running or completes successfully, which complements the existing ability to send an email message when a task fails to complete successfully or when it completes with any state, regardless of success or failure.

DS-426

Added support for recurring tasks, which can be used to automatically invoke certain kinds of administrative tasks based on a specified schedule.

At present, only certain kinds of tasks can be scheduled as recurring tasks. This includes both backups and LDIF exports, each of which provides retention support to limit the amount of disk space that the backups and LDIF files consume. It also includes support for any kind of task in which each instance of the task should use exactly the same values for all of the task-specific attributes. The Server SDK also provides an API for creating custom third-party recurring task implementations.

DS-4406

Implemented invocation logging for several server tools, which will write to logs/tools/tool-invocation.log by default upon startup and shutdown. Some of the information recorded by log entries include the tool's start and completion times, the command-line arguments used to initialize them, and the name of the system account used to launch the tool. To modify this behavior, edit the config/tool-invocation-logging.properties file.

DS-4570,DS-14281,DS-14282,DS-14283, DS-14284,DS-17197,DS-17366

The admin backend and the tool used to manage it, dsframework, have been replaced by the topology registry and dsconfig, respectively. The topology registry is automatically mirrored across all servers in the topology, so administrative information is kept in-sync on all servers at all times.

DS-6970

Added support for encrypted logging, using a key generated from an encryption settings definition. Encrypted log files may be decrypted with the encrypt-file tool.

DS-12157,DS-35896

Made a number of improvements to backend backup and restore, and to LDIF export and import:

* Added the ability to encrypt backups and LDIF exports with a key generated from a user-supplied passphrase or with a key generated from an encryption settings definition. Previously, encrypted backups and LDIF exports only used a secret key that was known only to servers within the replication topology. The new options make it easier to restore encrypted backups and import encrypted LDIF files in servers outside of the replication topology. The encrypt-file utility can be used to decrypt encrypted backups and LDIF exports, regardless of how the encryption key was obtained.

* Added the ability to limit the rate at which backups and LDIF exports will be written to disk, which can help avoid performance problems that result from these operations saturating the disk subsystem.

* Added new global configuration properties for automatically encrypting backups and LDIF exports by default, which will be set to true if data encryption is enabled during setup.

* Added new global configuration properties that can specify which encryption settings definitions will be used to obtain the encryption keys for automatically encrypted backups and LDIF exports. If not specified, then the server will use its preferred encryption settings definition, or an internal topology key if no encryption settings definitions are available.

* Added a new configuration property for automatically compressing encrypted LDIF exports.

* Updated the backup tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the backup. Added a new --doNotEncrypt argument that can be used to force a backup to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the backup may be written to disk.

* Updated the restore tool to add new --promptForEncryptionPassphrase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted backup. For backups encrypted with an encryption settings definition or an internal topology key, the server will automatically be able to determine the correct key.

* Updated the export-ldif tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the export. Added a new --doNotEncrypt argument that can be used to force an LDIF export to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the LDIF file may be written to disk.

* Updated the import-ldif tool to add new --promptForEncryptionPasshprase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted LDIF export. The --isEncrypted and --isCompressed arguments are no longer necessary, as the tool can automatically detect encryption and compression (although those arguments are still available to preserve backward] compatibility), and it can automatically identify the correct key for exports encrypted with a key obtained from an encryption settings definition or an internal topology key.

DS-15223,DS-35895

Added the ability to configure data encryption during setup using a randomly generated key, a key generated from a user-supplied passphrase, or a key obtained from an export of another server's encryption settings database. When setting up multiple instances, providing the same encryption passphrase to each instance will ensure that all instances have the same encryption key.

The encryption-settings tool has also been updated to allow creating encryption settings definitions from a passphrase, to allow providing a description when creating a new encryption settings definition, and to record a create timestamp for new definitions. It is now possible to create ciphers that use the Galois Counter Mode (GCM) cipher mode (for example, using a cipher transformation of "AES/GCM/PKCS5Padding") for authenticated encryption. Definitions created with with just a cipher algorithm but no transformation will now use stronger settings.

The default encryption settings export format now provides stronger encryption. Newer server instances should be able to import encryption settings exported from other servers without issue. When exporting encryption settings for import into older servers, use the new --use-legacy-export-format argument.

DS-16508

Updated the dsconfig list subcommands to list objects of all complexity levels rather than requiring the --advanced flag to list advanced and expert objects.

DS-17176

Updated the entry balancing request processor's handling of add operations within an atomic multi-update extended operation. Now, backend servers are not searched for a pre-existing entry if the parent entry was already found to not exist while processing the multi-update request. This eliminates some redundant searches, which reduces the load on the backend servers.

DS-17883

Added an ldap-debugger tool that acts as a simple LDAP proxy between a client and a directory server and decodes all requests and responses that pass through it.

DS-17891

Added a new manage-certificates tool that can be used to perform a number of functions related to TLS certificate management.

DS-18204

Fixed a defect where a web application extension's base context path could be set to "/" with no name.

DS-35523

The update tool now enforces specification of a new product license when updating to a new major version. The license can be specified using the --licenseKeyFile command-line options, or by copying the license file to the top-level directory of the server package used to perform the update. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html, or contact sales@pingidentity.com.

DS-35536

Support for the IBM JDK has been retired.

DS-35576

Updated the JMX connection handler's monitor provider so that when a JMX connection is closed, it is removed from the list of established connections. After a JMX client disconnects, it may take the server a few minutes to detect the closure and update the monitor.

DS-35581

Updated the server to include an instance of the Periodic Stats Logger Plugin that is enabled out-of-the-box to aid in diagnosing support issues. The "Historical Stats Logger" plugin will log performance statistics to logs/monitor-history/historical-dsstats.csv every five minutes. This works in concert with the "Monitor History" plugin, which logs the full contents of cn=monitor to logs/monitor-history every five minutes. The tail of this csv file is automatically included in the output generated by collect-support-data.

DS-35583

Fixed a defect where configuring a Directory server on a Windows machine with a space in the home directory pathname would cause server setup to fail.

DS-35596

Updated the Directory Proxy Server to improve performance when detecting that a server has become unavailable, especially when communication is secured with TLS and TLS negotiation is stalled.

Also added a new health-check-connect-timeout configuration property for LDAP external servers. This property can be used to specify a shorter timeout when connecting to a server for the purpose of evaluating the health of the server than when creating a connection that will be used to forward client requests to that server. If a health-check-connect-timeout value is not configured, then the Directory Proxy Server will continue to use the value of the connect-timeout property for health check connections as well as for connections used to process client requests.

DS-35601

Added a new Monitor Entry for SSL Cipher Suite and Protocol information. It is available under cn=SSL Context,cn=monitor.

DS-35648

Added a missing double-quote to bat/transform-ldif.bat, which prevented the command from being invoked successfully on Windows systems.

DS-35697

Fixed an issue in which the Directory Proxy Server did not set the correct response timeout for forwarded bind operations. If a backend server did not respond to a bind request, the Directory Proxy Server would have waited for up to five minutes before trying another server or returning an error response to the client.

DS-35709

Changed enable-sub-operation-timer on the Global Configuration to be true by default. This exposes operation timing information in the Sub-Operation Timing Monitor and any Operation Timing Access Log Publishers that have been configured. Enabling this tracking has about a 3% impact on operation throughput and latency, which will not be noticeable in most deployments and is an acceptable tradeoff for understanding where operation processing time is spent. However, it can be explicitly set to false to turn this tracking off.

DS-35727,DS-35728

Updated setup to include key usage, extended key usage, and subject alternative name extensions in the self-signed certificates that it generates.

DS-35750

Added support for multiple client connection policies for sensitive attributes. Support for different sensitive attributes per client requires the use of multiple client connection policies with the same names on the Directory Server and the Proxy Server. When a client request is processed by a Proxy Server, the Directory Server looks for a policy in its own configuration with the same name as the one in the Proxy Server. The Directory Server then uses this policy rather than the one associated with the Proxy Server's connection.

DS-35868

The create-systemd-script command now suggests placing the script created in "/etc/systemd/system."

DS-35990

Provided the means to request that the server dynamically reload the certificate key and trust stores used by all HTTP connection handler instances that provide support for HTTPS. The request can be made using a new reload HTTP connection handler certificates task, the reload-http-connection-handler-certificates tool, or programmatically from a Server SDK extension using the ServerContext#reloadHTTPConnectionHandlerCertificates method.

DS-36025

Added a close-connections-when-unavailable property to the LDAP Connection Handler configuration. This allows a connection handler to be closed whenever the server sets an unavailable alert type, such as when backend data is unavailable. This should trigger clients to failover to another server. When the unavailable alert type is cleared, the connection handler is started again. When using this configuration setting, we recommend using two connection handlers: one for client traffic, with this option set to true, and one for administration and monitoring, with this option set to false. This allows the server to be visible to administrators but not to clients.

DS-36054

Added an encrypt-file tool that can encrypt and decrypt data with a user-supplied passphrase, an encryption settings definition, or a topology key shared among server instances. It includes support for decrypting the content in encrypted backups, LDIF exports, and log files.

DS-36070

Fixed an issue with compressed logging that could leave some data buffered in memory and not actually written out to disk until the logger is closed.

DS-36075

Updated tools that interact with log or LDIF files to support reading from input files that are compressed and encrypted and writing to compressed and encrypted output files.

DS-36088

In addition to specifying an exact set of desired cipher suites for the LDAP and HTTP Connection Handlers, administrators can now specify inclusions to, or exclusions from, the set of cipher suites selected by the server.

DS-36093

Added support for TLS1.2 with STARTLS to connect to an SMTP server.

DS-36093

Added support for TLS1.2 with STARTLS to connect to SMTP server

DS-36230

Improved the server's handling of DNs and RDNs that contain characters whose UTF-8 encodings require more than two bytes.

DS-36315

Updated the Directory Proxy Server so that if it encounters a problem while trying to follow a referral on behalf of the client (for example, if it can't establish a connection to the server indicated in the referral), it will include additional information about the failure in the access log message for that operation.

DS-36328

Updated the server to reduce contention when converting between strings and the bytes that comprise those strings.

DS-36360

Increased the default size of the queue used to hold alert notifications so they can be asynchronously processed by a background thread. This makes it less likely that the queue will become full if many alerts are generated in a short period of time, which would cause subsequent attempts to generate alerts to block while the server catches up. Also updated the server to log a message when the queue becomes full so that administrators will be aware of the problem and will have suggestions for addressing it.

DS-36377

Fixed an issue where a configuration change to enable a Delegated Administrator could be incorrectly rejected after a configuration change to the parent Delegated Admin Resource Type.

DS-36455

Updated the proxy server to discard a connection whenever an operation times out rather than reusing it. A new connection is then established. This avoids a cascading error condition when a network problem allows traffic from the proxy server to the directory server but not the reverse.

DS-36545

Added a sanitize option to the Monitor History Plugin that, if enabled, will redact the small amount of potentially personally identifiable information that could appear in search filters and LDAP DNs within the monitor. This makes it easier to share the monitor history files with the support team in secure environments.