Upgrade Considerations
Important considerations for upgrading to this version of the Data Metrics Server
-
If you have upgraded a server that is in a cluster (i.e., has a cluster name set in the Server Instance configuration object) to version 8.1, you will not be able to make cluster configuration changes until all servers with the same cluster name have been upgraded to version 8.1. If needed, you could create temporary clusters based on server versions and modify each of the servers' cluster name appropriately to minimize the impact while you are upgrading.
- Updated the "delay bind response" failure lockout action to provide an option to delay the response to bind requests initiated by non-LDAP clients (for example, when using HTTP basic authentication). This option is disabled by default because delaying the bind response for non-LDAP clients may require temporarily blocking the thread used to process the request, which could increase the risk of a denial-of-service attack. To help mitigate this risk, if you enable delayed bind responses for non LDAP clients, we recommend that you also increase the number of request handler threads for all enabled HTTP connection handlers.
- Updated setup to create a second encryption settings definition if data encryption is enabled. It will continue to create a definition for 128-bit AES encryption for use as the preferred definition to preserve backward compatibility with existing servers in the topology, but it will now also generate a definition for 256-bit AES encryption. The 256-bit AES definition may become the preferred definition in a future release, but you can use it now by first ensuring that any existing instances are updated to contain the new definition (with the "encryption-settings export" and "encryption-settings import" commands) and then making it the preferred definition (with "encryption-settings set-preferred") in all instances.
What's New
These are new features for this release of the Data Metrics Server
-
In an ongoing effort to improve the use of containers for PingDirectory, several features have been implemented:
- The --outputFile option has been added to the collect-support-data tool. You can now specify either a path, a file name, or a path and file name for the resulting CSD file. This means an administrator can run the collect-support-data tool and put the output file into a directory outside of the container, allowing access to the file without having to actually connect to the container.
- The collect-support-data tool can now be run as a recurring task. Recurring tasks can be created using the Administration console which means that administrators do not have to connect to the container in order to run the tool.
- A Collect Support Tool Extended Operation has been added allowing LDAP clients to initiate the collect-support-data tool and to receive the output of the request. The LDAP SDK has been updated to support this, and the --remoteServer added to the collect-support-data tool can be used to send the request to another server. In other words, you can now run collect-support-data on the command line and reference another server, possibly in a container, and retrieve the output file remotely.
-
PingDirectory has a Consent REST API that allows users to create and store consents. A new feature now allows users to search for consents that have been granted to them by another party.
Known Issues/Workarounds
The following are known issues in the current version of the Data Metrics Server
-
Several known issues can occur when you use the Administrative Console with Tomcat 9.0.31. You can resolve these issues by upgrading to Tomcat 9.0.33 or later.
- If you use the create-systemd-script tool to create a forking systemd service, the service is stopped by the "systemctl stop ping-directory.service" command. At that time, you can see the status using the "systemctl status ping-directory.service" command. That status might contain an indication of failure: "Active: failed (Result: exit-code)". This error has to do with the way the service exits. It is harmless.
Resolved Issues
The following issues have been resolved with this release of the Data Metrics Server:
Ticket ID | Description |
---|---|
DS-5143, DS-11035 |
Updated support for logging access and error log messages to a syslog server. While the server previously supported logging these messages to a syslog server (through the "syslog-based access log publisher" and "syslog-based error log publisher" logger implementations), these loggers used an older version of the syslog protocol (described in RFC 3164) and only offered support for communicating over UDP. These loggers are still available for legacy backward compatibility, but we now also offer new "syslog text access log publisher" and "syslog text error log publisher" implementations that use a newer version of the syslog protocol (syslog version 1, described in RFC 5424) and support communicating over UDP or the more reliable TCP. When using TCP, it is also possible to encrypt communication with TLS and to configure multiple servers for better redundancy. These loggers use the same space-delimited text format as the former loggers. We also offer new "syslog JSON access log publisher" and "syslog JSON error log publisher" implementations that offer the same set of capabilities, but that format the message text as JSON objects, which can be more easily parsed by third-party software. |
DS-10320, DS-12550, DS-12551, DS-12552, DS-42116, DS-42162, DS-42179, DS-42222, DS-42223, DS-42224, DS-42225, DS-42416, DS-42437 |
Added a config/sample-dsconfig-batch-files directory with set of well commented dsconfig batch files that may be useful in enabling or configuring a variety of features in the server. |
DS-10775 |
Updated the dictionary password validator to support additional options:
|
DS-11524, DS-41860, DS-42112 |
Added support for new administrative alert types.
|
DS-13853 |
Added support for the OAUTHBEARER SASL mechanism (as described in RFC 7628) to allow LDAP clients to authenticate with OAuth 2.0 bearer tokens. |
DS-15848, DS-42360 |
Added support for invoking a specified set of password validators during bind operations. If the password used to authenticate fails to satisfy one or more of the configured validators, the bind attempt can be rejected, the user can be forced to change their password, or the server can generate an account status notification to take some alternative action (for example, notifying the end user or server administrators). |
DS-15864 |
Replaced the ldappasswordmodify tool with a new version that offers more functionality, including support for additional controls, support for multiple password change methods (the password modify extended operation, a regular LDAP modify operation, or an Active Directory-specific modify operation), and the ability to generate the new password on the client. |
DS-17903 |
Updated setup to provide a --populateToolPropertiesFile argument that will allow it to populate the config/tools.properties file with default values for command-line tool arguments. If requested, properties will be provided for the server address, port, and communication security, and may also include a default bind DN and optionally a bind password. When running setup interactively, it will now prompt to determine which properties (if any) should be populated in the properties file. |
DS-38110 |
Updated the System Information monitor with an "isDocker" attribute to identify if the server is running in a Docker container. |
DS-38118, DS-42495 |
Made several updates related to the server's handling of data written to
standard output and standard error:
|
DS-38868 |
Updated setup to create a second encryption settings definition if data encryption is enabled. It will continue to create a definition for 128-bit AES encryption for use as the preferred definition to preserve backward compatibility with existing servers in the topology, but it will now also generate a definition for 256-bit AES encryption. The 256-bit AES definition may become the preferred definition in a future release, but you can use it now by first ensuring that any existing instances are updated to contain the new definition (with the "encryption-settings export" and "encryption-settings import" commands) and then making it the preferred definition (with "encryption-settings set-preferred") in all instances. |
DS-39789 |
Updated the JVM memory usage monitor provider to fix an issue that could prevent the monitor from reporting the total amount of memory held by all memory consumers. Also, fixed an issue that could cause the memory-consumer attribute to use an incomplete message for consumers without a defined maximum size and added an additional memory-consumer-json attribute whose values are JSON objects with data that can be more easily extracted by automated processes. |
DS-40650 |
Updated the collect-support-data tool to make it possible to specify how much data should be captured from the beginning and end of each log file to include in the support data archive. You can also specify the capture size when invoking the tool through an administrative task, recurring task, or extended operation. |
DS-40967 |
Eliminated a misleading error message that could be logged at startup if the server was configured with one or more ACIs that only apply when using specific SASL mechanisms. |
DS-41989 |
Fixed an issue that could result in duplicate column headers being produced by the Periodic Stats Logger, even when the header-prefix-per-column attribute was set to true. |
DS-42045 |
Updated the Stats Collector Plugin with a new generate-collector-files configuration property. When using this plugin with the Data Metrics Server, leave this property at its default value of true. When using the plugin exclusively for providing metrics to one or more StatsD Monitoring Endpoints, set this property to false to prevent unnecessary I/O. |
DS-42059, DS-42060 |
Updated setup to add options for improving communication security:
The --rejectInsecureRequests and --rejectUnauthenticatedRequests arguments can also be used with manage-profile by including them in the setup-arguments.txt file of the server profile. |
DS-42061 |
Updated the interactive command-line tool framework to prefer establishing secure LDAP connections over insecure connections. Previously, when prompting for the information needed to establish a connection, the default option was to create an unencrypted LDAP connection. Now, tools default to creating an SSL-encrypted connection if the server supports it, or to creating a StartTLS-encrypted connection if that is available but SSL is not. Tools also default to using streamlined settings when establishing secure connections. Previously, they would always prompt about how to determine whether the server's certificate chain should be trusted. When using the streamlined settings, the tools only prompt about certificates that cannot automatically be considered trusted using information in the JVM's default trust store, the server's default trust store (config/truststore), or the server's topology registry. |
DS-42062 |
Updated the root password policy so that LDAP bind responses for root users and topology administrators will be delayed by one second after five consecutive failed authentication attempts. |
DS-42063 |
Updated the "delay bind response" failure lockout action to provide an option to delay the response to bind requests initiated by non-LDAP clients (for example, when using HTTP basic authentication). This option is disabled by default because delaying the bind response for non-LDAP clients may require temporarily blocking the thread used to process the request, which could increase the risk of a denial-of-service attack. To help mitigate this risk, if you enable delayed bind responses for non-LDAP clients, we recommend that you also increase the number of request handler threads for all enabled HTTP connection handlers. |
DS-42115 |
Updated the server's command-line tool framework to make it easier and more convenient to communicate with the server over a secure connection when no trust-related arguments are provided. Most non-interactive tools will now check the server's default trust store, the topology registry, and the JVM's default trust store to see if the presented certificate chain can be automatically trusted without the need to prompt the user. If the presented chain cannot be automatically trusted, the user may be interactively prompted to determine whether it should be trusted. |
DS-42279 |
Updated the server to require a minimum key size of 2048 bits when negotiating a TLS cipher suite that uses ephemeral Diffie-Hellman key exchange. |
DS-42298 |
Replaced the ldifsearch, ldifmodify, and ldif-diff command-line tools with more full-featured and robust implementations. |
DS-42331 |
Replaced the ldapcompare tool with a new version that offers more functionality, including support for multiple compare assertions, following referrals, additional controls, and multiple output formats (including tab-delimited text, CSV, and JSON). |
DS-42347 |
Updated the server to use /dev/urandom (on non-Windows systems where that path exists and is readable) instead of /dev/random as the primary source for secure random data. Attempts to read from /dev/random can block if the underlying system does not have sufficient entropy, which can have a severe adverse effect on performance. Reads from /dev/urandom will not block, and the data that it provides is no less secure than data from /dev/random in any way that matters for the server. |
DS-42504 |
Updated manage-profile replace-profile to set encryption settings definitions defined in the newer server profile as preferred in the encryption settings db. |
DS-42547 | Fixed an issue where manage-profile generate-profile would print null as the generated profile directory when writing to an existing directory. |
DS-42609 |
Fixed an issue in which the Directory REST API could fail to decode certain credentials when using basic authentication. |
DS-42632 |
Added support for creating or importing a key pair configuration object using an elliptic curve (EC) key algorithm. You can use this to designate the encryption key pair for a JWT access token validator that handles EC-encrypted access tokens. |
DS-42634 |
The JWT Access Token Validator can now validate JWT access tokens signed using the elliptic curve digital signature algorithms ES256, ES384, and ES512. |
DS-42635 |
The JWT Access Token Validator can now validate JWT access tokens encrypted using elliptic curve cryptographic algorithms. The following key encryption algorithms are now supported in addition to RSA-OAEP: ECDH-ES, ECDH-ES+A128KW, ECDH-ES+A192KW, and ECDH-ES+A256KW. To support best practices for JWT security, you must now also configure the JWT Access Token Validator with explicit allow lists for key encryption and content encryption algorithms. For backward compatibility, the key encryption allow list defaults to RSA-OAEP, while the content encryption allow list defaults to A128CBC-HS256, A192CBC-HS384, and A256CBC-HS512. We recommend setting both allow lists to the strict minimum set of algorithms needed by the Access Token Validator. |
DS-42651 |
Updated the manage-profile replace-profile subcommand to better support updating the server's keystore and truststore files. When using the --generateSelfSignedCertificate argument in a server profile's setup-arguments.txt file, the server maintains the original keystore and truststore files during replace-profile. Otherwise, replace-profile uses the keystore and truststore specified in the profile's setup-arguments.txt file. |
DS-42667 |
Updated the server to set a unique cluster name when started for the first time. |
DS-42669, DS-42748 |
Updated the online dsconfig step of the manage-profile replace-profile subcommand to support getting LDAP connection arguments from a tools.properties file on the server being updated. Fixed an issue where boolean LDAP connection arguments like --useSSL and --trustAll would cause manage-profile replace-profile to fail when applying dsconfig online. |
DS-42681, DS-42684 |
Performance statistics generated by the Sideband API can now be published by the Periodic Stats Logger. To enable this, use the "included-http-servlet-stat" property of the Periodic Stats Logger. |
DS-42673 | Updated the manage-profile setup subcommand to fail if the start-server command has a non-zero exit code. |
DS-42687 |
Upgrade to Jetty 9.4.30. |
DS-42740 |
Fixed an issue where the dsconfig list subcommand would not display requested properties. |
DS-42749 |
To support best practices for JWT security, you must now configure the JWT Access Token Validator with an explicit list of the JWT signing algorithms that it accepts. For backward compatibility, this list defaults to the RSA signing algorithms RS256, RS384, and RS512, but we recommend setting this list to the strict minimum set of signing algorithms needed by the Access Token Validator. |
DS-42751 |
Added new override-status-code and additional-response-contents attributes to the Availability State HTTP Servlet Extension. These new attributes can be used to customize the response code and JSON response body of the servlet. |
DS-42861 |
Updated the manage-profile tool logs to include the duration of each step the tool takes. The new --verbose argument can also be used to display timing information in the tool's console output. |
DS-42886 |
Updated non-interactive setup (including manage-profile setup) to allow the password for the initial root user to be provided in pre-encoded form using the PBKDF2, SSHA256, SSHA384, or SSHA512 password storage scheme. This eliminates the need to have access to the clear-text password when setting up the server. |
DS-42926 |
Fixed an issue where Ping Directory products configured to run as Microsoft Windows services were sometimes unable to automatically restart following an unplanned reboot, due to errors reading a corrupted server status file. |
DS-42952 |
For Windows only, there can be a hang on start when global configuration
property |
DS-42963 | Updated the manage-profile generate-profile subcommand to ignore files larger than 100 megabytes when generating a server profile. Fixed an issue where many large files in the server root could cause the tool to run out of memory. |
DS-43376 |
Updated log publisher logic to reduce the amount of CPU that the server consumes when it is idle. |
DS-43480 |
Updated the system information monitor provider to restrict the set of environment variables that can be included. Previously, the monitor entry included information about all defined environment variables, as that information can be useful for diagnostic purposes. However, some deployments might include credentials, secret keys, or other sensitive information in environment variables, and that should not be exposed in the monitor. The server now only includes values from a predefined set of environment variables that are expected to be the most useful for troubleshooting problems and that are not expected to contain sensitive information. |
DS-43517 |
Updated the jose4j library used for JWT signing and encryption to version 0.7.2. |
DS-43651 |
The Security Guide is now available online at pingidentity.com. The guide has been removed from the server packaging. |