Enhancements

The new feature for this release is:

  • The Administrative Console now supports using OpenID Connect for admin SSO, allowing you to set up the PingOne administration console to have one-click SSO access without typing a password.

Upgrade considerations

Upgrade considerations are no longer part of the release notes. That information is now in Upgrade overview and considerations.

Known issues and limitations

The following are known issues in the current version of Data Sync Server:

When signing on to the Administrative Console, you must specify an LDAPS port in the Server field. For example, to use port 636, you would specify pingdirectory:636. If you do not specify a port, you get a
"Server
      unavailable"
message.

Resolved Issues

The following issues have been resolved with this release of the Data Sync Server:

Ticket ID Description
DS-5040

Improved the dsframework tool to support multivalued server properties.

DS-5143, DS-11035

Updated support for logging access and error log messages to a syslog server. While the server previously supported logging these messages to a syslog server (through the "syslog-based access log publisher" and "syslog-based error log publisher" logger implementations), these loggers used an older version of the syslog protocol (described in RFC 3164) and only offered support for communicating over UDP.

These loggers are still available for legacy backward compatibility, but we now also offer new "syslog text access log publisher" and "syslog text error log publisher" implementations that use a newer version of the syslog protocol (syslog version 1, described in RFC 5424) and support communicating over UDP or the more reliable TCP. When using TCP, it is also possible to encrypt communication with TLS, and it is possible to configure multiple servers for better redundancy. These loggers use the same space-delimited text format as the former loggers.

We also offer new "syslog JSON access log publisher" and "syslog JSON error log publisher" implementations that offer the same set of capabilities, but that format the message text as JSON objects, which can be more easily parsed by third-party software.

DS-10320, DS-12550, DS-12551, DS-12552, DS-42116, DS-42162, DS-42179, DS-42222, DS-42223, DS-42224, DS-42225, DS-42416, DS-42437

Added a config/sample-dsconfig-batch-files directory with set of well commented dsconfig batch files that may be useful in enabling or configuring a variety of features in the server.

DS-10775

Updated the dictionary password validator to support additional options:

* It can now ignore non-alphabetic characters that appear at the beginning or end of the password before checking the dictionary.

* It can strip characters of diacritical marks, including accents, cedillas, circumflexes, diaereses, tildes, and umlauts, before checking the dictionary. If this option is used, then any character with such a mark will be replaced with a base version of the character without that mark (for example, a lowercase letter n with a tilde over it would be replaced with just a lowercase letter n).

* You can define maps with information about character substitutions to use for checking alternative versions of the provided password. For example, if you indicate that "0" might map to "o", "1" or "!" might map to "i", "7" might map to "t", and "3" might map to "e", then the validator can reject a proposed password of "pr0h1b!73d" if the dictionary contains the word "prohibited".

* It can reject a proposed password if a value from the provided dictionary makes up more than a specified percentage of that password.

DS-11524, DS-41860, DS-42112

Added support for new administrative alert types:

* We have added a new admin alert account status notification handler, which can generate administrative alerts whenever an applicable account status notification is generated within the server. For example, this account status notification handler can be added to the root password policy to generate an alert whenever a root user's password is updated or their account is locked as a result of too many failed authentication attempts. A separate alert type has been defined for each account status notification type.

* We have added a new "privilege-assigned" administrative alert that can be raised whenever a new entry is added or an existing entry is updated to include one or more privileges.

* We have added a new "insecure-request-rejected" administrative alert that can be raised whenever the server rejects a request as a result of the reject-insecure-requests global configuration property.

DS-13853

Added support for the OAUTHBEARER SASL mechanism (as described in RFC 7628) to allow LDAP clients to authenticate with OAuth 2.0 bearer tokens.

DS-15848, DS-42360

Added support for invoking a specified set of password validators during bind operations. If the password used to authenticate fails to satisfy one or more of the configured validators, the bind attempt can be rejected, the user can be forced to change their password, or the server can generate an account status notification to take some alternative action (for example, notifying the end user or server administrators).

DS-15864

Replaced the ldappasswordmodify tool with a new version that offers more functionality, including support for additional controls, support for multiple password change methods (the password modify extended operation, a regular LDAP modify operation, or an Active Directory-specific modify operation), and the ability to generate the new password on the client.

DS-16228

Added support for JSON-formatted Sync loggers and Sync Failed Ops loggers, which complement the existing file-based Sync logger and file- based Sync Failed Ops logger, respectively. The JSON-formatted Sync log messages provide information about synchronization events the server has processed, and can be written to files on the local filesystem, written to the JVM's standard output or standard error stream, or sent to a syslog server.

DS-17903

Updated setup to provide a --populateToolPropertiesFile argument that will allow it to populate the config/tools.properties file with default values for command-line tool arguments. If requested, properties will be provided for the server address, port, and communication security, and may also include a default bind DN and optionally a bind password. When running setup interactively, it will now prompt to determine which properties (if any) should be populated in the properties file.

DS-36088

Updated the crypto manager to make it possible to augment the set of enabled TLS cipher suites with specific suites to add to or remove from the default set of enabled suites. To enable one or more suites in addition to those in the default set, prefix the names of those suites with the "+" symbol. To disable one or more suites in the default set of enabled suites, prefix the names of those suites with the "-" symbol. This was already possible when configuring cipher suites for the LDAP and HTTP connection handlers, but it was not an option for the crypto manager.

DS-37890

Added a new property to sync classes called replace-all-attr-values- limit. The property is only used when replace-all-attr-values is true and the sync destination is an LDAP server. If a modification would replace more values of the same attribute than the value of replace-all- attr-values-limit, then the modification will use ADD or DELETE operations rather than a REPLACE operation.

DS-38110

Updated the System Information monitor with an "isDocker" attribute to identify if the server is running in a Docker container.

DS-38118, DS-42495

Made several updates related to the server's handling of data written to standard output and standard error:

* The server can now be configured to rotate the logs/server.out file once it reaches a given size, and it will retain a configurable number of those log files. By default, the server will rotate the file once it reaches 100 megabytes and will keep up to ten files.

* To better facilitate capturing log data in containerized environments, the server now supports writing JSON-formatted access and error log messages to the JVM's original standard output and error streams (which will be separate from the server.out file when the server is started with the --nodetach argument).

* It is now possible to prevent the server from logging messages during startup in non-JSON format. It is also possible to prevent messages about administrative alerts from being written to standard error, or to write those messages in JSON format. These options are especially useful when using JSON-based logging to the console in no-detach mode, as they can help ensure that everything written to standard output and standard error will be formatted as JSON objects.

DS-38868

Updated setup to create a second encryption settings definition if data encryption is enabled. It will continue to create a definition for 128-bit AES encryption for use as the preferred definition to preserve backward compatibility with existing servers in the topology, but it will now also generate a definition for 256-bit AES encryption. The 256-bit AES definition may become the preferred definition in a future release, but you can use it now by first ensuring that any existing instances are updated to contain the new definition (with the "encryption-settings export" and "encryption-settings import" commands) and then making it the preferred definition (with "encryption-settings set-preferred") in all instances.

DS-39646

Updated the manage-topology add-server command to configure failover for Data Sync servers when they are added to a topology.

DS-39789

Updated the JVM memory usage monitor provider to fix an issue that could prevent the monitor from reporting the total amount of memory held by all memory consumers. Also, fixed an issue that could cause the memory-consumer attribute to use an incomplete message for consumers without a defined maximum size and added an additional memory-consumer-json attribute whose values are JSON objects with data that can be more easily extracted by automated processes.

DS-40650

Updated the collect-support-data tool to make it possible to specify how much data should be captured from the beginning and end of each log file to include in the support data archive. You can also specify the capture size when invoking the tool through an administrative task, recurring task, or extended operation.

DS-40828

Fixed an issue where some state associated with a JMX connection was not freed after the connection was closed. This led to a slow memory leak in servers that were monitored by an application that created a new JMX connection each polling interval.

DS-40967

Eliminated a misleading error message that could be logged at startup if the server was configured with one or more ACIs that only apply when using specific SASL mechanisms.

DS-41350

Fixed an issue where disabling certain backends (such as 'alarms') caused an internal monitor to log unnecessary error messages every few seconds, about not being able to gather data from that backend.

Note that deliberately disabling the 'alarms' backend is not recommended in normal operation, but may occur during backup/restore operations.

DS-41807

Fixed an issue where LDAP DELETE operations were sometimes not synchronized from an Oracle Unified Directory sync source, due to variations in the format of the 'targetUniqueID' attribute obtained from the Oracle changelog.

DS-41964

Fixed an issue with the manage-profile tool where files in a server profile's dsconfig/ directory without a ".dsconfig" extension could cause failures in manage-profile replace-profile when validating updated dsconfig files.

DS-41989

Fixed an issue that could result in duplicate column headers being produced by the Periodic Stats Logger, even when the header-prefix-per-column attribute was set to true.

DS-42004

Added a new configuration property, changes-queue-size, that controls the size of the changes queue on a Sync Pipe. This was previously only configurable via a system property, and it would change the queue size of all Sync Pipes. DataSync will continue to accept the system property, but it is encouraged to migrate the setting into the configuration.

DS-42045

Updated the Stats Collector Plugin with a new generate-collector-files configuration property. When using the plugin exclusively for providing metrics to one or more StatsD Monitoring Endpoints, set this property to false to prevent unnecessary I/O.

DS-42049

Fixed an issue with Sync server where retry ops could get the server stuck and unable to process changes at the source.

DS-42059, DS-42060

Updated setup to add options for improving communication security:

* Non-interactive setup now offers a --rejectInsecureRequests argument that will configure the server to reject any request received over a connection that is not encrypted with SSL or StartTLS.

* Non-interactive setup now offers a --rejectUnauthenticatedRequests argument that will configure the server to reject any request received over a connection that is not authenticated (or that is authenticated as the anonymous user).

* Interactive setup now allows you to configure the server with the LDAP connection handler disabled (which was already an option when using non-interactive setup), or enabled but only for communication encrypted with StartTLS.

The --rejectInsecureRequests and --rejectUnauthenticatedRequests arguments can also be used with manage-profile by including them in the setup-arguments.txt file of the server profile.

DS-42061

Updated the interactive command-line tool framework to prefer establishing secure LDAP connections over insecure connections. Previously, when prompting for the information needed to establish a connection, the default option was to create an unencrypted LDAP connection. Now, tools will default to creating an SSL-encrypted connection if the server supports it, or to creating a StartTLS-encrypted connection if that is available but SSL is not. Tools will also default to using streamlined settings when establishing secure connections. Previously, they would always prompt about how to determine whether the server's certificate chain should be trusted. When using the streamlined settings, the tools will only prompt about certificates that cannot automatically be considered trusted using information in the JVM's default trust store, the server's default trust store (config/truststore), or the server's topology registry.

DS-42062

Updated the root password policy so that LDAP bind responses for root users and topology administrators will be delayed by one second after five consecutive failed authentication attempts.

DS-42063

Updated the "delay bind response" failure lockout action to provide an option to delay the response to bind requests initiated by non-LDAP clients (for example, when using HTTP basic authentication). This option is disabled by default because delaying the bind response for non-LDAP clients may require temporarily blocking the thread used to process the request, which could increase the risk of a denial-of-service attack. To help mitigate this risk, if you enable delayed bind responses for non-LDAP clients, we recommend that you also increase the number of request handler threads for all enabled HTTP connection handlers.

DS-42115

Updated the server's command-line tool framework to make it easier and more convenient to communicate with the server over a secure connection when no trust-related arguments are provided. Most non-interactive tools will now check the server's default trust store, the topology registry, and the JVM's default trust store to see if the presented certificate chain can be automatically trusted without the need to prompt the user. If the presented chain cannot be automatically trusted, the user may be interactively prompted to determine whether it should be trusted.

DS-42199

Optimized some searches commonly used by the status tool. This should improve the performance of the tool in more complex or large-scale environments.

DS-42265

Upgrade to jetty 9.4

DS-42276

Fixed an issue where using the encryption-settings tool to import definitions with the set-preferred flag could result in none of the imported definitions being set as the preferred definition.

DS-42279

Updated the server to require a minimum key size of 2048 bits when negotiating a TLS cipher suite that uses ephemeral Diffie-Hellman key exchange.

DS-42298

Replaced the ldifsearch, ldifmodify, and ldif-diff command-line tools with more full-featured and robust implementations.

DS-42331

Replaced the ldapcompare tool with a new version that offers more functionality, including support for multiple compare assertions, following referrals, additional controls, and multiple output formats (including tab-delimited text, CSV, and JSON).

DS-42347

Updated the server to use /dev/urandom (on non-Windows systems where that path exists and is readable) instead of /dev/random as the primary source for secure random data. Attempts to read from /dev/random can block if the underlying system does not have sufficient entropy, which can have a severe adverse effect on performance. Reads from /dev/urandom will not block, and the data that it provides is no less secure than data from /dev/random in any way that matters for the server.

DS-42349, DS-43209, DS-43210, DS-43323, DS-43324

Added support for JSON-formatted audit loggers, which complement the existing file-based LDIF-formatted error logger. The JSON-formatted audit log messages provide a record of changes to data in the server and can be written to files on the local filesystem, written to the JVM's standard output or standard error stream, or sent to a syslog server.

Added support for JSON-formatted HTTP operation loggers, which complement the existing file-based loggers using the W3C common log format and a proprietary space-delimited text format. The JSON-formatted HTTP operation log messages provide a record of interaction with HTTP clients and can be written to files on the local filesystem, written to the JVM's standard output or standard error stream, or sent to a syslog server.

Fixed an issue that caused JSON-formatted loggers to use a timestamp format that was not strictly compliant with the ISO 8601 format described in RFC 3339. Timestamps incorrectly omitted the colon between the hour and minute components of the time zone offset.

DS-42504

Updated manage-profile replace-profile to set encryption settings definitions defined in the newer server profile as preferred in the encryption settings db.

DS-42547

Fixed an issue where manage-profile generate-profile would print "null" as the generated profile directory when writing to an existing directory.

DS-42609

Fixed an issue in which the Directory REST API could fail to decode certain credentials when using basic authentication.

DS-42632

Added support for creating or importing a key pair configuration object using an elliptic curve (EC) key algorithm. You can use this to designate the encryption key pair for a JWT access token validator that handles EC-encrypted access tokens.

DS-42634

The JWT Access Token Validator can now validate JWT access tokens signed using the elliptic curve digital signature algorithms ES256, ES384, and ES512.

DS-42635

The JWT Access Token Validator can now validate JWT access tokens encrypted using elliptic curve cryptographic algorithms. The following key encryption algorithms are now supported in addition to RSA-OAEP: ECDH-ES, ECDH-ES+A128KW, ECDH-ES+A192KW, and ECDH-ES+A256KW.

To support best practices for JWT security, you must now also configure the JWT Access Token Validator with explicit allow lists for key encryption and content encryption algorithms. For backward compatibility, the key encryption allow list defaults to RSA-OAEP, while the content encryption allow list defaults to A128CBC-HS256, A192CBC-HS384, and A256CBC-HS512. We recommend setting both allow lists to the strict minimum set of algorithms needed by the Access Token Validator.

DS-42651

Updated the manage-profile replace-profile subcommand to better support updating the server's keystore and truststore files. When using the --generateSelfSignedCertificate argument in a server profile's setup-arguments.txt file, the server will maintain the original keystore and truststore files during replace-profile. Otherwise, replace-profile will use the keystore and truststore specified in the profile's setup-arguments.txt file.

DS-42665

Fixed an issue that caused the resync tool to not take the attribute-comparison-method of the sync class into account. This caused resync to ignore when byte-for-byte comparison was configured.

DS-42667

Updated the server to set a unique cluster name when started for the first time.

DS-42669, DS-42748

Updated the online dsconfig step of the manage-profile replace-profile subcommand to support getting LDAP connection arguments from a tools.properties file on the server being updated.

Fixed an issue where boolean LDAP connection arguments like --useSSL and --trustAll would cause manage-profile replace-profile to fail when applying dsconfig online.

DS-42673

Updated the manage-profile setup subcommand to fail if the start-server command has a non-zero exit code.

DS-42681, DS-42684

Performance statistics generated by the Sideband API can now be published by the Periodic Stats Logger. To enable this, use the "included-http-servlet-stat" property of the Periodic Stats Logger.

DS-42687

Upgrade to Jetty 9.4.30

DS-42740

Fixed an issue where the dsconfig list subcommand would not display requested properties.

DS-42749

To support best practices for JWT security, you must now configure the JWT Access Token Validator with an explicit list of the JWT signing algorithms that it accepts. For backward compatibility, this list defaults to the RSA signing algorithms RS256, RS384, and RS512, but we recommend setting this list to the strict minimum set of signing algorithms needed by the Access Token Validator.

DS-42751

Added new override-status-code and additional-response-contents attributes to the Availability State HTTP Servlet Extension. These new attributes can be used to customize the response code and JSON response body of the servlet.

DS-42850

Fixed a typo in the password-expiring template that caused "password_expiration_time_of_day" to be printed instead of the password expiration time.

DS-42861

Updated the manage-profile tool logs to include the duration of each step the tool takes. The new --verbose argument can also be used to display timing information in the tool's console output.

DS-42872

Added a JSON-formatted stats logger to the server's default configuration. The stats logger is disabled by default.

DS-42886

Updated non-interactive setup (including manage-profile setup) to allow the password for the initial root user to be provided in pre-encoded form using the PBKDF2, SSHA256, SSHA384, or SSHA512 password storage scheme. This eliminates the need to have access to the clear-text password when setting up the server.

DS-42926

Fixed an issue where Ping Directory products configured to run as Microsoft Windows services were sometimes unable to automatically restart following an unplanned reboot, due to errors reading a corrupted server status file.

DS-42939

The Administrative Console configuration settings have been updated to account for the new SSO functionality.

DS-42952

For Windows only, there can be a hang on start when global configuration property startup-error-logger-output-location is set to values that contain standard-error. For Windows only, standard-error values are silently mapped to equivalent standard-output values.

DS-42963

Updated the manage-profile generate-profile subcommand to ignore files larger than 100 megabytes when generating a server profile. Fixed an issue where many large files in the server root could cause the tool to run out of memory.

DS-43027

Added a new --adminPasswordFile argument to the manage-topology add-server command, to allow specifying the administrator password with a file rather than with the command line.

DS-43073, DS-43198

Added support for ID Token Validators, which validate the integrity and content of ID tokens issued by OpenID Connect providers. Use these validators with the OAuth Bearer SASL Mechanism Handler to enable single sign-on (SSO) for the Administrative Console using an OpenID Connect provider such as PingOne. Currently, only PingOne is supported for SSO.

DS-43074

Added three built-in identity mappers that you can use to look up administrative accounts stored in the server configuration: Root DN Users, Topology Admin Users, and All Admin Users.

DS-43288

Updated setup and the replace-certificate tool to improve the way we generate self-signed certificates and certificate signing requests to make them more palatable to clients.

To reduce the frequency with which administrators had to replace self-signed certificates, we previously used a very long lifetime for self-signed certificates generated by setup or the replace-certificate tool. However, some clients (especially web browsers and other HTTP clients) have started more strenuously objecting to certificates to long lifetimes, so we now generate self-signed certificates with a one-year validity period. The inter-server certificate (which is used internally within the server and does not get exposed to normal clients) is still created with a twenty-year lifetime.

Also, the replace-certificate tool's interactive mode has been updated to improve the process that it uses to obtain information to include in the subject DN and subject alternative name extension for self-signed certificates and certificate signing requests. The following changes have been made in accordance with CA/Browser Forum guidelines:

* When selecting the subject DN for the certificate, we listed a number of common attributes that may be used, including CN, OU, O, L, ST, and C. We previously indicated that CN attribute was recommended. We now also indicate that the O and C attributes are recommended as well.

* When obtaining the list of DNS names to include in the subject alternative name extension, we previously suggested all names that we could find associated with interfaces on the local system. In many cases, we now omit non-qualified names and names that are associated with loopback interfaces. We will also warn about any attempts to add unqualified or invalid names to the list.

* When obtaining the list of IP addresses to include in the subject alternative name extension, we previously suggested all addresses associated with all network interfaces on the system. We no longer suggest any IP addresses associated with loopback interfaces, and we no longer suggest any IP addresses associated in IANA-reserved ranges (for example, addresses reserved for private-use networks). The tool will now warn about attempts to add these addresses for inclusion in the subject alternative name extension.

DS-43305

Increased the maximum number of RDN components that a DN may have from 50 to 100.

DS-43376

Updated log publisher logic to reduce the amount of CPU that the server consumes when it is idle.

DS-43480

Updated the system information monitor provider to restrict the set of environment variables that may be included. Previously, the monitor entry included information about all defined environment variables, as that information can be useful for diagnostic purposes. However, some deployments may include credentials, secret keys, or other sensitive information in environment variables, and that should not be exposed in the monitor. The server will now only include values from a predefined set of environment variables that are expected to be the most useful for troubleshooting problems, and that are not expected to contain sensitive information.

DS-43517

Updated the jose4j library used for JWT signing and encryption to version 0.7.2.

DS-43651

The Security Guide is now available online at pingidentity.com. The guide has been removed from the server packaging.