Upgrade Considerations
Important considerations for upgrading to this version of the PingDirectoryProxy Server
-
Note: The product names have been updated to reflect the UnboundID acquisition by Ping Identity. This is a naming and branding change only; the code base is the same as in prior releases and will continue to be maintained into the future.
-
If upgrading the server that was running an older version of the JDK, run the dsjavaproperties --initialize command after the software upgrade to compare the settings of the older JDK with requirements for the new server software. Apply any necessary changes to the upgraded server based on previous performance settings.
-
The 6.0 release makes these changes to supported platforms:
- CentOS 7.2 and RedHat 7.2 are now supported operating system versions.
- SUSE 11 SP4 is now supported; SUSE 11 SP2 support has been retired.
- Linux KVM and VMWare ESXi 6.x have been added as supported virtual machine environments.
- Deprecation of JDKs 7.x. Customers are strongly advised to use JDK 8 with this release. A later release of the platform will remove support for JDK versions 7.x. At that time customers will be required to upgrade to JDK 8.x when upgrading servers. This will apply to all of our JDK flavors (OpenJDK, OracleJDK, IBM JDK) on all platforms.
- WildFly 9.x (renamed from JBoss) is now a supported application server for all web applications, including the Administrative Console and sample applications. Support for JBoss 7.x has been retired. Tomcat support remains unchanged in this release.
-
PBKDF2 is now the default encoding for root passwords. This only affects new installations.
-
In addition to changing the default password storage scheme for root users to PBKDF2, the default password storage scheme for regular users has been changed to salted 256-bit SHA-2.
-
HTTPS defaults to ON: servers now default to use HTTPS for console and API connections including the SCIM API. This may affect automation scripts and development environments where HTTPS has not been in use before.
-
Generated user passwords, for example those created by the server during a password reset sequence, are now created as pass-phrases instead of random character strings. This makes them them easier to type and remember. This change will not affect upgrades.
-
The /config directory file permissions have been changed so that they are only accessible by the server user.
-
Customers who choose to use the optional encryption algorithms provided by the third-party BouncyCastle library are encouraged to upgrade to BouncyCastle 1.54.
What's New
These are new features for this release of the PingDirectoryProxy Server
-
Added a new control for very large result sets 'maximum-sort-size-limit-without-vlv-index,' which allows client applications to request that the server gracefully degrades to unsorted results in cases where sorting a very large result set would have caused a time-out.
-
Added LDAP support for applications that authenticate users with Yubikey one-time passwords. The extensions include the UNBOUNDID-YUBIKEY-OTP SASL handler configuration object, extended operations and command line tools for registering a user’s Yubikey device, deregistering, and supporting authentication using either the one-time password (OTP) only, or the OTP together with a static password. The server can be configured to use the public Yubico validation service, or a different validation service. The Yubikey FIDO U2F, OATH HOTP, and PGP modes are not supported.
-
Added new "generate TOTP shared secret" and "revoke TOTP shared secret" extended operations to make it easier for applications to enable TOTP authentication for users. While these operations are primarily intended to be invoked programmatically, a generate-totp-shared-secret tool can be used to invoke these operations from the command line.
-
A new transform-ldif tool is available to read an LDIF file and write an updated file with a number of changes applied. The transformations include:
- Scramble, replace, redact, or exclude a specified set of attributes.
- Replace values of a specified attribute with a generated value that includes a sequential counter.
- Exclude entries matching a provided set of criteria.
- Add a given set of attribute values to entries matching a provided set of criteria.
- Rename attributes.
- Replace the base DN for entries in a specified subtree.
-
A new load-ldap-schema-file tool is available for loading LDAP schemas while a server is active and on-line.
-
A new register-yubikey-otp-device tool is available for creating or changing associations between users and specific OTP devices.
-
The *rate performance testing tools now includes some additional sample rate pattern files: hockey stick, step-function, sine, triangle, sawtooth and square wave patterns.
-
The setup command now logs its input arguments, making it easier to confirm or duplicate a setup process. This changes the content of the log and may affect automated scripts that read these log files.
-
The config-diff tool, which makes it easy to compare and reconcile settings between server instances, now also supports the --pretty-print option which adds line breaks to the generated lists of dsconfig commands.
-
The manage-account tool has been enhanced significantly to make it easier to perform operations that affect large sets of user accounts including bulk lock-outs, parallel processing of updates, support for input filter criteria and DN lists. In particular, the manage-account tool now supports explicitly setting user accounts to the "locked-out" state. This is an improvement over earlier versions which required manipulation of operational attributes. See the command help for a complete list of the options and new sub-commands.
-
For easier consumption by third-party analysis tools, the Directory Servers and Directory Proxy Servers can now output JSON log formats. Similar support will be added to the Data Sync and Governance Brokers in a later release.
-
To help avoid issues when indexes near their index-entry-limit, the verify-indexes command now has the following two options:--listKeysNearestIndexEntryLimit, and --listKeysExceedingIndexEntryLimit. The Admin Guide includes a new section, "Monitoring Index Entry Limits", which explains how to set, track, and tune the server's Index Entry Limit values.
-
Monitor entries have been added for a number of related metrics, all of which can be set to trigger alarms:
- ds-index-unique-keys-near-entry-limit-accessed-by-search-since-db-open
- ds-index-unique-keys-exceeding-entry-limit-accessed-by-search-since-db-open
- ds-index-unique-keys-near-entry-limit-accessed-by-write-since-db-open
- ds-index-unique-keys-exceeding-entry-limit-accessed-by-write-since-db-open
-
The Pass-Through Authentication plugin has a new "allowLaxPassThroughAuthenticationPasswords" option that permits password changes that do not comply with the Directory Server's password policy. This facilitates integration in cases where the pass-through system has less-strict rules for new passwords.
-
For Java developers whose tools and workflows make use of Maven, the Server SDK jar has been deployed to Maven Central so that a developer can now add the Server SDK as a project dependency by adding a few lines to a project's pom.xml. Also, developers can now generate a Server SDK project that Maven-aware IDEs such as IntelliJ IDEA can package into an extension bundle with no special configuration needed. This benefit extends similarly to continuous integration systems such as Jenkins.
-
The dsconfig tool provides the ability to search for and quickly navigate to configuration objects and properties in which the name, synopsis, or description matches a provided pattern.
-
A new rotate-log tool and task have been added, which can be used to trigger rotation of one or more log files.
-
The Configuration API is now fully supported for all servers. In this release, the API was changed to match SCIM conventions for attribute naming, resource modeling, and the standard HTTP verbs. The UnboundID SCIM 2 SDK (available through GitHub) can now be used with the Configuration API.
-
All servers have an updated web Administrative Console, which includes:
- New layouts for operational statistics, processing time, queues, all monitors, and the list of installed extensions.
- Alert and alarm displays, summarizing the data in cn=alerts and cn=alarms and based on the configured gauges. Plus, filtering and searching for these.
- A new LDAP Schema Editor for importing schema files, validity checking, creation and editing of object classes and attribute types. The editor also supports viewing of the attribute syntaxes, inheritance, and indexes that exist for each attribute and the dependencies between object classes and attributes.
-
The new Administrative Console can also be deployed to independent application servers instead of being co-hosted by the servers. This simplifies deployment models and increases separation between data and application layers.
-
To assist with situations where a very large number of changes may cause disk, memory, and server start time to increase unexpectedly, alerting and gauge features have been added to the Recent Changes Database.
-
Servers can now trigger events whenever log file rotation occurs. This includes "copy on rotate" and "summarize on rotate" listeners, as well as Server SDK support for creating custom log file rotation listeners.
-
It is now possible to create, change, and remove root user accounts across the topology using the dsconfig tool and Administrative Console.
Known Issues/Workarounds
The following are known issues in the current version of the PingDirectoryProxy Server
-
When deploying the Administrative Console in Tomcat 8, and accessing the Administrative Console application using Tomcat's Web Application Manager, some browsers (including Safari and Firefox) will generate a path URL that encodes the dash in ubid-console. This results in a path such as http://localhost:8888/ubid%2Dconsole/, which causes session management errors. To workaround this issue, copy and paste the generated link into a browser, and replace the encoded dash with a dash (-) character.
-
The PermSize and MaxPermSize JVM properties are no longer supported in JDK 8, and will safely be ignored. These properties can be removed by modifying the config/java.properties file and running "bin/dsjavaproperties" while the server is offline.
-
Security criteria for root passwords with the default configuration will be increased in a future release. This might affect automated installation scripts that currently use less secure passwords. This will not affect existing root accounts.
-
The dsconfig tool and the Administrative Console enables creating and managing new Root DN users in this release. However, there is a limitation with changing the password of the currently logged in administrator. The ldappasswordmodify command can be used to change the administrator's password by providing the current and new password.
Resolved Issues
The following issues have been resolved with this release of the PingDirectoryProxy Server:
Ticket ID | Description |
---|---|
DS-979 |
Added the ability to search for configuration objects and their properties by name with the dsconfig tool. |
DS-4235 |
Added support for log file rotation listeners, which allow for custom processing whenever a log file is rotated out of service so that the server will no longer write to it. A copy listener (which will copy the rotated log file to an alternate location, optionally compressing it in the process), and a summarize listener (which will invoke the summarize-access-log tool on the rotated log file) are included. The Server SDK also includes an API for creating custom log file rotation listeners. |
DS-7017 |
Added support for authenticating with one-time passwords generated by YubiKey devices. The server may be configured to require static passwords in conjunction with YubiKey one-time passwords as a form of two-factor authentication, or it may be configured so that a one-time password alone is sufficient for authentication. |
DS-7505,DS-13571,DS-13860 |
Updated the default file permissions for new installations on UNIX-based systems. Files and directories included in the zip file will be only be accessible to their owner (the user that unzipped the file) by default. Newly-created files and directories will also be assigned permissions that allow them to be accessed only by the account used to run the server. Existing configuration options for setting file permissions (the log-file-permissions and db-directory-permissions properties) will continue to behave as before. The new config/server.umask file will control the default permissions for all other newly-created files and directories. |
DS-9407,DS-15183,DS-15220 |
Updated the initial server configuration to improve security and usability. These changes apply only to new installations and will not be applied when updating an existing installation. Changes include:
|
DS-10312 |
Added the server's process ID to the output of the status tool. |
DS-10464 |
Added a new rotate-log tool to request the rotation of one or more log files. |
DS-10466,DS-10765,DS-14479,DS-15318,DS-16154 |
Addressed a few issues in config-diff. In some situations, config-diff would not generate commands in an order that respected all dependencies. This has been fixed. Most expected warnings are now excluded by default but can be included in the output with the --includeAllWarnings option. The --sourceBindPasswordFile and --targetBindPassword are now applied in conjunction with the --targetConfigGroup and --sourceConfigGroup options. |
DS-10946 |
Added a --dry-run option to dsconfig, which can be used in batch mode to validate the configuration changes in a batch file without applying them. |
DS-11076 |
Updated the Directory Proxy Server to forward Password Policy State and Validate TOTP Password extended operations to the backend Directory Server for processing. Also, the Directory Proxy Server is now able to forward the following extended operations to a backend Directory Server in an entry-balancing configuration: Deliver Password Reset Token, Deliver Single Use Tokens and Get Supported OTP Delivery Mechanisms. |
DS-12191 |
Added support for setting the request header size in the Jetty http configuration server properties. |
DS-12555 |
Changed the Directory Proxy Server search behavior to fail when processing a critical RouteToBackendSetRequestControl with an unknown entry balancing request processor ID included. Non-critical controls will ignore unknown request processor IDs. |
DS-13401 |
Improved the collect-support-data tool to include information provided by systemd on platforms that support it. |
DS-13700 |
Updated command-line tools based on the LDAP SDK tool APIs to add the following features:
|
DS-13823 |
Collect-support-data tool now captures Kerberos config and log information. |
DS-14213 |
Added the ability to create local constants in LDIF template files using the new 'local' keyword. |
DS-14430 |
Updated the Apache commons collections library to address the security vulnerability described by CVE 2015-4852. |
DS-14548 |
Added a monitor entry for each Server SDK extension. |
DS-14694 |
Added a --prettyPrint option to the config-diff tool to make the output more human-readable. |
DS-14699 |
The server now enforces that attributes referenced in configuration properties are defined explicitly in the local schema. This includes cached attribute types as well, such as, the attribute types that a Proxy Server caches from backend Directory Server instances. |
DS-14704 |
Updated the restore command so that it can no longer be used to restore a backup of the config backend. The command now points the administrator for safer ways to revert configuration changes, including using config-diff. |
DS-14749 |
Updated the server's support for the Twilio Messaging Service so that it uses the newer "Messages" API when sending SMS messages instead of the older "SMS" API. The older API has been deprecated, and Twilio now imposes a 120-character limit for messages sent via that API. The messages API allows the server to take advantage of the full 160 characters per SMS message. |
DS-14805 |
Improved the admin alert health check system to handle situations where a massive number of admin alerts generated on an external server could result in the server being seen as unavailable. |
DS-14807 |
Tools used to prepare a server for access by another server, such as prepare-external-server, now validate base DN entries before any modifications are performed on the prepared server. |
DS-14857 |
Fixed an issue with the dsjavaproperties tool where java properties for PermSize and MaxPermSize could be added when using JDK 8, which no longer supports these options. |
DS-14866 |
Fixed a race condition in the attribute index's cleanup thread, which was causing the Proxy Server to hang during startup. |
DS-14878 |
Improved memory utilization when processing entries with very large attributes, to prevent possible data retention in memory. |
DS-14889 |
Fixed an issue where the Proxy Server forwarded an atomic multi-update extended operation twice to the same backend set in an entry-balanced configuration, when the operation included a RouteToBackendSet control with absolute routing. |
DS-14919 |
Added support for JSON-formatted access and error log messages. |
DS-14922 |
Improved the subordinate subtree view processing logic to only use the exclude branch request control for subordinate views that are within the scope of the target DN, for search requests processed by the parent subtree view. |
DS-14923 |
Updated the bcrypt, crypt, PBKDF2 and scrypt password storage schemes so they can be used to create new instances. |
DS-14979 |
Fixed a case where attribute syntax configuration changes would not apply to undefined attributes, which rely on default attribute types. |
DS-15015 |
Server SDK extensions are now built with a Java source version of 1.7 by default. |
DS-15059 |
Fixed an issue with simple paged results that require spanning multiple backend servers through an entry balanced request processor to fill a single page. The operation would return with a 'no such object' result code if one of the backend servers used did not have any matching results. |
DS-15088 |
The former suite of Administrative Console applications, each of which were tied to a particular product (for example the dsconsole.war for the Directory Server) are no longer available, and have been superceded by a new version of the Administrative Console capable of managing any server product. You can choose to access the Administrative Console by hosting it within a server, or by deploying it in an external servlet container. For the former, enable an HTTP Connection Handler and add the Administrative Console Web Application Extension to the handler. For the latter, download and unzip the management-console-[version].zip file, and install the ubid-console.war file according to your container's instructions. |
DS-15108 |
Replaced the scramble-ldif tool with a more powerful transform-ldif tool with support for a number of additional transformation types. The new transform-ldif tool is backward compatible with the former scramble-ldif tool, and the scramble-ldif shell script and batch file are still included with the server to ensure compatibility with scripts that depend on that tool. |
DS-15132 |
Improved the locking strategy for multi-update requests to better accommodate delete and add requests for the same entry. This also enables graceful failures for bad requests, instead of lock timeouts. |
DS-15168 |
Increased the maximum size of the thread pool that is used to process entry balancing broadcast operations. The maximum is now 64 times the number of worker threads with an upper limit of 2048. |
DS-15175 |
The Configuration API now returns unquoted, native Javascript values for integer, real number, and boolean properties. Duration and size property values, for example '1 w' or '100 G', continue to be represented as Javascript string types. |
DS-15178 |
Improved the error messages and examples for create-rc-script and create-systemd-script by explicitly suggesting the use of sudo so that the scripts can modify protected files. |
DS-15186 |
Fixed a rare condition where priming the Directory Proxy Server would not complete if a backend Directory Server becomes unavailable. |
DS-15195 |
Added support for server affinity using extended operations. |
DS-15221 |
Changed interactive setup default value for HTTPS enablement. |
DS-15337 |
Improved the error messages for create-rc-script and create-systemd-script when the directory in which the script will be created does not exist. |
DS-15349 |
Added support for a "generate TOTP shared secret" extended operation that allows a client to request that the server generate a shared secret for a specified user that will be stored in the user's entry and returned to the client. That shared secret can be used to generate time-based one-time passwords for use in the course of authenticating to the server through the UNBOUNDID-TOTP SASL mechanism. A "revoke TOTP shared secret" extended operation was also added to allow a shared secret to be eliminated if it is no longer needed or may have been compromised. The password policy state extended operation and the manage-account command-line tool have also been updated to provide support for manipulating the set of TOTP shared secrets for a user. |
DS-15361,DS-15363,DS-15434 |
Updated interactive setup to display default values, and improved the overall layout and appearance. |
DS-15400 |
Addressed an issue where dsconfig incorrectly allowed certain configuration objects to be deleted. |
DS-15412 |
Improved the error messages produced by the manage-extensions tool when attempting to install invalid extensions. |
DS-15417 |
Updated the global ACIs that ship with the server to use a separate ACI for each control or extended request to allow by default, rather than grouping all desired controls together in one ACI and all desired extended requests together in a second ACI. This change will only be reflected in new installations, and not when updating an existing deployment. |
DS-15422 |
Root DN User configuration entries can now be fully managed through the configuration management interfaces such as dsconfig and the Administrative Console. |
DS-15466 |
Added more logging information when initializing web application and servlet extensions in case an extension causes conflicts or delays. |
DS-15521 |
Updated setup to encode the root password with the PBKDF2 password storage scheme instead of SSHA512. |
DS-15522 |
Updater tool will increase PermSize and MaxPermSize parameters to recommended value to prevent Java JVM pauses. |
DS-15559 |
Added support for an UNBOUNDID-EXTERNALLY-PROCESSED-AUTHENTICATION SASL mechanism that indicates that an application attempted to verify the identity of a user whose account is stored in the server but that used a form of authentication that is external to the server (for example, via social login). The server will not alter the authentication state of the underlying connection, but may veto a successful external authentication if the user's account is not in a usable state (for example, the account is locked or disabled, or the password is expired), or it may update password policy state for the user to reflect the authentication attempt (for example, updating the last login time and IP address for a successful authentication, or recording the failed attempt and potentially locking the account for an unsuccessful authentication). |
DS-15571 |
Increase the minimum memory requirements for the server process from 256MB to 384MB to accommodate the Administrative Console. |
DS-15576 |
Added a load-ldap-schema-file tool that will allow the server to recognize a new schema file, or an updated version of an existing schema file, and make the definitions immediately available without needing to restart the server. |
DS-15592 |
Fixed an error that could occur during upgrade when the configuration can not be loaded due to missing custom schema. |
DS-15621 |
Updated the Groovy Scripting Language version to 2.4.6. |
DS-15622 |
Fixed an issue that prevented the deletion of disabled debug loggers. |
DS-15789 |
Updated the server to allow users with expired passwords to authenticate with SASL mechanisms that do not involve passwords. |
DS-15827 |
Updated the globally-unique attribute plugin so that the filter property applies to conflict searches, and matches entries being added or modified. |
DS-15849,DS-15850,DS-15851,DS-15852,DS-15853 |
Rewrote the manage-account tool to provide many new features:
|
DS-15920 |
Improved the warnings given when the maximum memory that all server components can consume is greater than the available memory in the JVM. |
DS-15943 |
Addressed an issue where jsonObjectFilterExtensibleMatch queries in the proxy would fail if any DN Mappers were configured. |
DS-16053 |
Improved the way we handle unexpected errors and invalid DNs during proxy transformations. |
DS-16104 |
Updated the password policy state extended operation and the manage-account tool to provide a way to obtain a list of the SASL mechanisms and OTP delivery mechanisms that are available to a user, to determine whether a user has a TOTP shared secret, and to retrieve and manipulate the set of public IDs for the YubiKey OTP devices registered for a user. |
DS-16224 |
Updated the sanitize-log tool to add support for JSON-formatted access and error log files. |