Important considerations for upgrading to this version of PingDirectoryProxy Server
To ensure correct search results with Delegated Admin, disable client caching by updating the Delegated Admin HTTP Servlet Extension to return response headers, and then stop and restart the server, as follows:
dsconfig set-http-servlet-extension-prop --extension-name "Delegated Admin" --set "response-header:Cache-Control: no-cache, no-store, must-revalidate" --set "response-header:Expires: 0" --set "response-header:Pragma: no-cache"
These are new features for this release of PingDirectoryProxy Server
New features for data encryption in transit and at rest: added support for TLS 1.3, ability to encrypt and automatically decrypt sensitive files such as tools.properties and keystore pin files using the server data encryption keys, and the ability to more easily and securely separate master keys from data encryption keys by protecting the server encryption settings database using either Amazon Key Management Service (AWS KMS) or HashiCorp Vault.
Added support for Amazon Corretto JDK 8, Windows Server 2019, Red Hat Enterprise Linux 7.6, CentOS 7.6, Amazon Linux 2, and Docker 18.09.0 on Ubuntu 18.04 LTS.
New load balancing algorithm for performance and improved reliability. By default for new configurations, the load balancing algorithm uses a failover strategy for writes and fewest operations strategy for reads. This means that normally all writes will go to the same server, which reduces replication conflicts or unique attribute conflicts, while all reads are distributed to the servers with lightest load, which distributes server utilization and improves performance. This combination strategy works in tandem with a change to PingDirectoryProxy to use assured replication by default for several types of write operations, which reduces the chances of data consistency issues when write operations are immediately followed by read operations for the same objects and attributes.
The following issues have been resolved with this release of PingDirectoryProxy Server:
The following enhancements were made to the topology manager to make it easier to diagnose the connection errors described in PDSTAGING-570:
- Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.
- Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period.
The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared as soon as there is connection symmetry.
The dsreplication tool has been fixed to work when the node being used to enable replication is currently out-of-sync with the topology master.
Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a key from the Amazon Key Management Service.
Added an HTTP servlet extension that can be used to retrieve the server's current availability state. It accepts any GET, POST, or HEAD request sent to a specified endpoint and returns a minimal response whose HTTP status code may be used to determine whether the server considers itself to be AVAILABLE, DEGRADED, or UNAVAILABLE. The status code for each of these states is configurable, and the response may optionally include a JSON object with an "availability-state" field with the name of the current state.
Two instances of this servlet extension are now available in the default configuration. A request sent to /available-state will return an HTTP status code of 200 (OK) if the server has a state of AVAILABLE, and 503 (Service Unavailable) if the server has a state of DEGRADED or UNAVAILABLE. A request sent to the /available-or-degraded-state will return an HTTP status code of 200 for a state of AVAILABLE or DEGRADED, and 503 for a state of UNAVAILABLE. The former may be useful for load balancers that you only want to have route requests to servers that are fully available. The latter may be useful for orchestration frameworks if you wish to destroy and replace any instance that is completely unavailable.
Updated the ldapdelete command-line tool to improve robustness and add features. Some of the new features include support for client-side subtree delete, deleting entries that match search filters, following referrals, writing failures to a rejects file, rate limiting, and support for a variety of additional controls.
Made visible the index-priming-idle-listener-timeout property of the Entry Balancing Request Processor configuration object. During global index priming, this property specifies the amount of time an extended operation response listener can be idle while in progress. For example, if the global index is priming and a backend server stops returning results but does not disconnect, this timeout can be used to force a retry of the operation.
HTTP Connection Handlers now accept client-provided correlation IDs by default. To adjust the set of HTTP request headers that may include a correlation ID value, change the HTTP Connection Handler's correlation-id-request-header property.
Make Fingerprint Certificate Mapper and Subject DN to User Attribute Certificate Mapper disabled by default on fresh installations. This will not affect upgrades from installations where these mappers are enabled.
Added support for insignificant configuration archive attributes.
The configuration archive is a collection of the configurations that have been used by the server at some time. It is updated whenever a change is made to data in the server configuration, and it is very useful for auditing and troubleshooting. However, because the entries that define root users and topology administrators reside in the configuration, changes to those entries will also cause a new addition to the configuration archive. This is true even for changes that affect metadata for those entries, like updates to the password policy state information for one of those users. For example, if last login time tracking is enabled for one of those users, especially with high-precision timestamps, a new configuration may be generated and added to the configuration archive every time that user authenticates to the server. While it is important for this information to be persisted, it is not as important for it to be part of the server's configuration history.
This update can help avoid the configuration archive from storing information about updates that only affect this kind of account metadata. If a configuration change only modifies an existing entry, and if the only changes to that entry affect insignificant configuration archive attributes, then that change may not be persisted in the server's configuration archive.
By default, the following attributes are now considered insignificant for the purpose of the configuration archive:
* ds-auth-delivered-otp * ds-auth-password-reset-token * ds-auth-single-use-token * ds-auth-totp-last-password-used * ds-last-access-time * ds-pwp-auth-failure * ds-pwp-last-login-ip-address * ds-pwp-last-login-time * ds-pwp-password-changed-by-required-time * ds-pwp-reset-time * ds-pwp-retired-password * ds-pwp-warned-time * modifiersName * modifyTimestamp * pwdAccountLockedTime * pwdChangedTime * pwdFailureTime * pwdGraceUseTime * pwdHistory * pwdReset
The Delegated Admin configuration has changed significantly. Delegated Admin Resource Types were removed and replaced by REST Resource Types. Delegated Administrators and Delegated Group Administrators were removed and replaced by Delegated Admin Rights and Delegated Admin Resource Rights. Previous configurations are converted to the new configuration definitions by the update tool when the server is updated.
Updated create-initial-proxy-config to change the load-balancing configuration that it generates.
Previously, the tool generated a fewest operations load-balancing configuration for all operations. While this may provide the best overall performance throughput, it increases the risk of replication conflicts, unique attribute conflicts, and application issues resulting from replication latency.
The tool now generates a criteria-based load-balancing configuration that uses a failover strategy for writes and fewest operations for reads. This ensures that under normal circumstances when all servers are available, all write requests will go to the same server, which eliminates the change for replication conflicts or unique attribute conflicts. When paired with a corresponding change in the Directory Server to use assured replication by default for all add, delete, and modify DN operations, as well as for all modify operations that involve passwords or certain password policy state attributes, the risk of read-after-write issues resulting from replication latency is also dramatically reduced, so it is therefore safe to continue using a fewest operations load-balancing algorithm for read operations.
Updated the server to support encrypting the contents of the PIN files needed to unlock certificate key and trust stores. If data encryption is enabled during setup, then the default PIN files will automatically be encrypted.
Also, updated the command-line tool framework so that the tools.properties file (which can provide default values for arguments not provided on the command line), and passphrase files (for example, used to hold the bind password) can be encrypted.
Updated the server to enable TLSv1.3 by default on JVMs that support it (Java 11 and higher).
Fixed an issue in the installer where the Administrative Console's trust store type would be incorrectly set if it differed from the key store type.
The response header used for correlation IDs may now be set at the HTTP Servlet Extension level using the correlation-id-response-header configuration property. If set, this property overrides the HTTP Connection Handler's correlation-id-response-header property.
Added the --skipHostnameCheck command line option to the setup script, which bypasses validation of the provided host name for the server.
Reduced the default max-connection-age value in LDAP External Server configuration objects to 10 minutes. This should avoid the problems created when firewalls between the proxy and the external server silently close connections that have been open for too long.
Fixed an issue that could prevent certain types of initialization failures from appearing in the server error log by default.
Updated the file retention recurring task to no longer log an informational message if there are no log matching files to delete.
Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a secret passphrase obtained from a HashiCorp Vault instance.
Fixed an issue in which backups of the encryption settings database could be encrypted with a key from the encryption settings database.
Fixed a bug where the startIndex value for SCIM requests would be incorrect if the used LDAPSearch element had more than one baseDN defined in the scim-resources XML file.
Fixed an issue where inter-server bind requests would fail if the cipher used reported a maximum unencrypted block size of 0.
Added an indent-ldap-filter tool that can make it easier to visualize the structure and components of a complex search filter.
Changed the default value of the HTTP Configuration property include-stack-traces-in-error-pages from 'true' to 'false'. Disabling this property prevents information about exceptions thrown by servlet or web application extensions from being revealed in HTTP error responses.
Internal connections created by HTTP requests are now associated with one of the configured client connection policies. A client connection policy may be selected using simple client connection criteria matching the client address, the user performing the request, and the protocol "HTTP/1.1". This change affects the following HTTP interfaces: SCIM, Directory REST API, Consent API and Delegated Admin API.
Fixed two issues in which the server could have exposed some clear-text passwords in files on the server file system.
* When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.
* When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.
In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.
We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords.
Added a set of message types to Trace Log Publishers that records events related to access token validation.
Fixed an issue that would throw an exception when trying to delete an entry containing uncached attributes if the LDAP changelog was enabled and using reversible form.
Removed the version information page from the docs/build-info.txt endpoint. This information is now available in build-info.txt, which is located in the root directory.