Upgrade Considerations

Important considerations for upgrading to this version of PingDataGovernance Server:

  • WARNING: OAuth scope configurations for resource access control, including fine-grained access control, and JEXL-based policies are no longer supported. Manual steps are necessary to migrate configuration and policies in order to restore the functionality of SCIM APIs. Please contact your account executive to schedule time for migration assistance.

  • If you are updating a multi-server topology from PingDataGovernance 7.0.x to 7.3.0.0, you must use the --skipMirroredSubtreeUpdateTask option for the updater or the update fails. Alternatively, you can uninstall all but one of the servers to retain the base configuration, update the standalone server, install fresh servers on the new version, and add them back to the topology with the peer options. However, using the --skipMirroredSubtreeUpdateTask option is the recommended path.

What's New

These are new features for this release of PingDataGovernance Server:

  • New features for data encryption in transit and at rest: added support for TLS 1.3, ability to encrypt and automatically decrypt sensitive files such as tools.properties and keystore pin files using the server data encryption keys, and the ability to more easily and securely separate master keys from data encryption keys by protecting the server encryption settings database using either Amazon Key Management Service (AWS KMS) or HashiCorp Vault.

  • Added support for Amazon Corretto JDK 8, Windows Server 2019, Red Hat Enterprise Linux 7.6, CentOS 7.6, Amazon Linux 2, and Docker 18.09.0 on Ubuntu 18.04 LTS.

  • Fine-grained data access control for JSON-based APIs. Configured as a reverse proxy to existing customer API endpoints, PingDataGovernance enforces dynamic authorization policies to inbound API calls or outbound API responses. For inbound calls, policies can inspect request attributes and request bodies to allow or deny the HTTP call. For outbound responses, policies can whitelist or blacklist JSON objects and specific attributes, thus sanitizing the HTTP response data per use case.

  • New Policy Administration GUI. Data owners and other stakeholders can now collaborate with IT and developers to build and test data access control policies. IT and developers configure services and attributes that gather, extract, and transform data dynamically from REST APIs, RBDMS, LDAP, and more. Data owners and other stakeholders build expressions to check and compare these attributes as part of a hierarchy of policies and rules. The Policy Administration GUI supports testing with mock input data, and it displays test results in a graphical tree to help policy writers understand and troubleshoot policy logic.

Resolved Issues

The following issues have been resolved with this release of PingDataGovernance Server:

Ticket ID Description
PDSTAGING-570,DS-38334

The following enhancements were made to the topology manager to make it easier to diagnose the connection errors described in PDSTAGING-570:

- Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.

- Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period.

PDSTAGING-570,DS-38344

The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared as soon as there is connection symmetry.

DS-15734

Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a key from the Amazon Key Management Service.

DS-18060

Added an HTTP servlet extension that can be used to retrieve the server's current availability state. It accepts any GET, POST, or HEAD request sent to a specified endpoint and returns a minimal response whose HTTP status code may be used to determine whether the server considers itself to be AVAILABLE, DEGRADED, or UNAVAILABLE. The status code for each of these states is configurable, and the response may optionally include a JSON object with an "availability-state" field with the name of the current state.

Two instances of this servlet extension are now available in the default configuration. A request sent to /available-state will return an HTTP status code of 200 (OK) if the server has a state of AVAILABLE, and 503 (Service Unavailable) if the server has a state of DEGRADED or UNAVAILABLE. A request sent to the /available-or-degraded-state will return an HTTP status code of 200 for a state of AVAILABLE or DEGRADED, and 503 for a state of UNAVAILABLE. The former may be useful for load balancers that you only want to have route requests to servers that are fully available. The latter might be useful for orchestration frameworks if you want to destroy and replace any instance that is completely unavailable.

DS-37617

HTTP Connection Handlers now accept client-provided correlation IDs by default. To adjust the set of HTTP request headers that may include a correlation ID value, change the HTTP Connection Handler's correlation-id-request-header property.

DS-37753

PingDataGovernance now contains Server SDK support for Advices.

DS-37839

Make Fingerprint Certificate Mapper and Subject DN to User Attribute Certificate Mapper disabled by default on fresh installations. This will not affect upgrades from installations where these mappers are enabled.

DS-37959

Added support for insignificant configuration archive attributes.

The configuration archive is a collection of the configurations that have been used by the server at some time. It is updated whenever a change is made to data in the server configuration, and it is very useful for auditing and troubleshooting. However, because the entries that define root users and topology administrators reside in the configuration, changes to those entries will also cause a new addition to the configuration archive. This is true even for changes that affect metadata for those entries, like updates to the password policy state information for one of those users. For example, if last login time tracking is enabled for one of those users, especially with high-precision timestamps, a new configuration may be generated and added to the configuration archive every time that user authenticates to the server. While it is important for this information to be persisted, it is not as important for it to be part of the server's configuration history.

This update can help avoid the configuration archive from storing information about updates that only affect this kind of account metadata. If a configuration change only modifies an existing entry, and if the only changes to that entry affect insignificant configuration archive attributes, then that change may not be persisted in the server's configuration archive.

By default, the following attributes are now considered insignificant for the purpose of the configuration archive:

* ds-auth-delivered-otp * ds-auth-password-reset-token * ds-auth-single-use-token * ds-auth-totp-last-password-used * ds-last-access-time * ds-pwp-auth-failure * ds-pwp-last-login-ip-address * ds-pwp-last-login-time * ds-pwp-password-changed-by-required-time * ds-pwp-reset-time * ds-pwp-retired-password * ds-pwp-warned-time * modifiersName * modifyTimestamp * pwdAccountLockedTime * pwdChangedTime * pwdFailureTime * pwdGraceUseTime * pwdHistory * pwdReset

DS-38050

Updated the server to support encrypting the contents of the PIN files needed to unlock certificate key and trust stores. If data encryption is enabled during setup, then the default PIN files will automatically be encrypted.

Also, updated the command-line tool framework so that the tools.properties file (which can provide default values for arguments not provided on the command line), and passphrase files (for example, used to hold the bind password) can be encrypted.

DS-38072

Updated the server to enable TLSv1.3 by default on JVMs that support it (Java 11 and higher).

DS-38085

Fixed an issue in the installer where the Administrative Console's trust store type would be incorrectly set if it differed from the key store type.

DS-38089,DS-38705

The Open Banking Account Request servlet now supports versions 1.1, 2.0, and 3.0 of the Open Banking Read/Write Data API.

Error responses returned by the Account Request servlet are now formatted as described in the Open Banking Read/Write Data API specification, v3.0.

DS-38090,DS-38564,DS-38567

The response header used for correlation IDs may now be set at the HTTP Servlet Extension level using the correlation-id-response-header configuration property. If set, this property overrides the HTTP Connection Handler's correlation-id-response-header property.

DS-38109

Added the --skipHostnameCheck command-line option to the setup script, which bypasses validation of the provided host name for the server.

DS-38403

Fixed an issue that could prevent certain types of initialization failures from appearing in the server error log by default.

DS-38512

Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a secret passphrase obtained from a HashiCorp Vault instance.

DS-38550

Fixed an issue in which backups of the encryption settings database could be encrypted with a key from the encryption settings database.

DS-38670

Fixed a bug where the startIndex value for SCIM requests would be incorrect if the used LDAPSearch element had more than one baseDN defined in the scim-resources XML file.

DS-38737

Fixed an issue where inter-server bind requests would fail if the cipher used reported a maximum unencrypted block size of 0.

DS-38864

Changed the default value of the HTTP Configuration property include-stack-traces-in-error-pages from 'true' to 'false'. Disabling this property prevents information about exceptions thrown by servlet or web application extensions from being revealed in HTTP error responses.

DS-38897,DS-38908

Fixed two issues in which the server could have exposed some clear-text passwords in files on the server file system.

* When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor.

* When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, ldappasswordmodify, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.

In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.

We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations. You also might want to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you might want to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords.

DS-38913

Added a set of message types to Trace Log Publishers that records events related to access token validation.

DS-39086

Removed the version information page from the docs/build-info.txt endpoint. This information is now available in build-info.txt, which is located in the root directory.

DS-39102

Updated the server SDK class AccessTokenValidator's method initializeTokenValidator's parameters. The method's first parameter is now of type ServerContext instead of BrokerContext. This change is incompatible with earlier versions of the server SDK.