PingDataGovernance 8.0.0.0 Release Notes

Upgrade Considerations

Important considerations for upgrading to this version of the PingDataGovernance Server:
  • Changes have been made to the Trust Framework in the default policies shipped with PingDataGovernance Server. See the PingDataGovernance Server Administration Guide for instructions on updating existing policy deployments.
  • Token Resource Lookup Methods, which are invoked after access token validation to obtain an access token owner's attributes from an external identity store, have been updated so that they do not strictly require SCIM. In this release, the existing SCIM-based method is provided, in addition to a new ability to create custom Token Resource Lookup Methods using the Server SDK.
  • Token Resource Lookup Methods which were configured in existing deployments will be automatically migrated as SCIM Token Resource Lookup Methods during upgrade. Any existing dsconfig scripts that create Token Resource Lookup Methods should be updated to specify the --type parameter with the value "scim" before using these scripts with an upgraded server.
  • An issue has also been fixed in which Token Resource Lookup Methods were not invoked after validating an access token with an Access Token Validator which was created using the Server SDK. The TokenValidationResult object returned by third-party Access Token Validators no longer includes the tokenOwner field, and extensions that set this field must be updated.

  • API Endpoints, which were introduced in 7.3.0.0, have been renamed to Gateway API endpoints as of version 7.3.0.2.

    WARNING: When performing an update, existing API Endpoint configuration objects are migrated automatically. To reflect this change, manually update your dsconfig scripts and other automated deployments or configurations.

  • The Allow Attributes and Prohibit Attributes advices have been deprecated. If a deployment requires the behavior that these advices provided, use the Server SDK to implement the appropriate behavior.
  • Changes to the server configuration in this release of PingDataGovernance are incompatible with previous releases. This entails special consideration when upgrading a topology of servers that were set up using the setup tool's peer setup option. After a server has been upgraded to the new version, an admin must manually apply configuration changes that could not be automatically applied by the update tool. The update tool will print out the instructions on how to do this.
  • Changes to the server configuration in this release of PingDataGovernance are incompatible with previous releases. This entails special consideration when reverting a topology of servers to their previous versions. All servers must be put into their own cluster before running revert-update using a dsconfig command like the following: 
    dsconfig set-server-instance-prop --instance-name <server-instance-name> \ 
--set cluster-name:<unique-cluster-name>

    In the previous command, it is recommended that the cluster name be set to the server instance name, which is guaranteed to be unique.

  • If you are upgrading from PingDataGovernance 7.3.0.x to 8.0.0.0, an updated version of the Policy Administration GUI is required

  • If you are updating a multi-server topology from PingDataGovernance 7.0.x to 8.0.0.0, you must use the --skipMirroredSubtreeUpdateTask option for the updater or the update fails. Alternatively, you can uninstall all but one of the servers to retain the base configuration, update the standalone server, install fresh servers on the new version, and add them back to the topology with the peer options. However, using the --skipMirroredSubtreeUpdateTask option is the recommended path.

What's New

These are new features for this release of the PingDataGovernance Server:

  • Use Server Profiles to reduce risk and improve consistency following the DevOps principle of infrastructure-as-code. Administrators can export the configuration of a PingDataGovernance instance to a directory of text files called a Server Profile, track changes to these files in version control like Git, and install new instances of PingDataGovernance or update existing instances of PingDataGovernance from a Server Profile. Server Profiles support variable substitution in order to remove the settings unique to each pre-production or production environment from the Server Profile that is stored in version control.
  • Use PingDataGovernance with existing API Lifecycle Gateways. Previously, the PingDataGovernance Server functioned only as a reverse proxy. A new Sideband API introduces an alternate deployment mode in which PingDataGovernance Server uses a plugin to connect to an existing API Lifecycle Gateway. In sideband deployment, the API Lifecycle Gateway handles requests between API clients and backend API services. The integration plugin intercepts all data and passes it through the PingDataGovernance Sideband API. PingDataGovernance continues to enforce policy, authorizing requests and responses, and filtering or modifying request and response data.
  • Improved handling of sensitive data through API Lifecycle Gateways. The Sideband API, which is the integration method between API Lifecycle Gateways and PingDataGovernance (introduced in 7.3.0.2), now supports filtering and modifying of API responses, in addition to authorizing requests. In this configuration, the integration plugin intercepts all response data, and passes it through the PingDataGovernance server, which filters and modifies the response data based on policies.
  • Use external authorization in non-API use cases. Organizations can now externalize the authorization logic from other enforcement points, like legacy web applications, and manage these authorization policies centrally in PingDataGovernance. With this licensing option, other authorization enforcement points can call into the core policy engine of PingDataGovernance via a Policy Decision Point API (PDP API) that complies with the XACML JSON Profile Request API.
  • More API request and response modification capabilities. Policy administrators can take advantage of new advice to replace JSON data values, even attributes that are deeply nested within API requests or responses. Also, administrators can define policy to manipulate the query string of API requests, useful for limiting upstream API calls based on the attributes of the caller or context.
  • Additional HTTP request/response data is now provided to policies when an outbound response is processed by either the API Security Gateway or the Sideband API. A policy request for an outbound response may now include the following attributes, in addition to those already supported:
    • HttpRequest.Response
    • HttpRequest.RequestHeaders
    • HttpRequest.RequestBody
    • HttpRequest.ResponseHeaders
    • HttpRequest.AccessToken
    • TokenOwner

    The existing HttpRequest.Headers policy request attribute is deprecated and will be removed in a future release of PingDataGovernance.

Known Issues/Workarounds

The following are known issues in the current version of the PingDataGovernance Server:
  • When the PDP API receives a valid request, it first authorizes the client request itself before sending the client's policy request to the policy engine. As currently implemented, the PDP API ignores any advices in the decision, so the policy writer has no control over either the HTTP status code or the error message.
  • The following are suggested solutions for problems with slow DNS:
    • Maintain a connection pool in the client app rather than opening new connections for each bind.
    • Add appropriate records, including PTR records, to DNS.
    • Add options timeout:1 or options single-request in the /etc/resolv.conf file.
    • If IPv6 requests specifically are causing issues, add -Djava.net.preferIPv4Stack=true to the start-server.java-args line in the PingDirectory config/java.properties file, run bin/dsjavaproperties, and restart the server to stop the issuance of IPv6 PTR requests.

    • Some server tools, such as collect-support-data and rebuild-index, will fail with errors if they are run with an encrypted tools.properties file.

      Workaround: Add the --noPropertiesFile argument to the server tools to prevent them from pulling information from the encrypted file.

  • The working directory value used by exec tasks is not implemented for recurring exec tasks.
  • Deploying the Admin Console to an external container using JDK 11 requires downloading the following dependencies and making them available at runtime (for example, by copying them to the WEB-INF/lib directory of the exploded WAR file).
    • groupId:jakarta.xml.bind, artifactId:jakarta.xml.bind-api, version:2.3.2
    • groupId:org.glassfish.jaxb, artifactId:jaxb-runtime, version:2.3.2
    Workaround: Deploy the Console in an external container using JDK 8.

Resolved Issues

The following issues have been resolved with this release of the PingDataGovernance Server:

Ticket ID Description
DS-17278 Added a cn=Server Status Timeline,cn=monitor monitor entry to track a history of the local server's last 100 status changes and their timestamps. Updated the LDAP external server monitor to include attributes tracking health check state changes for external servers. The new attributes include the number of times a health check transition has occurred, timestamps of the most recent transitions, and messages associated with the most recent transitions.
DS-37504, DS-38765, DS-39011 Fixed an issue in the Passthrough SCIM resource type that could cause an access token validator's token subject lookup to fail if the user store was unavailable when PingDataGovernance Server was started. This issue would typically manifest as a SCIM schema error in the debug trace log, such as "Attribute uid in path uid is undefined."
DS-37565 A new advice type has been added, modify-attributes, which can modify the values of attributes.
DS-37720

Added Token Resource Lookup Methods as a new type of Server SDK extension. A Token Resource Lookup Method can be used to customize the way that PingDataGovernance looks up access token owners to populate the TokenOwner attribute used to make policy decisions.

For example, a developer could build a Token Resource Lookup Method that maps an access token subject to an identity stored in an RDBMS or an arbitrary REST API.

DS-37881 The PingFederate Access Token Validator will now refresh its cached value of the PingFederate server's token introspection endpoint. A new attribute, endpoint-cache-refresh, has been added to the PingFederate Access Token Validator, which will determine how often this refresh occurs.
DS-37955 To support multiple trace loggers, each trace logger now has its own resource key, which is shown in the Resource column in the output of status. This key allows multiple alarms, due to sensitive message types for multiple trace loggers.
DS-38053 The JWT Access Token Validator no longer requires a restart after a change to one of its signing certificates.
DS-38515 The requestURI, requestQueryParams, headers, and correlationID attributes of the HTTP request have been made available when constructing an Error Template.
DS-38560 Updated manage-profile replace-profile to apply configuration changes directly, when possible. If the new server profile used by replace-profile has changed only the dsconfig batch files from the original profile, then only the dsconfig files are applied. If no changes are detected between profiles, replace-profile takes no action. If changes other than dsconfig are detected, the full replace-profile process is followed.
DS-38597 The Policy Administration GUI setup has been redesigned, allowing users to generate configuration through a command-line tool more consistent with other Ping products.
DS-38777 Added support for updating the server version during manage-profile replace-profile. The server must have been originally set up with a server profile.
DS-38832 Fixed an issue that could cause the server to leak a small amount of memory each time it failed to establish an LDAP connection to another server.
DS-38832 A property has been added to Advice types that can limit their application to PERMIT or DENY decisions.
DS-38863 Updated the manage-profile setup subcommand to set a server's cluster name to match its instance name by default. This prevents servers in the same replication topology from being in the same cluster, reducing the risk of unintentionally overwriting parts of an existing server's configuration in a DevOps environment. The --useDefaultClusterName argument can be used to leave the cluster name unchanged.
DS-38867

Updated the PBKDF2 password storage scheme to add support for variants that use the 256-bit, 384-bit, and 512-bit SHA-2 digest algorithms. At present, the SHA-1 variant remains the default to preserve backward compatibility with older versions.

Also, in accordance with the recommendations in NIST SP 800-63B, we have increased the default iteration count from 4096 to 10,000, and the default salt length from 64 bits to 128 bits.

DS-38869 Updated the remove-defunct-server tool's --ignoreOnline option. When using --ignoreOnline in a mixed-version environment, all servers must support the option.
DS-38968 A new advice type, modify-query, has been added which can modify the request query parameters before the request to the upstream server is made.
DS-39037 The provided PingDataGovernance policies and deployment packages now apply access token validation policies to inbound, SCIM, and OpenBanking requests only. With this change, an access token is no longer required to issue a Sideband API response request.
DS-39176, DS-39308

Updated the Groovy scripting language version to 2.5.7. For a list of changes, go to groovy-lang.org and view the Groovy 2.5 release notes.

As of this release, only the core Groovy runtime and the groovy-json module are bundled with the server. To deploy a Groovy-scripted Server SDK extension that requires a Groovy module not bundled with the server, such as groovy-xml or groovy-sql, download the appropriate jar file from groovy-lang.org and place it in the server's lib/extensions directory.

DS-39253 Added a replace-certificate tool, which can help an administrator replace the listener or inter-server certificate for a server instance.
DS-39322 Added support for PingDataGovernance to the manage-profile tool and its subcommands.
DS-39490, DS-39616

The API Endpoint configuration type has been renamed to Gateway API Endpoint.

Any existing dsconfig scripts referencing an API Endpoint should be updated. For example, a dsconfig command of create-api-endpoint would need to be changed to create-gateway-api-endpoint.
DS-39518 Fixed an issue in which escaped characters in schema extensions may not be handled properly. If used in attribute type constraints (such as X-VALUE-REGEX), this could cause unexpected or incorrect behavior.
DS-39564 Fixed an issue in which the Gateway would respond with a 404 for requests handled by an API Endpoint with an inbound-base-path of "/".
DS-39592 HTTP External Servers have a new attribute, ssl-cert-nickname, which defines the alias of a specific certificate within their keystore to be used as a client certificate.
DS-39593 Fixed an issue where policy decision logs contained content that was considered invalid by the Policy Administration GUI Log Visualizer.
DS-39603 Fixed an issue where Server SDK extensions could not be configured by dsconfig batch files in the manage-profile tool.
DS-39626, DS-40357 The trace log publisher will now record an access token's scopes after the token is successfully validated.
DS-39643 Fixed an issue where a PUT request that attempted to delete less than 50 percent of the total items of a multivalued sub-attribute object resulted in the deletion of all items for that object.
DS-39654 Added support for the --topologyFilePath argument to the manage-topology add-server subcommand.
DS-39671 Updated the manage-topology add-server subcommand to require being run from the older server in a mixed-version environment.
DS-39681 When PingDataGovernance receives a 401 Unauthorized response from an external policy decision server, it will now convert the status to a 503 Service Unavailable for the upstream client.
DS-39715 Updated the Server SDK to add support for sending email messages.
DS-39735 The Server SDK's Advice API has been updated to provide the ability to modify multiple attributes of an HTTP request/response rather than just the body. Existing Advice extensions must be updated to use the new API.
DS-39857 Added the StatsD monitoring endpoint. When the Stats Collector Plugin is enabled, this endpoint sends metric data from the server in StatsD format to the configured destination.
DS-39877 Fixed an issue in which using an empty Error Template would cause the Sideband API to respond with a 500 Internal Server Error.
DS-39908 Added a new JVM-default trust manager provider that can be used to automatically trust any certificate signed by an authority included in the JVM's default set of trusted issuers. Also, updated other trust manager providers to offer an option to use the JVM-default trust addition to the trust that they normally provide.
DS-39913 Fixed a rare NullPointerException that could occur when recording advice metadata to the policy decision log.
DS-40114 Added a new cn=Status Health Summary,cn=monitor monitor entry that provides a summary of the server's current assessment of its health. This simplifies monitoring with third party tools that support retrieving monitoring data over JMX. The Periodic Stats Logger has also been updated to allow some of this monitoring information to be logged. No new information is logged by default.
DS-40234 The Open Banking account request endpoint no longer requires the x-fapi-financial-id to be present. Instead, it now includes the configured fapi-financial-id value in policy requests via Gateway.FapiFinancialId attribute. A policy can choose to deny account requests based on the presence and value of this attribute.
DS-40332 A check has been added to all DataGovernance policy requests which will cause them to fail if the version of the configured external Ping DataGovernance Policy Administration Point is not the same as the DataGovernance server. This will prevent potential errors that may otherwise arise from mismatched versions.
DS-40344 The API security gateway no longer forwards CORS-related request headers to upstream API servers. Likewise, it no longer forwards CORS-related response headers to clients. To use CORS with an API protected by the API security gateway, assign an HTTP Servlet Cross Origin Policy to the Gateway HTTP Servlet Extension.
DS-40354 Fixed a problem with config-diff when writing properties that span multiple lines using the --prettyPrint argument.
DS-40360 A new gauge has been created, DataGovernance Servlet Average Response Time (Milliseconds), which watches the average response time from Ping DataGovernance servlets. This gauge can generate alarms and affect the server's AVAILABLE or DEGRADED state. You must configure the gauge for it to have any effect; see the DataGovernance Server Administration Guide for details.
DS-40366 Fixed an issue where the server was attempting to connect by an IP address rather than a hostname when DNS lookup was successful.
DS-40371, DS-40382, DS-40427

SCIM 2 search responses can now be authorized and filtered with an optimized authorization mode that uses a single policy request to process the entire result set. This authorization mode is optional; by default, the server will continue to create a policy request for each member of a result set.

This authorization mode is enabled on a per-request basis. To enable, a policy that targets the 'SCIM2' service and the search action must provide an advice with the ID combine-scim-search-authorizations but with no payload. The subsequent search response is then authorized using a single policy request with the 'SCIM2' service and the search-results action. Any advices returned in the policy results are applied iteratively to each SCIM resource in the result set.

For more information, see the PingDataGovernance Server Administration Guide.

DS-40372 Added a new PDP API endpoint servlet extension. The PDP API endpoint accepts XACML-JSON requests and hands them off to the policy engine. It then converts the resulting policy decision to a XACML-JSON response for consumption by the client. To use this feature, customers must configure their PingDataGovernance servers with PDP API-enabled licenses.
DS-40377 Added support for logging to a JSON file in the Periodic Stats Logger Plugin.
DS-40517 Added metrics for status summary, replication database, and LDAP changelog to the Stats Collector Plugin.
DS-40542, DS-40554 Because the API Security Gateway may alter requests and responses as a result of policy processing, it no longer forwards request and response headers used for HTTP resource versioning and conditional requests. This includes the following headers: If-None-Match, If-Modified-Since, If-Unmodified-Since, ETag, and Last-Modified.
DS-40543 Updated manage-profile replace-profile to copy the tool log file to the server being updated.
DS-40556 Added support for specifying a working directory for exec tasks.
DS-40730 Updated the encrypt-file tool to prevent using the same path for both the input file and the output file.
DS-40771 Added a --duration argument to collect-support-data. When used, only the log files covering the specified duration before the current time will be collected.
DS-40784 Access Token Validator extensions built with the Server SDK may now provide the original access token value in addition to parsed claims when building a TokenValidationResult object. This access token value may be used by Token Resource Lookup Method extensions that do not need the parsed token claims to perform a subject lookup.