Use Cases

Performing a near-zero downtime PingFederate upgrade

If you cannot incur a full PingFederate outage when you upgrade, you can perform a near-zero downtime upgrade.

What it is

Best practice when possible is to schedule downtime when upgrading PingFederate. During a scheduled downtime upgrade, PingFederate is unavailable for the entire duration of the upgrade. This can create a significant user impact in some scenarios, such as if your company is globally based, and your users need access to resources 24 hours a day.

Although there is no officially supported method for zero downtime PingFederate upgrades at this time, if you cannot incur a full outage, you can perform a near-zero downtime upgrade. A near-zero downtime upgrade is a method of upgrading that can reduce the PingFederate downtime to minutes, minimizing the impact to your users.

This document outlines how to execute the upgrade without incurring service downtime or unexpected results caused by mixed version nodes communicating on the cluster communication ports. It does not cover all the upgrade considerations, pre-upgrade planning, pre-upgrade tasks, or post-upgrade tasks required to be successful. All documentation links refer to the latest version.

During a near-zero downtime upgrade of the runtime engines, you essentially split the cluster into two using the cluster auth password, then bring it back together into one cluster at the end.

The success of this process largely depends on isolating traffic from engine nodes and changing the cluster password so that the nodes on different versions are never joined to the cluster concurrently. A portion of your cluster must be capable to handle 100% of traffic while the rest is upgraded.

The Upgrade Utility does not affect your existing source deployment. It creates a new upgraded installation side by side.

Some requests could fail during the upgrade if states and references generated on nodes won’t replicate or might not be available to the full cluster. If that occurs, the user must authenticate again.

What you’ll need

  • Have persistent authentication sessions enabled. This can reduce disruption during the upgrade, as sessions will be stored in external storage and not only in node memory. Learn more in Sessions and Defining a datastore for persistent authentication sessions.

  • Have all proper license files ready if you are going to a new major version.

  • Arrange for access to your load balancer.

  • Schedule your upgrade during off hours to minimize impact on your business.

  • Verify that a portion of your cluster is capable of handling 100% of traffic, or the traffic expected during off hours, while you are upgrading the rest of your cluster.

  • Review the upgrade documentation that’s specific to the version to which you are upgrading as well as for any releases between your current production release and that version to plan your pre and post-upgrade tasks. Learn more about upgrade specifics in Upgrading PingFederate in the PingFederate documentation.

What you’ll do

To perform your upgrade, you’ll:

  1. Upgrade the admin node.

  2. Upgrade the first half of the engine nodes.

  3. Upgrade the remaining engine nodes.

  4. Restore the user traffic flow to all of the engine nodes.

Step 1: Upgrade the admin node

Because it doesn’t serve any user traffic, you’ll begin your upgrade on the admin node.

  1. On the admin node, stop the PingFederate service.

  2. Run the upgrade utility command.

    Learn more in Upgrading PingFederate installations.

    • If you’re upgrading a Windows install, change the file service properties in the upgraded node’s <pf_install>/pingfederate/sbin/wrapper/PingFederateService.conf to a unique name so that you can start either the new version or the old version if you need to roll back.

    • If you’re upgrading a Linux install, modify the init script to point to the new location.

  3. Before you start up the service for the new version, change the pf.cluster.auth.pwd in the run.properties file so that it cannot communicate with the nodes that have not been upgraded yet.

  4. Start the new admin node service.

  5. Sign on to the PingFederate administrative console, upload the new license if needed, and validate that your configuration is correct.

Step 2: Upgrade the first half of the engine nodes

Next, you’ll upgrade the first half of your engine nodes while the remaining servers continue to process requests.

  1. Stop the traffic flow to 50% of your runtime servers.

  2. Stop the PingFederate service on the first set of engine nodes that you are upgrading.

  3. Run the upgrade utility command..

    • If you’re upgrading a Windows install, change the file service properties in the upgraded node’s <pf_install>/pingfederate/sbin/wrapper/PingFederateService.conf to a unique name so that you can start either the new version or the old version if you need to roll back.

    • If you’re upgrading a Linux install, modify the init script to point to the new location.

  4. Before you start up the service for the new version, change the pf.cluster.auth.pwd in the run.properties file to the new password that you used on the upgraded admin node.

  5. Start the engine nodes service and validate that they are working as expected by pointing the hosts file on a client to resolve the PingFederate base URL to each upgraded node.

    The admin node should see these upgraded nodes joined to the new cluster.

Step 3: Upgrade the remaining engine nodes

After you upgrade the first half of your engine nodes, you’ll upgrade the rest of the engine nodes while the nodes that you upgraded in step 2 process requests.

  1. After validating that the load balancer works as expected, switch all user traffic flow to the upgraded nodes.

  2. Stop traffic to the remaining nodes.

  3. Stop the PingFederate service on the remaining engine nodes to be upgraded.

  4. Run the upgrade utility command:

    • If you’re upgrading a Windows install, change the file service properties in the upgraded node’s <pf_install>/pingfederate/sbin/wrapper/PingFederateService.conf to a unique name so that you can start either the new version or the old version if you need to roll back.

    • If you’re upgrading a Linux install, modify the init script to point to the new location.

      Do not change the pf.cluster.auth.pwd in the run.properties file on these nodes until you validate that they work as expected.

  5. Start the engine nodes service and validate that they are working as expected by pointing the hosts file on a client to resolve the PingFederate base URL to each upgraded node.

  6. After you’ve successfully validated that the nodes are working correctly, change the pf.cluster.auth.pwd in the run.properties file to the new password that you used on the upgraded admin node, then restart the engine nodes service.

    The admin node should see all of the engine nodes joined to the new cluster.

    This is when the downtime in near-zero downtime occurs.

Step 4: Restore user traffic flow

After you’ve upgraded all of your nodes and validated that they’re working correctly, you’ll restore the user traffic flow to all of them.

What’s next

Learn more about your upgraded version in the PingFederate documentation.