--- title: Upgrade AM instances description: To upgrade an AM deployment, you must upgrade each server instance in your site by upgrading the AM software on each server. You only need to upgrade the configuration on one server in the site. component: pingam version: 8.1 page_id: pingam:upgrade:upgrade-servers canonical_url: https://docs.pingidentity.com/pingam/8.1/upgrade/upgrade-servers.html keywords: ["Upgrade"] page_aliases: ["upgrade-guide:upgrade-servers.adoc"] section_ids: high_level_upgrade_steps: High-level upgrade steps before-you-upgrade: Before you upgrade upgrade-tomcat: Upgrade Tomcat upgrade-config: Upgrade the server and configuration upgrade-amupgrade: amupgrade amupgrade_rules: amupgrade rules before_you_start: Before you start upgrade_from_an_amster_configuration_export: Upgrade from an Amster configuration export upgrade_from_a_file_based_configuration: Upgrade from a file-based configuration upgrade-wizard: Upgrade wizard upgrade-rest: Upgrade over REST update_tools_scripts_and_components: Update tools, scripts, and components update-schema: Update the schema configuration_store: Configuration store identity_store: Identity store after_upgrade: After upgrade --- # Upgrade AM instances To upgrade an AM deployment, you must upgrade each *server instance* in your site by upgrading the AM software on each server. You only need to upgrade the configuration on *one* server in the site. | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | | * AM 8.1 supports Apache Tomcat 10 as a web application container. If you use Apache Tomcat, you *must* upgrade to at least version 10 before you upgrade to AM 8.1. * AM 8.1 supports a file-based configuration store (FBC) in production deployments. If you migrate your configuration to FBC, there is no longer the concept of separate *sites*. Each server has its own ID and its own local configuration. Learn more in [FBC and AM sites](../installation/fbc.html#fbc-and-am-sites). | ## High-level upgrade steps Upgrading AM involves the following high-level steps: * On one server in your site 1. If you use Apache Tomcat as an application container: 1. Stop AM or the container where it runs. 2. [Upgrade Tomcat](#upgrade-tomcat) if needed. 2. Upgrade the AM server software. The server is upgraded when you deploy the `.war` file of the new version. You must deploy the new `.war` file on each server in the site. 3. Upgrade the configuration. Use one of the following methods: * [`amupgrade`](#upgrade-amupgrade) This utility is provided in the `AM-8.1.0.zip` file and upgrades a file-based configuration (FBC) or a configuration exported using Amster. This is the recommended way to update the configuration. * [Upgrade wizard](#upgrade-wizard) The upgrade wizard is launched when you replace a deployed AM `.war` file with a newer version, then go to the deployment URL. * [Upgrade over REST](#upgrade-rest) Send a `GET` request to the `config/upgrade/upgrade.htm` endpoint to initiate the upgrade over REST. 4. Update the schema Generally, the configuration store schema is updated when you update the configuration. From time to time, updates are required to the schema of other datastores, such as the identity store or the CTS store. You must make these updates manually. Learn more in [Update the schema](#update-schema). 5. Update tools and scripts. Read the [Release notes](https://docs.pingidentity.com/pingam/release-notes) to understand the changes introduced in each version before you update AM tools and scripts. 6. Restart AM or the container where it runs. * On the remaining servers in your site 1. If you use Apache Tomcat as an application container: 1. Stop AM or the container where it runs. 2. [Upgrade Tomcat](#upgrade-tomcat) if needed. 2. Upgrade the AM server software. The server is upgraded when you deploy the `.war` file of the new version. You must deploy the new `.war` file on each server in the site. 3. Restart AM or the container where it runs. | | | | - | ---------------------------------------------------------------------------------------------------------------------------------------------- | | | If you're upgrading to AM 8.1 from an *unsupported* version of AM, you might need to upgrade to a supported version first, and then to AM 8.1. | ## Before you upgrade You *must* follow these steps before you start an upgrade. 1. [Plan your upgrade](upgrade-planning.html). 2. [Prepare a customized AM `.war` file](upgrade-planning.html#pre-upgrade-customization). 3. [Back up your deployment](upgrade-planning.html#pre-upgrade-backup). 4. [Route client application traffic to another site](upgrade-planning.html#plan-upgrade-downtime) during the upgrade. 5. Make the transient state encryption key available to *all instances* in the site. An AES 256-bit key called `directenctest` must be available to all instances in the site. This might mean, for example, copying the keystore across the site. This key doesn't need to be the same key that AM provides in the default keystore. If you don't provide this key, AM won't start up after the upgrade. > **Collapse: What is the key for?** > > AM uses the `directenctest` key to encrypt information stored in the [transient state](../auth-nodes/store-values-shared-state.html#store-values-in-transient-state) of authentication trees. > > The upgrade process maps this key to the `am.authn.trees.transientstate.encryption` secret label. > **Collapse: How do I make the key available?** > > The alias must exist in a secret store configured globally, so that all realms can access it. You can configure additional secrets by realm, if required, after the upgrade. > > \+ You can create a new secret store to house this secret, or you can add it to one of your existing stores. > > \+ The following example creates the key alias in the AM keystore, or in a keystore configured in a secret store: > > \+ > > ```bash > $ keytool \ > -genseckey \ > -alias directenctest \ > -keyalg AES \ > -keysize 256 \ > -storetype JCEKS \ > -keystore /path/to/keystore.jceks > ``` > **Collapse: Where do I find the keystore passwords?** > > The passwords are stored as secrets in a separate secret store. For example, a file system volume secret store. After the upgrade, you can rename the key alias or set a different key in the secret label mapping. Make sure the secret label is always mapped to an existing, resolvable secret or key alias. ## Upgrade Tomcat From AM 8.0 Apache Tomcat 10.x is supported. Support for earlier Tomcat versions is removed. If you use Apache Tomcat as an application container, you must install Apache Tomcat 10.x and deploy AM 8.1 into that container. Running AM 8 in an Apache Tomcat 8.5 or 9 container *does not work*. In addition to downloading and installing Apache Tomcat 10.x, make the following changes before you deploy AM: * Update any custom scripts, configurations, and allowlist entries to reflect the change to Jakarta EE 9 (Servlet API 5.0). In particular, change the following references: * `javax.servlet` → `jakarta.servlet` * `javax.websocket` → `jakarta.websocket` You don't need to change other `javax` references, such as `javax.xml` and `javax.inject`. * The Tomcat system property to manage encoded slash (`/`) characters in requests has changed. The `org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH` property has been removed in Tomcat 10 and replaced with the `encodedSolidusHandling` property. Update references to the old property in the `Connector` element in your Tomcat `server.xml` file. | | | | - | ------------------------------------------------------------------------------ | | | In production environments, avoid resource names that contain forward slashes. | ## Upgrade the server and configuration Use one of the following methods to upgrade the AM server software and configuration: ### `amupgrade` The `amupgrade` utility converts AM configuration files so that the configuration can be used with the latest AM version. Use this utility with `Amster` and to upgrade file-based configurations. The utility has the same Java requirements as AM. Learn more about these [requirements](https://docs.pingidentity.com/pingam/release-notes/requirements.html) in the Release notes. #### `amupgrade` rules The `amupgrade` utility uses *upgrade rules* to convert configuration between versions. The upgrade rules are *sequential*, that is, they upgrade configuration from one version to the next [minor](https://docs.pingidentity.com/pingam/release-notes/stability.html#release-levels) consecutive version. If you're upgrading to a version greater than the next minor consecutive version, you must run multiple `amupgrade` commands sequentially until you reach the required configuration. In other words, if you're upgrading your configuration from version 7.2.x to version 8.0.x: 1. Run `amupgrade` with the `7.2.x-to-7.3.x.groovy` rules file to get a 7.3 configuration. 2. Run `amupgrade` on the 7.3 configuration with `7.3.x-to-7.4.x.groovy` rules file to get a 7.4 configuration. 3. Continue running successive upgrades until you have an 8.0 configuration. | | | | - | ----------------------------------------------------------- | | | You can't use `amupgrade` to upgrade a running AM instance. | #### Before you start 1. Download the AM `.zip` file from the [Ping Identity Download Center](https://backstage.pingidentity.com/downloads). 2. Extract the contents of the `.zip` file. 3. In the extracted directory, locate the `Config-Upgrader-8.1.0.zip` file. 4. Extract the `Config-Upgrader-8.1.0.zip` file. #### Upgrade from an `Amster` configuration export 1. [Export configuration data](../amster/export-config.html). 2. Upgrade the exported configuration to the new version: ```bash $ amupgrade \ -i exported configuration \ -o output directory \ -a amster version \ rules/amster/from-to-to.groovy ``` Where: * exported configuration is the path to the configuration directory you generated in step 1 * output directory is the directory to which the command writes the upgraded configuration * amster version is the version of Amster you will use to import the configuration * from is the AM version from which you're upgrading * to is the AM version to which you're upgrading * *rules* specifies the path to a configuration rules file in the `/path/to/amupgrade/rules` directory For example: ```bash $ amupgrade \ -i /path/to/AM7.5Config/ \ -o /path/to/AM8Config \ -a 8.1.0 \ rules/amster/7.5.x-to-8.1.x.groovy Reading existing configuration from files in /path/to/AM7.5-config/…​ Modifying configuration based on rules in [rules/amster/7.5.x-to-8.1.x.groovy]…​ reading configuration from Amster json files Writing configuration to new location at /path/to/AM8Config…​ Upgrade Completed, modified configuration saved to /path/to/AM8Config ``` 3. Install a new version of DS with an [`am-config` profile](https://docs.pingidentity.com/pingds/8.1/install-guide/profile-am-config.html). Don't upgrade or use an existing DS configuration store because configuration associated with the older DS version can cause conflicts. 4. Stop AM or the container where it runs. 5. Unconfigure AM by removing the configuration files in the $HOME directory of the user running the web application container. For example: ```bash $ rm -rf $HOME/am $HOME/.openamcfg ``` To uninstall AM and its associated configuration files, delete the following directories: * *The configuration directory*. If you didn't use the default configuration location (`$HOME/am`), check the value of the `Base installation directory` property under Deployment > Servers > *server name* > General > System. * *The hidden directory that holds a file pointing to the configuration directory*. For example, if you are using Apache Tomcat as the web container, this file could be `$HOME/.openamcfg/AMConfig_path_to_tomcat_webapps_am_`. 6. Undeploy the AM web application. For example, if you are using Apache Tomcat as the web container, remove the `.war` file and expanded web application from the container: ```bash $ cd /path/to/tomcat/webapps/ $ rm -rf am.war am/ ``` 7. [Upgrade Tomcat](#upgrade-tomcat). 8. [Install](../installation/interactive-install.html) the new version of AM using the new DS server for your configuration store. 9. [Import configuration data](../amster/import-config.html). 10. Restart AM or the container where it runs. #### Upgrade from a file-based configuration 1. Run `amupgrade` to upgrade the configuration: ```bash $ amupgrade \ -i existing-config \ -o output-directory \ --fileBasedMode \ /path/to/amupgrade/rules/fbc/latest.groovy \ --ignoreNoRuleTracking ``` Where: * *existing-config* is the path to your existing FBC directory * *output-directory* is the directory to which the command writes the upgraded configuration * `--fileBasedMode` indicates that you're upgrading file-based configuration (FBC) rather than configuration exported with Amster * `rules` specifies the path to an upgrade rules file in the `/path/to/amupgrade/rules` directory * `--ignoreNoRuleTracking` lets you upgrade without specifying a configuration version For example: ```bash $ amupgrade \ -i /path/to/am/config/services/ \ -o /tmp/upgraded-services \ --fileBasedMode \ /path/to/amupgrade/rules/fbc/latest.groovy \ --ignoreNoRuleTracking ``` 2. Stop AM or the container in which it runs. 3. Move the existing configuration from the AM configuration directory to a temporary location: ```bash $ mv /path/to/am/config/services/* /tmp/old-services/ ``` 4. Move the upgraded configuration into the AM configuration directory: ```bash $ mv /tmp/upgraded-services/* /path/to/am/config/services/ ``` 5. Copy the `noninteractive-install.properties` from the old services directory to the AM configuration directory: ```bash $ cp /tmp/old-services/noninteractive-install.properties /path/to/am/config/services/ ``` | | | | - | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | You can also set the startup properties as environment variables. Find more information in [FBC initial startup](../installation/passive-install-fbc.html#fbc-startup). | 6. Deploy your customized AM `.war` file. When you deploy the new `.war` file, you might need to delete working files left by the old deployment. For example, if you deploy on Apache Tomcat, replace `/path/to/tomcat/webapps/am.war` with the customized `.war` file, then recursively delete the `/path/to/tomcat/webapps/am/` and `/path/to/tomcat/work/Catalina/localhost/am/` directories before restarting the server. 7. Restart AM or the container where it runs. ### Upgrade wizard The upgrade wizard brings the AM configuration, including the version number, up to date with the new version. 1. Stop AM or the container where it runs. 2. Deploy your customized AM `.war` file. When you deploy the new `.war` file, you might need to delete working files left by the old deployment. For example, if you deploy on Apache Tomcat, replace `/path/to/tomcat/webapps/am.war` with the customized `.war` file, then recursively delete the `/path/to/tomcat/webapps/am/` and `/path/to/tomcat/work/Catalina/localhost/am/` directories before restarting the server. 3. Restart AM or the container where it runs. 4. After deploying AM, but before upgrading, your application container serves AM's upgrader user interface. Suspend any external network access to the application container until the upgrade is complete. After the upgrade is complete, AM prevents access to the upgrader UI. 5. Go to the deployment URL and follow the prompts to upgrade. 6. Restart AM or the container where it runs. 7. Update the identity store schema: 1. Log into AM. 2. Go to Realms > *realm name* > Identity Stores. 3. Select your external identity store and toggle Load Schema to ON. 4. Click Save to apply your changes. 5. If you have additional identity stores, repeat the previous steps for each store. | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | * The AM upgrade wizard uses three libraries that should be removed after upgrade, for security reasons. When your upgrade is complete, remove the following .jar files from the `WEB-INF/lib` directory: * `click-extras-2.3.0.jar` * `click-nodeps-2.3.0.jar` * `velocity-1.7-p1.jar` These files are used *only* by the install and upgrade wizards. Removing them won't affect your installed instance. You must also remove the references to `click-servlet` from the deployment descriptor file. Edit `/path/to/tomcat/webapps/am/WEB-INF/web.xml` to remove the following mappings: ```xml click-servlet org.apache.click.ClickServlet ... click-servlet *.htm ``` | ### Upgrade over REST Sending a `GET` request to the `config/upgrade/upgrade.htm` endpoint initiates the same upgrade process as the upgrade wizard, and brings the AM configuration, including the version number, up to date with the new version. 1. Stop AM or the container where it runs. 2. Deploy your customized AM `.war` file. When you deploy the new `.war` file, you might need to delete working files left by the old deployment. For example, if you deploy on Apache Tomcat, replace `/path/to/tomcat/webapps/am.war` with the customized `.war` file, then recursively delete the `/path/to/tomcat/webapps/am/` and `/path/to/tomcat/work/Catalina/localhost/am/` directories before restarting the server. 3. Restart AM or the container where it runs. 4. Send an HTTP GET request as follows: ```bash $ curl "https://am.example.com:8443/am/config/upgrade/upgrade.htm?actionLink=doUpgrade&actionLink=saveReport&acceptLicense=true" true ``` 5. Restart AM or the container where it runs. 6. Update the identity store schema: 1. Log into AM. 2. Go to Realms > *realm name* > Identity Stores. 3. Select your external identity store and toggle Load Schema to ON. 4. Click Save to apply your changes. 5. If you have additional identity stores, repeat the previous steps for each store. | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | * The AM upgrade wizard uses three libraries that should be removed after upgrade, for security reasons. When your upgrade is complete, remove the following .jar files from the `WEB-INF/lib` directory: * `click-extras-2.3.0.jar` * `click-nodeps-2.3.0.jar` * `velocity-1.7-p1.jar` These files are used *only* by the install and upgrade wizards. Removing them won't affect your installed instance. You must also remove the references to `click-servlet` from the deployment descriptor file. Edit `/path/to/tomcat/webapps/am/WEB-INF/web.xml` to remove the following mappings: ```xml click-servlet org.apache.click.ClickServlet ... click-servlet *.htm ``` | ## Update tools, scripts, and components 1. Update Amster. Follow [Install Amster](../amster/install-amster.html) to install an updated version of Amster. When you have confirmed the new tools are working, delete the old tools. 2. Make sure the AM scripts are current and contain the modifications your environment requires. To avoid overwriting changes made in customized scripts, the upgrade process doesn't update scripts from earlier versions of AM. Check the scripts in your environment are compatible with the version of AM you're upgrading to by following these steps: 1. Read the [Release notes](https://docs.pingidentity.com/pingam/release-notes) for information about possible changes. 2. Install an AM 8.1.0 test environment, and compare the scripts. New installations always have the current scripts. 3. Review the information in [Upgrade components and services](upgrade-components.html) and decide if you need to reconfigure any of AM's services or features. | | | | - | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Reconfigure any custom advanced properties if necessary. These properties are lost during the upgrade, and you'll need to re-add them in the AM admin UI.> **Collapse: How do I configure advanced server properties?** > > * To configure advanced server properties for all instances of the AM environment, go to Configure > Server Defaults > Advanced in the AM admin UI. > > * To configure advanced server properties for a particular instance, go to Deployment > Servers > *server name* > Advanced. > > If the property you want to add or edit is already configured, click on the pencil ([icon: pencil-alt, set=fas]) button to edit it. When you are finished, click on the tick ([icon: check, set=fas]) button. > > Click Save Changes. | ## Update the schema You might need to update the schema for specific datastores, depending on the version from which you are upgrading and the datastore type. ### Configuration store Generally, updating your configuration makes the required schema updates to the configuration store. After you've updated the configuration, add an access control instruction (ACI) to the configuration store directory to give the AM administrative user server-side sorting privileges. The ACI should be similar to the following: ```ldif aci: (targetcontrol="1.2.840.113556.1.4.473")(version 3.0; acl "Allow server-side sorting"; allow (read) (userdn = "ldap:///uid=am,ou=admins,dc=example,dc=com");) ``` You can find more information about configuring AM configuration stores in [Prepare configuration stores](../installation/prepare-configuration-store.html). ### Identity store Depending on the version you're upgrading from, and the features you've configured, you might need to update your identity store schema manually. * Upgrade from AM 7.0 with a PingDS identity store 1. In the path where you deployed the `am.war` file (for example, `/path/to/tomcat/webapps/am`) locate the following file in the `WEB-INF/template/ldif/opendj` directory: `opendj_retry_limit_node_count.ldif` 2. Update the identity store schema as follows: ```bash $ /path/to/opendj/bin/ldapmodify \ --hostname 'ds.example.com' \ --port 1636 \ --useSsl \ --usePkcs12TrustStore /path/to/opendj/config/keystore \ --truststorepassword:file /path/to/opendj/config/keystore.pin \ --continueOnError \ --bindDN uid=admin \ --bindPassword str0ngAdm1nPa55word \ /path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_retry_limit_node_count.ldif ``` | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | This schema update is required for the Save Retry Limit to User feature in the [Retry Limit Decision node](https://docs.pingidentity.com/auth-node-ref/8.1/retry-limit-decision.html). The feature is enabled by default.Even if you aren't currently using the Retry Limit Decision node, you *should* make this schema update, in case you decide to use the node in the future. If you can't update the identity store schema at this point, you *must* disable the feature. | * Using push or web authentication, or using the Ping SDKs for device profiling Apply the following LDIF files: * `/path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_webauthndevices.ldif` * `/path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_deviceprofiles.ldif` * `/path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_bounddevices.ldif` For example: ```bash $ /path/to/opendj/bin/ldapmodify \ --hostname 'ds.example.com' \ --port 1636 \ --useSsl \ --usePkcs12TrustStore /path/to/opendj/config/keystore \ --truststorepassword:file /path/to/opendj/config/keystore.pin \ --continueOnError \ --bindDN uid=admin \ --bindPassword str0ngAdm1nPa55word \ /path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_webauthndevices.ldif \ /path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_deviceprofiles.ldif \ /path/to/tomcat/webapps/am/WEB-INF/template/ldif/opendj/opendj_bounddevices.ldif ``` If you don't make this schema change, you *must* remove the `webauthnDeviceProfilesContainer` object class from the user configuration after the upgrade: 1. In the AM admin UI, go to Realms > *realm name* > Identity Stores > *identity store name*. 2. On the User Configuration tab, remove `webauthnDeviceProfilesContainer` from the LDAP User Object Class. 3. Save your changes. Make the same change for each identity store that doesn't have the schema change, and in each realm that uses the identity store. ## After upgrade 1. Validate the service is performing as expected. 2. Allow client application traffic to flow to the upgraded site.