IDM 7.3.1

Upgrade

This guide shows you how to upgrade an existing deployment to the latest ForgeRock® Identity Management release.

The upgrade process is largely dependent on your deployment and on the extent to which you have customized IDM. Engage ForgeRock Support Services for help in upgrading an existing deployment. Also, read the Release notes before you start an upgrade; specifically, Incompatible changes.

ForgeRock Identity Platform™ serves as the basis for our simple and comprehensive Identity and Access Management solution. We help our customers deepen their relationships with their customers, and improve the productivity and connectivity of their employees and partners. For more information about ForgeRock and about the platform, refer to https://www.forgerock.com.

The ForgeRock Common REST API works across the platform to provide common ways to access web resources and collections of resources.

About upgrades

The automated update process available with previous IDM versions is no longer supported. This chapter describes the manual process required to upgrade an existing IDM deployment. At a high level, the manual update process involves the following steps:

  1. Install IDM 7.3.

  2. Migrate your existing IDM configuration to the new installation.

  3. Update your repository.

  4. Test your scripts and customizations work as expected.

  5. Migrate existing data to the new installation.

Supported upgrade paths

The following table contains information about the supported upgrade paths to IDM 7.3:

Upgrade Paths
Version Upgrade Supported to IDM 7.3

IDM 7.2.x

YES

IDM 7.1.x

YES

IDM 7.0.x

YES

IDM 6.5.x

YES

IDM 6.0.x

YES

IDM 5.5.x

YES

IDM 5.0.x

YES

Depending on how you have customized your deployment, there might be incompatible configuration changes when you upgrade from versions prior to IDM 6.5.x. Read the upgrade documentation for each interim release and apply all required script and configuration changes.

Before you upgrade

Fulfill these requirements before you upgrade IDM, especially before upgrading the software in a production environment. Also refer to the requirements listed in Before you install and the changes listed in Incompatible changes.

Before you start, verify that you have a supported Java version installed:

Supported Java Versions
Vendor Versions

OpenJDK, including OpenJDK-based distributions:

  • AdoptOpenJDK/Eclipse Temurin

  • Amazon Corretto

  • Azul Zulu

  • Red Hat OpenJDK

ForgeRock tests most extensively with AdoptOpenJDK/Eclipse Temurin. ForgeRock recommends using the HotSpot JVM.

11, 17*

Oracle Java

11, 17*

* Version 17.0.3 or higher.

If the server uses an older version that is no longer supported, install a newer Java version before you update, and follow the instructions in Java requirements.

Then, follow these steps:

  1. Back up your existing deployment by archiving the openidm directory and creating a backup of the repository and all other applicable databases.

    If you use workflow, you must manually dump the workflow database tables, and then import them before you start the new instance of IDM for the first time. The workflow database tables start with the prefix ACT_. For information on how to dump/import individual tables, refer to the documentation for your database.

  2. To save a record of the audit logs from your existing IDM installation, manually copy the log files from the /path/to/openidm/audit/ directory, before you start the upgrade.

  3. Download and extract IDM-7.3.1.zip from the ForgeRock BackStage download site.

Migrate your configuration

This chapter covers the steps required to migrate your IDM configuration to IDM 7.3.

There is no automated way to migrate a customized configuration to IDM 7.3, so you must migrate customized configuration files manually. Assuming you are upgrading from IDM 7.2.x, there are three ways to do this:

  • Use the new IDM 7.3 configuration files as a base, and copy any customizations you have made to the new files.

    This is the preferred option, particularly if you have used version control on your configuration and can determine the exact changes you have applied.

  • Use your existing configuration files as a base, and add any new IDM 7.3 configuration to your existing files.

  • Use your existing configuration "as is" with no IDM 7.3 changes.

In most cases, a customized IDM 7.2.x configuration will work without further modification on IDM 7.3.

Migrate configuration files

For customized files in your project’s conf/ directory, check that the customizations are compatible with the changes outlined in Incompatible changes. If there are no incompatible changes, either copy your old configuration files to your IDM 7.3 installation, or copy any customization into the corresponding new configuration files.

If you create custom configuration files, ForgeRock recommends not using spaces or special characters in the filenames, in accordance with the OSGi specification.

Migrate boot.properties

On the IDM 7.3 installation, edit the resolver/boot.properties file to match any customizations that you made on your IDM 7.2.x server. Specifically, check the following elements:

  • The HTTP, HTTPS, and mutual authentication ports.

    If you changed the default ports in your IDM 7.2.x deployment, make those same changes in the new boot.properties file.

  • Check that the keystore and truststore passwords match the current passwords for the keystore and truststore of your existing IDM deployment.

Migrate security settings

Copy the contents of your IDM 7.2.x security/ folder to the IDM 7.3 installation.

If you do not copy your old truststore and keystore files to your new instance, you will be unable to decrypt anything that was encrypted by your old instance of IDM.

Migrate custom scripts

Migrate any custom scripts or default scripts that you have modified to the script directory of your IDM 7.3 instance. In general, custom and customized scripts should be located in the openidm/script directory of your existing IDM deployment.

For custom scripts, review Incompatible changes. If you are confident that the scripts will work as intended on IDM 7.3, copy these scripts to the new instance.

If you modified a default IDM script, compare the default versions of the IDM 7.2.x and IDM 7.3 scripts. If nothing has changed between the default versions, review your customizations against Incompatible changes. If a default script has changed since the IDM 7.2.x release, test that your customizations work with the new default script. If you are confident that your changes will work as intended on the new version, copy the customized scripts to the new script directory.

If you modify any shell scripts, such as startup.sh, you must migrate your changes manually to the new version of the script.

Migrate custom bundles

If your existing deployment includes any custom JAR files in the bundles directory, migrate these to the new deployment. Pay particular attention to any files that support JDBC database drivers.

Migrate provisioner files

Change any customized provisioner configurations in your existing deployment to point to the connectors that are provided with IDM 7.3. Specifically, make sure that the connectorRef properties reflect the new connector versions, where applicable. For example:

"connectorRef" : {
    "bundleName": "org.forgerock.openicf.connectors.ldap-connector",
    "bundleVersion": "[1.4.0.0,1.6.0.0)",
    "connectorName": "org.identityconnectors.ldap.LdapConnector"
},

Alternatively, copy the connector .jar files from your existing deployment into the openidm/connectors directory of the new installation.

Migrate UI customizations

If you have customized the admin UI, review any custom UI files from your IDM 7.2.x deployment (generally in the openidm/ui/admin/extension directory), and compare them against the corresponding IDM 7.3 files.

For each customized file, copy the corresponding default IDM 7.3 UI files to a openidm/ui/admin/extension directory on the new instance.

Apply your customizations to files in the new openidm/ui/admin/extension directory.

Update the repository

When you have migrated your configuration to the new IDM installation, you need to handle the data that is stored in your repository. There are two options to update a repository:

When you have upgraded the repository, or created a new repository, start the IDM server and test that all your scripts are working as expected, before migrating your data.

Upgrade an existing repository

Upgrading an existing repository means that you do not need to migrate data. However, you must run a series of scripts that modify the repository, to use the new features in IDM 7.3.

Because the repository upgrade scripts are incremental, you must review each major version upgrade after your current release. For example, when upgrading from 6.5.x to 7.3.x, review the upgrade process and scripts for 7.0.x, 7.1.x, 7.2.x, and 7.3.x (this version).

Repository upgrade procedures:

Prepare an existing repository for IDM 7.3 as follows:

  1. Shut down IDM, if it is running.

  2. Clear all configobjects related tables. For example, in MySQL run:

    DELETE FROM openidm.configobjects;
DELETE FROM openidm.configobjectproperties;

  3. From your IDM 7.3 installation, run the schema update scripts for your database type:

    • For Microsoft SQL (MSSQL) only, /path/to/openidm/db/mssql/scripts/updates/00-update-to-nvarchar.sql. This script updates the deprecated SQL data type NTEXT to NVARCHAR(MAX).

      This script is only required for explicit managed_user tables.

  4. If you are using workflow, you must run the Flowable upgrade scripts for your database type. These upgrade scripts are incremental and must be run in order, starting with the correct script based on your current Flowable version.

    1. To determine your current Flowable version, check the /path/to/openidm/bundle/flowable-engine-versionNumber.jar file in your old IDM installation.

    2. Run the upgrade scripts from /path/to/openidm/db/database-type/scripts/updates/ in order, starting with your current flowable version:

      1. flowable.database-type.upgradestep.6.6.0.to.6.7.0.all.sql

      2. flowable.database-type.upgradestep.6.7.0.to.6.7.1.all.sql

      3. flowable.database-type.upgradestep.6.7.1.to.6.7.2.all.sql

      4. flowable.database-type.upgradestep.6.7.2.to.6.8.0.all.sql

  5. Launch IDM and run the following Groovy script to clear the reconprogressstate data in your repository:

    def result = openidm.query(
  "repo/reconprogressstate", [ "_queryFilter" : "true", "_fields" : "_id" ]).result;
for ( item in result ) {
  openidm.delete("repo/reconprogressstate/" + item["_id"], null);
}
return result.size() + " reconprogressstate records deleted";

    This script works for all repository types and can be sent as a REST call. For example:

    curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
--data '{
  "type":"groovy",
  "source":"def result = openidm.query(\"repo/reconprogressstate\", [ \"_queryFilter\" : \"true\", \"_fields\" : \"_id\" ]).result; for ( item in result ) { openidm.delete(\"repo/reconprogressstate/\" + item[\"_id\"], null); }; return result.size() + \" reconprogressstate records deleted\";"
}' \
"http://localhost:8080/openidm/script?_action=eval"
"1 reconprogressstate records deleted"

  6. Verify that all scripts and functions behave as expected.

Create a new repository

  1. Set up a new repository, following the steps in Select a repository. A new repository is already configured for all the new capabilities in IDM, but does require migrating existing data to that repository.

    If you create a new repository, you must still update your configuration files to use the new features.

  2. After you have set up the new repository, migrate your data to that repository.

Migrate data

The data migration service helps you move information stored in an IDM repository to a new deployment. You can use this service when you are upgrading to a new version, or when you are migrating to a different repository type. The migration service is off by default. To enable it, copy migration.json from samples/example-configurations/conf/ into your conf/ directory, and set "enabled": true.

Migration is run from your new installation through IDM’s recon service, using your previous deployment as a data source. The data migration service supports importing information from IDM instances back to version 4. If you are migrating from a version of IDM earlier than that, you will need to follow previous update instructions to get your deployment into a state where it can be migrated using this service.

Because the migration service migrates information that may be encrypted, such as passwords, you must make sure you have copied the truststore and keystore files from your previous deployment before you start the migration.
Default Data Imported by the Migration Service

  • Internal Roles

  • Internal Users

  • Internal User Metadata

  • Managed Roles

  • Managed Users

  • Managed Assignments

  • Links and Relationships

  • Scheduler jobs

    If you are migrating scheduler jobs from IDM 4.0 or 4.5, you will need to modify the entry in migration.json to be:

    {
    "source" : "scheduler",
    "target" : "scheduler/job"
}

If you have additional object types (for example, managed devices), modify migration.json to include these objects.

Configure the Migration Service

The data migration service is configured through migration.json. The default file assumes a default schema; modify the file if you have added custom managed data. The migration.json file can have the following properties:

enabled

Boolean, true or false. Enables the migration service.

connection

Configures the connection to the source IDM instance you are migrating from. Available properties:

instanceUrl

The URI for the source IDM instance.

authType

The authentication mechanism to the source IDM instance. Can be basic (username/password) or bearer (authentication using AM bearer tokens).

userName

Used for authenticating to the source IDM instance, if the authType is basic.

password

Used for authenticating to the source IDM instance, if the authType is basic.

clientId

Used for authenticating to the source IDM instance, if the authType is bearer.

clientSecret

Used for authenticating to the source IDM instance, if the authType is bearer.

tokenEndpoint

Used for authenticating to the source IDM instance, if the authType is bearer.

scope (optional)

List of OAuth scopes.

scopeDelimiter (optional)

Delimiter for the list of OAuth scopes.

tlsVersion (optional)

Lets you override the default TLS version.

connectionTimeout (optional)

Timeout for connecting to the source IDM instance (defaults to 10s).

reuseConnections (optional)

Lets you override the default setting (defaults to true).

retryRequests (optional)

Lets you override the default setting (defaults to true).

hostnameVerifier (optional)

The SSL hostname verification policy. Specifies whether the host name presented by the remote server certificate is verified upon establishing new SSL connections (defaults to STRICT). Possible values:

  • STRICT : Requires that the host name match the host name presented in the certificate. Wild-cards only match a single domain.

  • ALLOW_ALL : Accepts any host name (disables host name verification).

maxConnections (optional)

Lets you override the default maximum number of connections (default is 64).

proxy (optional)

Lets you specify connection through a proxy server. Includes the following properties:

proxyUri

The proxy host and port to which IDM should connect.

userName

The user account to connect to the remote proxy.

password

The password of the proxy user.

socketTimeout

The TCP socket timeout, when waiting for HTTP responses. If you do not set a duration, the default is no timeout.

Example valid duration values:

  • 4 days

  • 59 minutes and 1 millisecond

  • 1 minute and 10 seconds

  • 42 millis

  • unlimited

  • none

  • zero

mappings

A list of the endpoints that will be migrated from your old IDM instance to your new instance, expressed as mappings between the old and new instances. The complete list of mapping properties is the same as any regular synchronization mapping. Properties with particular significance for data migration include the following:

allowEmptySourceSet

Specifies whether the migration service should continue if it encounters an empty source mapping. This is enabled by default.

correlationQuery

You can specify a custom correlation query. By default, this is:

"var map = {'_queryFilter': '_id eq \"' + source._id + '\"'}; map;"

For more information about writing correlation queries, refer to Correlate source objects with existing target objects.

enableLinking

Specifies whether links are maintained between source and target objects. If enableLinking is set to false, links are not maintained. This is the default behavior for the migration service, where it is expected that you will run the migration only once. If you intend to run the migration more than once, set this parameter to true.

onCreate

The script used by the migration service for creating the data that is being migrated to the new installation. By default, this points to a Groovy script: update/mapLegacyObject.groovy.

onUpdate

The script used by the migration service for updating the data that is being migrated in the new installation. By default, this points to a Groovy script: update/mapLegacyObject.groovy.

policies

An array of policies to apply to the data being migrated.

properties

An array of properties to perform additional actions on, such as modifying the contents of a property during the migration. This follows the pattern you would find in a standard reconciliation. For more information about transforming data during a reconciliation, refer to Transform Attributes in a Mapping.

reconSourceQueryPageSize

Specifies the number of results to return per page, if paging is turned on. By default, 1000 results per page are returned.

reconSourceQueryPaging

Specifies whether the migration service should use paging when querying the source IDM instance. By default, this is set to false. Turn paging on if you have a large data set and are concerned about memory usage.

For large data sets, you might be able to improve migration performance by turning paging on and increasing the query page size (using reconSourceQueryPageSize). The most effective page size will vary, depending on the available resources.

runTargetPhase

Specifies whether the migration should run the target phase of reconciliation. By default, this is set to false as there is no data in the target repository.

source

This is the only property that is required for data migration. The source should be the path to the resource within the repo; for example, repo/managed/user.

By default, the migration services use the repo endpoint, rather than the managed endpoint for both the source and the target. Create, read, update, and delete operations will therefore not trigger an implicit synchronization to the target resource.
sourceQuery

The query on the source system, used to find all objects to be migrated. Defaults to "_queryFilter" : "true&fields=_id", which returns the IDs of all source objects.

You can improve migration performance by returning the whole source entry (setting the sourceQuery to "_queryFilter" : "true").

If you are migrating from IDM 6.5.x

Any explicitly mapped resource coming from repo/<mappingName> must include:

"sourceQuery": {
    "_queryFilter": "true",
    "_fields": ""
}
sourceQueryFullEntry

(Optional). Specifies whether the defined source query returns full object data (true) or IDs only (false). Defaults to true.

If you do not set this parameter, IDM attempts to detect whether the full object is returned, based on the query results.

target

The path to the resource within the target repository. By default, this will be the same as the source path.

validSource

You can specify a script to validate the source object prior to migration. By default, this property is empty.

endpoint

By default, the migration service endpoint is migration. You can use the endpoint property to change this if needed.

Because the data migration service performs a reconciliation between your old installation and your new installation, the general reconciliation optimizations also apply to the data migration service. For more information about reconciliation optimization, refer to Tuning reconciliation performance.

Run the Data Migration

Before you run your migration, make sure that you have done the following:

  • Paused any scheduled jobs on the source deployment.

  • Configured your conf/migration.json and update/mapLegacyObject.groovy files on the new IDM installation.

  • Moved your configuration files from the old deployment to the new one.

  • If you use workflow, you must manually dump the workflow database tables, and then import them before you start the new instance of IDM for the first time. The workflow database tables start with the prefix ACT_. For information on how to dump/import individual tables, refer to the documentation for your database.

When you launch the new IDM installation, a new migration endpoint should be available. This endpoint supports the following actions:

  • migrate: Triggers a migration of all legacy objects from the remote system. Optionally takes a mapping parameter in order to specify a specific mapping to migrate. For example:

    curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/migration?_action=migrate&mapping=repoManagedUser_repoManagedUser"

  • status: Returns the last status for all reconciliations triggered by the migration service.

  • mappingConfigurations: Returns the full list of migration mapping configurations.

  • mappingNames: Returns the list of migration mapping names.

The period of time a migration takes will depend on the amount of information being migrated. Migrated data will retain the same object IDs they had in the previous deployment.

If requests sent to the source server include an X-Requested-With header, the value of the header will be set to RemoteIDMProxy.

Upgrade a clustered deployment

Follow these general steps when you are updating servers in a cluster:

  1. Redirect client traffic to a different IDM system or cluster.

  2. Shut down every node in the cluster.

  3. Update one node in the cluster.

  4. Clone the first node to the other nodes in that cluster.

Update to a maintenance release

The Maintenance releases incorporate a collection of fixes and minor RFEs. IDM 7.3.1 is the latest maintenance release for IDM 7.3. To upgrade an existing IDM 7.3.x deployment, follow these steps:

  1. Download and extract the IDM 7.3.1 binary from the ForgeRock BackStage download site.

  2. Copy any customized configuration files, scripts, or workflow definitions from your existing deployment to the comparable directory in your 7.3.1 deployment.

  3. If you are still running an earlier version of IDM 7.3.x, copy the conf/authentication.json file from your existing deployment to the conf/ directory in your 7.3.1 deployment.

  4. Copy the keystore and truststore from your existing deployment to the 7.3.1 deployment. For example:

    cp -r /path/to/openidm73x/security /path/to/openidm731

  5. Configure the IDM 7.3.1 server to point to your existing repository:

    • If you are using an external DS repository, verify the accuracy of the conf/repo.ds.json file in your new deployment.

    • If you are using a JDBC repository, verify the accuracy of the following files in your new deployment:

      • conf/repo.jdbc.json

      • conf/datasource.jdbc-default.json

      • resolve/boot.properties (particularly the values for openidm.repo.host and openidm.repo.port)

  6. If you are using workflow, you must run the Flowable upgrade scripts for your database type. These upgrade scripts are incremental and must be run in order, starting with the correct script based on your current Flowable version.

    1. To determine your current Flowable version, check the /path/to/openidm/bundle/flowable-engine-versionNumber.jar file in your old IDM installation.

    2. Run the upgrade scripts from /path/to/openidm/db/database-type/scripts/updates/ in order, starting with your current flowable version:

      1. flowable.database-type.upgradestep.6.6.0.to.6.7.0.all.sql

      2. flowable.database-type.upgradestep.6.7.0.to.6.7.1.all.sql

      3. flowable.database-type.upgradestep.6.7.1.to.6.7.2.all.sql

      4. flowable.database-type.upgradestep.6.7.2.to.6.8.0.all.sql

  7. Shut down your existing IDM 7.3.x server.

  8. Start your IDM 7.3.1 server.