Upgrade
This guide shows you how to upgrade an existing deployment to the latest ForgeRock® Identity Management release.
The upgrade process is largely dependent on your deployment and on the extent to which you have customized IDM. Engage ForgeRock Support Services for help in upgrading an existing deployment. Also, read the Release notes before you start an upgrade; specifically, Incompatible changes.
ForgeRock Identity Platform™ serves as the basis for our simple and comprehensive Identity and Access Management solution. We help our customers deepen their relationships with their customers, and improve the productivity and connectivity of their employees and partners. For more information about ForgeRock and about the platform, refer to https://www.forgerock.com.
The ForgeRock Common REST API works across the platform to provide common ways to access web resources and collections of resources.
About upgrades
The automated update process available with previous IDM versions is no longer supported. This chapter describes the manual process required to upgrade an existing IDM deployment. At a high level, the manual update process involves the following steps:
-
Install IDM 7.3.
-
Migrate your existing IDM configuration to the new installation.
-
Update your repository.
-
Test your scripts and customizations work as expected.
-
Migrate existing data to the new installation.
Supported upgrade paths
The following table contains information about the supported upgrade paths to IDM 7.3:
Version | Upgrade Supported to IDM 7.3 |
---|---|
IDM 7.2.x |
YES |
IDM 7.1.x |
YES |
IDM 7.0.x |
YES |
IDM 6.5.x |
YES |
IDM 6.0.x |
YES |
IDM 5.5.x |
YES |
IDM 5.0.x |
YES |
Depending on how you have customized your deployment, there might be incompatible configuration changes when you upgrade from versions prior to IDM 6.5.x. Read the upgrade documentation for each interim release and apply all required script and configuration changes. |
Before you upgrade
Fulfill these requirements before you upgrade IDM, especially before upgrading the software in a production environment. Also refer to the requirements listed in Before you install and the changes listed in Incompatible changes.
Before you start, verify that you have a supported Java version installed:
Vendor | Versions | ||
---|---|---|---|
OpenJDK, including OpenJDK-based distributions:
|
11, 17* |
||
Oracle Java |
11, 17* |
* Version 17.0.3 or higher.
If the server uses an older version that is no longer supported, install a newer Java version before you update, and follow the instructions in Java requirements.
Then, follow these steps:
-
Back up your existing deployment by archiving the
openidm
directory and creating a backup of the repository and all other applicable databases.If you use workflow, you must manually dump the workflow database tables, and then import them before you start the new instance of IDM for the first time. The workflow database tables start with the prefix ACT_
. For information on how to dump/import individual tables, refer to the documentation for your database. -
To save a record of the audit logs from your existing IDM installation, manually copy the log files from the
/path/to/openidm/audit/
directory, before you start the upgrade. -
Download and extract
IDM-7.3.1.zip
from the ForgeRock BackStage download site.
Migrate your configuration
This chapter covers the steps required to migrate your IDM configuration to IDM 7.3.
There is no automated way to migrate a customized configuration to IDM 7.3, so you must migrate customized configuration files manually. Assuming you are upgrading from IDM 7.2.x, there are three ways to do this:
-
Use the new IDM 7.3 configuration files as a base, and copy any customizations you have made to the new files.
This is the preferred option, particularly if you have used version control on your configuration and can determine the exact changes you have applied.
-
Use your existing configuration files as a base, and add any new IDM 7.3 configuration to your existing files.
-
Use your existing configuration "as is" with no IDM 7.3 changes.
In most cases, a customized IDM 7.2.x configuration will work without further modification on IDM 7.3.
Migrate configuration files
For customized files in your project’s conf/
directory, check that the customizations are compatible with the changes outlined in Incompatible changes. If there are no incompatible changes, either copy your old configuration files to your IDM 7.3 installation, or copy any customization into the corresponding new configuration files.
If you create custom configuration files, ForgeRock recommends not using spaces or special characters in the filenames, in accordance with the OSGi specification. |
Migrate boot.properties
On the IDM 7.3 installation, edit the resolver/boot.properties
file to match any customizations that you made on your IDM 7.2.x server. Specifically, check the following elements:
-
The HTTP, HTTPS, and mutual authentication ports.
If you changed the default ports in your IDM 7.2.x deployment, make those same changes in the new
boot.properties
file. -
Check that the keystore and truststore passwords match the current passwords for the keystore and truststore of your existing IDM deployment.
Migrate security settings
Copy the contents of your IDM 7.2.x security/
folder to the IDM 7.3 installation.
If you do not copy your old truststore and keystore files to your new instance, you will be unable to decrypt anything that was encrypted by your old instance of IDM. |
Migrate custom scripts
Migrate any custom scripts or default scripts that you have modified to the script
directory of your IDM 7.3 instance. In general, custom and customized scripts should be located in the openidm/script
directory of your existing IDM deployment.
For custom scripts, review Incompatible changes. If you are confident that the scripts will work as intended on IDM 7.3, copy these scripts to the new instance.
If you modified a default IDM script, compare the default versions of the IDM 7.2.x and IDM 7.3 scripts. If nothing has changed between the default versions, review your customizations against Incompatible changes. If a default script has changed since the IDM 7.2.x release, test that your customizations work with the new default script. If you are confident that your changes will work as intended on the new version, copy the customized scripts to the new script
directory.
If you modify any shell scripts, such as |
Migrate custom bundles
If your existing deployment includes any custom JAR files in the bundles
directory, migrate these to the new deployment. Pay particular attention to any files that support JDBC database drivers.
Migrate provisioner files
Change any customized provisioner configurations in your existing deployment to point to the connectors that are provided with IDM 7.3. Specifically, make sure that the connectorRef
properties reflect the new connector versions, where applicable. For example:
"connectorRef" : {
"bundleName": "org.forgerock.openicf.connectors.ldap-connector",
"bundleVersion": "[1.4.0.0,1.6.0.0)",
"connectorName": "org.identityconnectors.ldap.LdapConnector"
},
Alternatively, copy the connector .jar files from your existing deployment into the openidm/connectors
directory of the new installation.
Migrate UI customizations
If you have customized the admin UI, review any custom UI files from your IDM 7.2.x deployment (generally in the openidm/ui/admin/extension
directory), and compare them against the corresponding IDM 7.3 files.
For each customized file, copy the corresponding default IDM 7.3 UI files to a openidm/ui/admin/extension
directory on the new instance.
Apply your customizations to files in the new openidm/ui/admin/extension
directory.
Update the repository
When you have migrated your configuration to the new IDM installation, you need to handle the data that is stored in your repository. There are two options to update a repository:
-
Create a new IDM 7.3 repository, then migrate your data to the new repository.
When you have upgraded the repository, or created a new repository, start the IDM server and test that all your scripts are working as expected, before migrating your data.
Upgrade an existing repository
Upgrading an existing repository means that you do not need to migrate data. However, you must run a series of scripts that modify the repository, to use the new features in IDM 7.3.
Because the repository upgrade scripts are incremental, you must review each major version upgrade after your current release. For example, when upgrading from 6.5.x to 7.3.x, review the upgrade process and scripts for 7.0.x, 7.1.x, 7.2.x, and 7.3.x (this version). Repository upgrade procedures: |
Prepare an existing repository for IDM 7.3 as follows:
-
Shut down IDM, if it is running.
-
Clear all
configobjects
related tables. For example, in MySQL run:DELETE FROM openidm.configobjects; DELETE FROM openidm.configobjectproperties;
-
From your IDM 7.3 installation, run the schema update scripts for your database type:
-
For Microsoft SQL (MSSQL) only,
/path/to/openidm/db/mssql/scripts/updates/00-update-to-nvarchar.sql
. This script updates the deprecated SQL data typeNTEXT
toNVARCHAR(MAX)
.This script is only required for explicit managed_user
tables.
-
-
If you are using workflow, you must run the Flowable upgrade scripts for your database type. These upgrade scripts are incremental and must be run in order, starting with the correct script based on your current Flowable version.
-
To determine your current Flowable version, check the
/path/to/openidm/bundle/flowable-engine-versionNumber.jar
file in your old IDM installation. -
Run the upgrade scripts from
/path/to/openidm/db/database-type/scripts/updates/
in order, starting with your current flowable version:-
flowable.database-type.upgradestep.6.6.0.to.6.7.0.all.sql
-
flowable.database-type.upgradestep.6.7.0.to.6.7.1.all.sql
-
flowable.database-type.upgradestep.6.7.1.to.6.7.2.all.sql
-
flowable.database-type.upgradestep.6.7.2.to.6.8.0.all.sql
-
-
-
Launch IDM and run the following Groovy script to clear the
reconprogressstate
data in your repository:def result = openidm.query( "repo/reconprogressstate", [ "_queryFilter" : "true", "_fields" : "_id" ]).result; for ( item in result ) { openidm.delete("repo/reconprogressstate/" + item["_id"], null); } return result.size() + " reconprogressstate records deleted";
This script works for all repository types and can be sent as a REST call. For example:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Content-Type: application/json" \ --request POST \ --data '{ "type":"groovy", "source":"def result = openidm.query(\"repo/reconprogressstate\", [ \"_queryFilter\" : \"true\", \"_fields\" : \"_id\" ]).result; for ( item in result ) { openidm.delete(\"repo/reconprogressstate/\" + item[\"_id\"], null); }; return result.size() + \" reconprogressstate records deleted\";" }' \ "http://localhost:8080/openidm/script?_action=eval" "1 reconprogressstate records deleted"
-
Verify that all scripts and functions behave as expected.
Create a new repository
-
Set up a new repository, following the steps in Select a repository. A new repository is already configured for all the new capabilities in IDM, but does require migrating existing data to that repository.
If you create a new repository, you must still update your configuration files to use the new features.
-
After you have set up the new repository, migrate your data to that repository.
Migrate data
The data migration service helps you move information stored in an IDM repository to a new deployment.
You can use this service when you are upgrading to a new version, or when you are migrating to a different
repository type. The migration service is off by default. To enable it, copy migration.json
from
samples/example-configurations/conf/
into your conf/
directory, and set "enabled": true
.
Migration is run from your new installation through IDM’s recon service, using your previous deployment as a data source. The data migration service supports importing information from IDM instances back to version 4. If you are migrating from a version of IDM earlier than that, you will need to follow previous update instructions to get your deployment into a state where it can be migrated using this service.
Because the migration service migrates information that may be encrypted, such as passwords, you must make sure you have copied the |
Default Data Imported by the Migration Service
-
Internal Roles
-
Internal Users
-
Internal User Metadata
-
Managed Roles
-
Managed Users
-
Managed Assignments
-
Links and Relationships
-
Scheduler jobs
If you are migrating scheduler jobs from IDM 4.0 or 4.5, you will need to modify the entry in
migration.json
to be:{ "source" : "scheduler", "target" : "scheduler/job" }
If you have additional object types (for example, managed devices), modify migration.json
to include these objects.
Configure the Migration Service
The data migration service is configured through migration.json
. The default file assumes a default schema; modify the file if you have added custom managed data. The migration.json
file can have the following properties:
- enabled
-
Boolean,
true
orfalse
. Enables the migration service. - connection
-
Configures the connection to the source IDM instance you are migrating from. Available properties:
- instanceUrl
-
The URI for the source IDM instance.
- authType
-
The authentication mechanism to the source IDM instance. Can be
basic
(username/password) orbearer
(authentication using AM bearer tokens). - userName
-
Used for authenticating to the source IDM instance, if the
authType
isbasic
. - password
-
Used for authenticating to the source IDM instance, if the
authType
isbasic
. - clientId
-
Used for authenticating to the source IDM instance, if the
authType
isbearer
. - clientSecret
-
Used for authenticating to the source IDM instance, if the
authType
isbearer
. - tokenEndpoint
-
Used for authenticating to the source IDM instance, if the
authType
isbearer
. - scope (optional)
-
List of OAuth scopes.
- scopeDelimiter (optional)
-
Delimiter for the list of OAuth scopes.
- tlsVersion (optional)
-
Lets you override the default TLS version.
- connectionTimeout (optional)
-
Timeout for connecting to the source IDM instance (defaults to
10s
). - reuseConnections (optional)
-
Lets you override the default setting (defaults to
true
). - retryRequests (optional)
-
Lets you override the default setting (defaults to
true
). - hostnameVerifier (optional)
-
The SSL hostname verification policy. Specifies whether the host name presented by the remote server certificate is verified upon establishing new SSL connections (defaults to
STRICT
). Possible values:-
STRICT
: Requires that the host name match the host name presented in the certificate. Wild-cards only match a single domain. -
ALLOW_ALL
: Accepts any host name (disables host name verification).
-
- maxConnections (optional)
-
Lets you override the default maximum number of connections (default is
64
). - proxy (optional)
-
Lets you specify connection through a proxy server. Includes the following properties:
proxyUri
-
The proxy host and port to which IDM should connect.
userName
-
The user account to connect to the remote proxy.
password
-
The password of the proxy user.
- socketTimeout
-
The TCP socket timeout, when waiting for HTTP responses. If you do not set a duration, the default is no timeout.
Example valid duration values:
-
4 days
-
59 minutes and 1 millisecond
-
1 minute and 10 seconds
-
42 millis
-
unlimited
-
none
-
zero
-
- mappings
-
A list of the endpoints that will be migrated from your old IDM instance to your new instance, expressed as mappings between the old and new instances. The complete list of mapping properties is the same as any regular synchronization mapping. Properties with particular significance for data migration include the following:
- allowEmptySourceSet
-
Specifies whether the migration service should continue if it encounters an empty source mapping. This is enabled by default.
- correlationQuery
-
You can specify a custom correlation query. By default, this is:
"var map = {'_queryFilter': '_id eq \"' + source._id + '\"'}; map;"
For more information about writing correlation queries, refer to Correlate source objects with existing target objects.
- enableLinking
-
Specifies whether links are maintained between source and target objects. If
enableLinking
is set tofalse
, links are not maintained. This is the default behavior for the migration service, where it is expected that you will run the migration only once. If you intend to run the migration more than once, set this parameter totrue
. - onCreate
-
The script used by the migration service for creating the data that is being migrated to the new installation. By default, this points to a Groovy script:
update/mapLegacyObject.groovy
. - onUpdate
-
The script used by the migration service for updating the data that is being migrated in the new installation. By default, this points to a Groovy script:
update/mapLegacyObject.groovy
. - policies
-
An array of policies to apply to the data being migrated.
- properties
-
An array of properties to perform additional actions on, such as modifying the contents of a property during the migration. This follows the pattern you would find in a standard reconciliation. For more information about transforming data during a reconciliation, refer to Transform Attributes in a Mapping.
- reconSourceQueryPageSize
-
Specifies the number of results to return per page, if paging is turned on. By default, 1000 results per page are returned.
- reconSourceQueryPaging
-
Specifies whether the migration service should use paging when querying the source IDM instance. By default, this is set to
false
. Turn paging on if you have a large data set and are concerned about memory usage.For large data sets, you might be able to improve migration performance by turning paging on and increasing the query page size (using
reconSourceQueryPageSize
). The most effective page size will vary, depending on the available resources. - runTargetPhase
-
Specifies whether the migration should run the target phase of reconciliation. By default, this is set to
false
as there is no data in the target repository. - source
-
This is the only property that is required for data migration. The source should be the path to the resource within the repo; for example,
repo/managed/user
.By default, the migration services use the repo
endpoint, rather than themanaged
endpoint for both thesource
and thetarget
. Create, read, update, and delete operations will therefore not trigger an implicit synchronization to the target resource. - sourceQuery
-
The query on the source system, used to find all objects to be migrated. Defaults to
"_queryFilter" : "true&fields=_id"
, which returns the IDs of all source objects.You can improve migration performance by returning the whole source entry (setting the
sourceQuery
to"_queryFilter" : "true"
).If you are migrating from IDM 6.5.xAny explicitly mapped resource coming from
repo/<mappingName>
must include:"sourceQuery": { "_queryFilter": "true", "_fields": "" }
- sourceQueryFullEntry
-
(Optional). Specifies whether the defined source query returns full object data (
true
) or IDs only (false
). Defaults totrue
.If you do not set this parameter, IDM attempts to detect whether the full object is returned, based on the query results.
- target
-
The path to the resource within the target repository. By default, this will be the same as the source path.
- validSource
-
You can specify a script to validate the source object prior to migration. By default, this property is empty.
- endpoint
-
By default, the migration service endpoint is
migration
. You can use theendpoint
property to change this if needed.
Because the data migration service performs a reconciliation between your old installation and your new installation, the general reconciliation optimizations also apply to the data migration service. For more information about reconciliation optimization, refer to Tuning reconciliation performance. |
Run the Data Migration
Before you run your migration, make sure that you have done the following:
-
Paused any scheduled jobs on the source deployment.
-
Configured your
conf/migration.json
andupdate/mapLegacyObject.groovy
files on the new IDM installation. -
Moved your configuration files from the old deployment to the new one.
-
If you use workflow, you must manually dump the workflow database tables, and then import them before you start the new instance of IDM for the first time. The workflow database tables start with the prefix
ACT_
. For information on how to dump/import individual tables, refer to the documentation for your database.
When you launch the new IDM installation, a new migration
endpoint should be available. This endpoint supports the following actions:
-
migrate
: Triggers a migration of all legacy objects from the remote system. Optionally takes amapping
parameter in order to specify a specific mapping to migrate. For example:curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/migration?_action=migrate&mapping=repoManagedUser_repoManagedUser"
-
status
: Returns the last status for all reconciliations triggered by the migration service. -
mappingConfigurations
: Returns the full list of migration mapping configurations. -
mappingNames
: Returns the list of migration mapping names.
The period of time a migration takes will depend on the amount of information being migrated. Migrated data will retain the same object IDs they had in the previous deployment.
If requests sent to the source server include an X-Requested-With header, the value of the header will be set to RemoteIDMProxy .
|
Upgrade a clustered deployment
Follow these general steps when you are updating servers in a cluster:
-
Redirect client traffic to a different IDM system or cluster.
-
Shut down every node in the cluster.
-
Update one node in the cluster.
-
Clone the first node to the other nodes in that cluster.
Update to a maintenance release
The Maintenance releases incorporate a collection of fixes and minor RFEs. IDM 7.3.1 is the latest maintenance release for IDM 7.3. To upgrade an existing IDM 7.3.x deployment, follow these steps:
-
Download and extract the IDM 7.3.1 binary from the ForgeRock BackStage download site.
-
Copy any customized configuration files, scripts, or workflow definitions from your existing deployment to the comparable directory in your 7.3.1 deployment.
-
If you are still running an earlier version of IDM 7.3.x, copy the
conf/authentication.json
file from your existing deployment to theconf/
directory in your 7.3.1 deployment. -
Copy the keystore and truststore from your existing deployment to the 7.3.1 deployment. For example:
cp -r /path/to/openidm73x/security /path/to/openidm731
-
Configure the IDM 7.3.1 server to point to your existing repository:
-
If you are using an external DS repository, verify the accuracy of the
conf/repo.ds.json
file in your new deployment. -
If you are using a JDBC repository, verify the accuracy of the following files in your new deployment:
-
conf/repo.jdbc.json
-
conf/datasource.jdbc-default.json
-
resolve/boot.properties
(particularly the values foropenidm.repo.host
andopenidm.repo.port
)
-
-
-
If you are using workflow, you must run the Flowable upgrade scripts for your database type. These upgrade scripts are incremental and must be run in order, starting with the correct script based on your current Flowable version.
-
To determine your current Flowable version, check the
/path/to/openidm/bundle/flowable-engine-versionNumber.jar
file in your old IDM installation. -
Run the upgrade scripts from
/path/to/openidm/db/database-type/scripts/updates/
in order, starting with your current flowable version:-
flowable.database-type.upgradestep.6.6.0.to.6.7.0.all.sql
-
flowable.database-type.upgradestep.6.7.0.to.6.7.1.all.sql
-
flowable.database-type.upgradestep.6.7.1.to.6.7.2.all.sql
-
flowable.database-type.upgradestep.6.7.2.to.6.8.0.all.sql
-
-
-
Shut down your existing IDM 7.3.x server.
-
Start your IDM 7.3.1 server.