Migrate to a file-based configuration

AM stores different types of information. For the purposes of this topic, we distinguish between the following types:

Configuration: Relatively static information that doesn’t change frequently after initial setup. Only administrative users can change it.
Data: Information that changes at runtime, often due to end user action. Examples of data are identities, CTS tokens, policies, agents, and applications.

Currently, AM stores configuration and data in LDAP directories.

In a future release, AM will read its configuration only from JSON files, not directory servers. Using LDAP data stores for configuration will be deprecated and file-based configuration (FBC) will be the only supported configuration storage mechanism. Dynamic data will continue to be stored in LDAP directories.

To prepare to migrate your configuration from LDAP directories to JSON files, AM 7.4 provides a technology preview of a configuration migration utility based on the existing amupgrade command. The purpose of this technology preview is to let you test migrating custom configuration to FBC.

When you test the migration utility, focus on schema that you have customized or modified, to identify combinations of attributes that the utility doesn’t yet consider. In such cases, the migration utility shouldn’t fail, but will produce output highlighting any errors it encountered.

Although this feature isn’t yet supported in production, we’d appreciate you reporting these errors by submitting feedback on this topic. Click the feedback icons at the top of this page.

Any reports you submit will be taken into account to ensure you can migrate seamlessly when FBC becomes the mandatory configuration storage mechanism.

The interface stability for the file-based configuration (FBC) migration utility is Technology Preview. Technology previews offer access to new technology not yet supported. Technology preview features may be functionally incomplete and subject to change without notice. For details, refer to Interface stability.

The purpose of this technology preview is to allow you to test the migration of your configuration data. The technology preview should function correctly but may highlight areas that need improvement before the supported release of this feature.

AM configuration stored in PingDS remains supported as documented in this release. In a future AM release, LDAP configuration stores will be deprecated in favor of FBC.

Before you start

Before you migrate your configuration to FBC, read the following:

Supported migration paths
Identify where your configuration is stored
Export existing configuration data
Structure the output

Do not run the migration utility in a production environment. Use a development or test environment.
The migration utility assumes an external LDAP configuration store. The embedded DS datastore isn’t supported in production deployments. From AM 8.0.0 onwards, the embedded DS datastore will be removed entirely.

Supported migration paths

You can use this migration utility only to migrate configuration data from AM versions 7.1.x, 7.2.x, and 7.3.x. You can only use the generated file-based configuration with AM version 7.4.0.

To migrate configuration data from earlier versions, first upgrade the configuration to AM version 7.1, 7.2, or 7.3, and then run the migration utility to get the FBC.

Identify where your configuration is stored

The migration utility creates files that replace the configuration stored in LDAP directories. Most deployments have one or more external LDAP directories to store configuration and data (identities, CTS tokens, policies, agents, and applications).

An LDAP directory that stores configuration and data will still contain the configuration after migration, but AM won’t use that configuration. The migration utility output helps you clean up the datastore to remove redundant configuration after migration.

Identify the datastore that contains your configuration. If you use the default options to install DS for AM configuration, the configuration is stored under ou=services,ou=am-config. Everything else under ou=am-config is regarded as runtime data rather than configuration.

The migration utility doesn’t migrate this data to FBC. It does, however, write the data out to a separate LDIF file, nonmigrated-entries.ldif. You can import this data from the LDIF file to a standalone LDAP server.

Export existing configuration data

The migration tool consumes an LDIF export of the existing configuration.

If the configuration store is PingDS, you can use the export-ldif command to export the data, supplying the following parameters:

The backend ID of the configuration store, cfgStore by default.
The base DN of the branch containing the configuration, ou=am-config by default.
An LDIF filename in which to store the exported configuration.

The following example, run on the DS server, assumes the default parameters when DS is installed for AM configuration:

$ /path/to/opendj/bin/export-ldif \
--backendId cfgStore \
--includeBranch ou=am-config \
--ldifFile /path/to/export-config/exported-configuration.ldif \
--offline
…
 category=BACKEND severity=INFO seq=36 msg=Exported 1480 entries and skipped 0 in 0 seconds (average rate 5522.4/sec)

The exported file contains the existing configuration under ou=am-config:

$ more /path/to/export-config/exported-configuration.ldif
dn: ou=am-config
objectClass: top
objectClass: untypedObject
ou: am-config
…

Structure the output

By default, the configuration is exported to JSON files as a flat list of properties. For some services, the AM admin UI structures the configuration in logical groups. You can mimic this structure in the migrated JSON files by pointing the migration utility to a target AM .war file, resulting in FBC that’s easier to read.

Before you run the migration utility, ready the target AM .war file so that you can generate this structured output.

Migrate your configuration

The migration utility is an enhanced version of the AM configuration upgrader available in AM-7.5.1.zip.

If you don’t already have it, download the AM ZIP file from the BackStage download site.
Extract the contents of the ZIP file.
In the extracted openam directory, locate and extract the Config-Upgrader-7.5.zip file.

To migrate the LDAP configuration, run the amupgrade command with the migrationMode argument. Refer to the Migration command-line arguments for details of each argument:

$ /path/to/amupgrade \
--migrationMode \
--inputConfig input-directory \
--output output-directory \
--sectionsSource target-war-file \
rules-file-path

Migration command-line arguments

Because migration mode isn’t yet supported in production deployments, some of these arguments aren’t exposed in the --help command.

-i | --inputConfig: The directory containing the LDIF configuration files that you’re migrating.
-o | --output directory: The directory to which the migration utility writes the generated FBC.
-M | --migrationMode: Runs the upgrade command in migration mode (required if you’re testing migration to FBC).

Default: false.

-s | --sectionsSource (optional)

A file that specifies the structure (section details) of the output configuration.

Without this option, the migration utility generates the configuration properties as flat files.

This file can be a section.properties file for a specific service or an AM.war file that includes the structure for all services.

Example section.properties file for JSON audit handler

########################################################################################################################
# Common handler section properties
########################################################################################################################
commonHandler=enabled
commonHandler=topics

########################################################################################################################
# Common handler plugin section properties
########################################################################################################################
commonHandlerPlugin=handlerFactory

########################################################################################################################
# JSON handler section properties
########################################################################################################################
jsonConfig=location
jsonConfig=elasticsearchCompatible
jsonConfig=rotationRetentionCheckInterval
jsonFileRotation=rotationEnabled
jsonFileRotation=rotationMaxFileSize
jsonFileRotation=rotationFilePrefix
jsonFileRotation=rotationFileSuffix
jsonFileRotation=rotationInterval
jsonFileRotation=rotationTimes
jsonFileRetention=retentionMaxNumberOfHistoryFiles
jsonFileRetention=retentionMaxDiskSpaceToUse
jsonFileRetention=retentionMinFreeSpaceRequired
jsonBuffering=bufferingMaxSize
jsonBuffering=bufferingWriteInterval

rules-file-path

The path to the upgrade rules file.

Although you’re not using this command to upgrade to a new version, the amupgrade command requires an upgrade rules file.

Use the noop.groovy file (located in (/path/to/amupgrade/rules/fbc/noop.groovy) to migrate the configuration without applying any rules.

Example

$ /path/to/amupgrade \
--migrationMode \
--inputConfig /path/to/export-config \
--output /path/to/am/config/services \
--sectionsSource /path/to/AM-7.5.0.war \
/path/to/amupgrade/rules/fbc/noop.groovy

In the preceding example, the new JSON configuration files are generated in /path/to/am/config/services. By default, AM reads FBC from the $AM_HOME/config/services directory, so use this as the output directory.

After the migration, the output directory contains JSON representations of the global and realm configuration services and a nonmigrated-entries.ldif file with data that wasn’t migrated to JSON.

For example, the configuration of the alpha realm is as follows:

$ more /path/to/am/config/services/global/realms/root-alpha.json
{
  "metadata": {
    "realm": "/alpha",
    "entityType": "",
    "entityId": "L2FscGhh",
    "uid": "o=alpha,ou=services,ou=am-config",
    "sunServiceID": null,
    "objectClass": [
      "sunRealmService",
      "top"
    ],
    "pathParams": {},
    "ou": []
  },
  "data": {
    "_id": "root-alpha",
    "_type": {
      "_id": "",
      "name": "",
      "collection": false
    },
    "active": true,
    "aliases": [
      "alpha"
    ],
    "parentPath": "/",
    "name": "alpha"
  }
}

Configure AM for FBC

When you’ve migrated your configuration to JSON files, you can point your AM instances to the FBC.

If your policy and application data is still in the original LDAP data store, you’ll need to continue using that datastore for the runtime data. Eventually, you should move the runtime data to its own datastore, and retire the original directory server, or remove the redundant configuration from the original datastore.

These instructions assume a local AM server running in an Apache Tomcat container. Adjust the instructions to suit your environment.

Configure AM for file-based mode:
1. Stop AM or the container in which it runs.
2. Set the following Java variables to true:
  
  com.sun.identity.sm.sms_object_filebased_enabled=true com.sun.identity.sm.filebased_embedded_enabled=true
3. Pass those variables to AM in some way. For example, for Tomcat you can set them in the CATALINA_OPTS options:
  
  export CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.identity.sm.sms_object_filebased_enabled=true -Dcom.sun.identity.sm.filebased_embedded_enabled=true"
Provide the configuration files to AM.

AM reads FBC from its home directory. You’ll need access to this directory on your local host or in your Docker container.

For local hosts, change to the $AM_HOME/config/services directory:
```
$ cd $AM_HOME/config/services
```
Copy the migrated configuration into the services directory.
```
$ cp /path/to/new-config $AM_HOME/config/services
```
If the directory contains default configuration, delete it before copying in your migrated configuration.
Restart AM or the container in which it runs.
Start up and test your environment.

Check that AM has the configuration you expect. You can also make changes to your configuration through the AM admin UI and check that those changes are made in the JSON configuration files.

Report any problems you encounter during the migration process, including significant performance issues.

If you use an external policy and application store, you must restart AM with the com.sun.identity.sm.filebased_embedded_enabled Java variable turned off after you’ve set up the policy and application store.

Stop AM or the container in which it runs.
Set com.sun.identity.sm.filebased_embedded_enabled to false or simply remove it from the variables passed to AM. For example:

export CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.identity.sm.sms_object_filebased_enabled=true"

PingAM 7.5.1