Configuration expressions in exported configuration files

Amster supports the use of configuration expressions as the values of configuration properties in the exported configuration files. Amster substitutes the expressions with values obtained from the Amster shell, expression files, environment variables, and others, when importing the configuration files into an AM instance. Property value substitution enables you to achieve the following:

Define a configuration that is specific to a single instance.

For example, setting the location of the keystore on a particular host.
Define a configuration whose parameters vary between different environments.

For example, the URLs and passwords for test, development, and production environments.
Disable certain capabilities on specific AM instances.

For example, you might want to disable a particular instance from sending notifications to agents.

Property value substitution uses expressions to introduce variables into the server configuration. You set expressions as the values of configuration properties. The effective property values can be evaluated in a number of ways.

Expressions

Expressions share their syntax and underlying implementation with other Ping Identity Platform components. Expressions have the following characteristics:

To distinguish them from static values, configuration expressions are preceded by an ampersand (& ) and enclosed in braces ({} ). Use the dot (. ) character as a separator character for the expression token. For example, &{smtp.port}.
You can use a default value in a configuration expression by including it after a vertical bar (| ) character following the token.

For example, the following SMTP port expression sets the default value of the SMTP port to 1349: &{smtp.port|1349}.
A configuration property can include a mix of static values and expressions.

For example, suppose hostname is set to openam. Then &{hostname}.example.com evaluates to openam.example.com.

You can also use expressions in conjunction with Unix environment variables once these are made available to the Amster shell. For more information, refer to Scripts.
You can define nested properties (that is, a property definition within another property definition).

For example, suppose listen.port is set to &{port.prefix}389, and port.prefix is set to 2. Then &{listen.port} evaluates to 2389.
To use an expression in the key of a key:value pair, use the special AMSTER marker, rather than the dollar sign ($).

For example, to update the fully-qualified domain name mappings, you could use syntax such as the following:

"com.sun.identity.server.fqdnMap[AMSTER{realm.dns.alias}]" : "${realm.dns.alias}"

Amster defines the following expressions by default:

&{amster.import.dir}

This expression is resolved in the following ways:

As the directory containing the configuration files being imported into AM when using the import-config --path directory command.
As the parent directory of the configuration file being imported into AM when using the import-config --path file command.

&{amster.import.url}

This expression is resolved in the same way as &{amster.import.dir}, but in URL format. For example, file://path/to/directory.

Expression evaluation and order of precedence

You must define expression values before importing the configuration into AM. When evaluated, an expression must return the appropriate type for the configuration property. For example, the smtp.port property takes an integer. If you set the property using an expression, the result of the evaluated expression must be an integer.

If the type is wrong, AM may fail to start after a configuration import, with unexpected errors. For more information about data type coercion, refer to Transforming data types.

Amster can obtain expressions from the following sources.

Environment variables

You set an environment variable in your operating system shell. For example, export SMTP_PORT=1342.

The environment variable name must be composed of uppercase characters and underscores. The name maps to the expression token as follows:

Uppercase characters are converted into lower case characters.
Underscores (_ ) are replaced with dot (. ) characters.

In other words, the value of SMTP_PORT replaces &{smtp.port} in the AM configuration files.

Java system properties

You set a Java system property to hold the value when you call the amster command with parameters. For example:

"-Dsmtp.port=3306"

Amster shell variables

You set Amster shell variables to hold values.

To define expressions as Amster shell variables, remove the dot (.) character, and use the standard camel case notation for naming variables in Groovy.

For example, the &{smtp.port} expression would be defined as:

am> smtpPort = "1342"
===> 1342

In the configuration file, however, you still define the expression as &{smtp.port}.

For more information about how to convert Java system properties and environment variables into Amster shell variables, see Variables in Amster scripts.

Expression files

You set a key in a .json or .properties file to hold the value.

Keys in .properties files must match expression tokens exactly. In other words, the value of the smtp.port key replaces &{smtp.port} in the server configuration.

The following is an example properties expression file:

smtp.port=1342
smtp.user=Greg
stateless.tokens.enabled=true

JSON expression files can contain nested objects.

JSON field names map to expression tokens as follows:

The JSON path name matches the expression token.
The . character serves as the JSON path separator character.

The following is an example of a JSON expression file:

{
   "smtp" : {
      "port" : 1342,
      "user" : "Greg"
   },
   "stateless" : {
      "tokens" : {
         "enabled" : "true"
      }
   },
   "blacklist" : {
      "java" : {
         "classes" : [
            "java.lang.Class",
            "java.security.AccessController",
            "java.lang.reflect.*"
         ]
      }
   }
}

You can set multiple configuration files to store your properties. For example, you could have a file to store authentication tree values and another file to store policy-related values.

Note the following constraints when using expression files:

Amster scans the files in the provided directory in a non-deterministic order.
Amster reads all files with .json and .properties extensions.
Amster does not have a predictable order of precedence for handling multiple configuration tokens with the same name. You are responsible for ensuring name uniqueness of configuration tokens across multiple expression files.

To provide expression files to Amster, use the envconfig command followed by the full path to a directory or a file. For example:

am> envconfig /path/to/expressionfiles/

Expression evaluation

Expressions are evaluated and replaced with the expected values when importing a configuration into an AM instance. Attempting to import AM JSON configuration files containing expressions that are not defined causes an error message similar to the following:

amster openam.example.com:8443> import-config \
 --path /tmp/myExportedConfigFiles/realms/root/EmailService.json
Importing file /tmp/myExportedConfigFiles/realms/root/EmailService.json
---------------------------------------------------------------------
IMPORT ERRORS
---------------------------------------------------------------------
Failed to import /tmp/myExportedConfigFiles/realms/root/EmailService.json  :
Can’t substitute source vale: unresolved variables ([email.service.port]) or cycles detected ([])

The following list reflects the order of precedence:

Environment variables override default expressions, Amster variables, Java system properties and settings in expression files.
Amster variables override Java system properties, tokens found in expression files, and default expressions.
Java system properties override tokens found in expression files and the default expression tokens.

Transforming data types

By default, when configuration tokens are resolved, the result is always a string. However, when evaluated, an expression must return the appropriate type for the configuration property. For example, the smtp.port property takes an integer. If you set the property using an expression, the result of the evaluated expression must be an integer. If the type is wrong, AM may fail to start after a configuration import, with unexpected errors.

You can transform the output type of the evaluated token to match the type that is required by the property. Amster can coerce expressions to resolve as the following types:

$int. Coerce to an integer.
$number. Coerce to integers, doubles, longs, and floats.
$bool. Coerce to a boolean. For example, true.
$array. Coerce to a JSON array. For example, ["a","b","c"].
$list. Coerce to a JSON list. For example, 1,2,3.
$object. Coerce to a JSON object. For example, {"a","b"}.
$base64-encode and $base64-decode. Encode or decode the string to or from Base64.

To convert the value of a property into a different type in the exported configuration file, specify the new type as follows:

"port" : {
    "$int" : "&{smtp.port}"
}

You can also replace configuration values with the contents of a file by using the $inline coercion function. You can specify the path to the file, or replace it with an expression. For example:

{
 "message" : {
    "$inline" : "&{config.path}/emailcontent.txt"
 },
 "message2" : {
    "$inline" : "/path/to/emailcontent.txt"
 }
}

Consider the following points when using the $inline coercion function:

When the file contains a script, such as an authentication script, you must transform its value to Base-64. For example:
```
"script": {
  "$base64:encode": {
    "$inline": "/path/to/scripted-decision.groovy"
  }
}
```
When the file contains a value of a type that is different from the configuration value type, you must transform its value. For example:
```
"port": {
  "$int": {
      "$inline": "myfile.txt"
  }
}
```

Recognizing the type of a particular configuration property in the JSON files may not always be straightforward. When in doubt, try the following approaches:

Check the property in the AM console.
Configure the property in the AM console, then export the configuration with Amster for an example.

Expression example files

This section contains an example expression file, and some excerpts of exported configuration files with expressions inserted in them.

Expression file

 {
   "env" : {
      "name" : "DEV"
   },
   "oauth" : {
      "authmodule" : {
         "issuer" : "dev-oathissuer.example.com",
         "authlevel" : 2,
         "checksum" : "true"
      },
      "devprof" : {
         "kstore" : {
            "path" : "&{product.install.dir}/&{env.name}/keystore.jceks",
            "type" : "JCEKS",
            "encpas" : "AAAAA0FFUwIQ1WDDMsxGoZMiRHhDQ+ywUfTMdGtYqEsvZZLV9W8ygfHi/5kBWjMpyg=="
         }
      }
   }
}

As well as configuration details, such as hostnames and ports, passwords and secrets are likely to differ between AM instances.

The previous example demonstrates an expression file tailored for the development environment. Note how the &{devprof.kstore.encpas} expression holds the value of the encrypted keystore password for the OAuth device profile keystore configuration.

For security reasons, Amster only exports passwords in configuration files if the transport key exists in AM’s keystore.

If your environments have different passwords, you could manage passwords using expressions as follows:

Configure AM with the desired password values for each of your environments.
Export the configurations using the same transport key.

Using this technique ensures that the passwords for all the environments are properly encrypted. You can safely create expression files by environment with the appropriate values.

Configuration file excerpts

{
   "data":{
      "_id":"",
      "defaults":{
         "oathIssuerName":"&{oath.authmodule.issuer}",
         "totpTimeStepsInWindow":2,
         "authenticationLevel":{
            "$int":"&{oath.authmodule.authlevel}"
         }
      },
      "passwordLength":"6",
      "addChecksumToOtpEnabled":{
         "$bool":"&{oath.authmodule.checksum}"
      }
   },
   "data":{
      "_id":"",
      "defaults":{
         "oathAttrName":"oathDeviceProfiles",
         "authenticatorOATHDeviceSettingsEncryptionKeystorePrivateKeyPassword":null,
         "authenticatorOATHDeviceSettingsEncryptionScheme":"NONE",
         "authenticatorOATHDeviceSettingsEncryptionKeystore":"&{oath.devprof.kstore.path}",
         "authenticatorOATHDeviceSettingsEncryptionKeystoreType":"&{oath.devprof.kstore.type}",
         "authenticatorOATHDeviceSettingsEncryptionKeystorePassword":null,
         "authenticatorPushDeviceSettingsEncryptionKeystorePassword-encrypted":"&{oath.devprof.kstore.encpas}",
         "authenticatorOATHDeviceSettingsEncryptionKeystoreKeyPairAlias":null,
         "authenticatorOATHSkippableName":"oath2faEnabled"
      }
   },
   "data":{
      "_id":"01e1a3c0-038b-4c16-956a-6c9d89328cff",
      "name":"Authentication Tree Decision Node Script &{env.name}",
      "description":"&{product.install.dir}/&{env.name}/authdecisionnode_desc.txt",
      "script":{
         "$base64:encode":{
            "$inline":"&{product.install.dir}/&{env.name}/scripted-decision.groovy"
         }
      }
   }
}

Note how the files used by the $inline coercion function are stored under a directory that is referenced by the &{env.name} expression. For example:

"$inline":"&{product.install.dir}/&{env.name}/scripted-decision.groovy"

This is just an example of how you can separate your configuration files by environment.