--- title: Property value substitution description: Property value substitution lets you achieve the following: component: pingidm version: 8.1 page_id: pingidm:setup-guide:using-property-substitution canonical_url: https://docs.pingidentity.com/pingidm/8.1/setup-guide/using-property-substitution.html keywords: ["ESV", "Properties"] section_ids: expression-evaluation: Expression resolvers environment_variables: Environment variables java_system_properties: Java system properties expression_files: Expression files framework_configuration_properties: Framework configuration properties configuration_files: Configuration files order-of-precedence: Evaluation order of precedence value-coercion: Transforming data types substitution-repository: Configuration property value storage substitution-limitations: Limitations of property value substitution substitution-limitations-ui: Admin UI limitations substitution-limitations-connectors: Connector configuration limitations --- # Property value substitution Property value substitution lets you achieve the following: * Define a configuration that is specific to a single instance. For example, setting the location of the keystore on a particular host. * Define a configuration whose parameters vary between different environments. For example, the URLs for test, development, and production environments. * Disable certain capabilities on specific nodes. For example, you might want to disable the workflow engine on specific instances. Property value substitution uses *configuration expressions* to introduce variables into the server configuration. You set configuration expressions as the values of configuration properties. The effective property values can be evaluated in a number of ways. For more information about property evaluation, refer to [Expression Resolvers](#expression-evaluation). Configuration expressions have the following characteristics: * To distinguish them from static values, configuration expressions are preceded by an ampersand and enclosed in braces. For example: `&{openidm.port.http}`. The configuration token in the example is `openidm.port.http`. The `.` serves as the separator character. * You can use a default value in a configuration expression by including it after a vertical bar following the token. For example, the following expression sets the default HTTP port value to 8080: `&{openidm.port.http|8080}`. With this configuration, the server attempts to substitute `openidm.port.http` with a defined configuration token. If no token definition is found, the server uses the default value, `8080`. * A configuration property can include a mix of static values and expressions. For example, suppose `hostname` is set to `ds`. Then, `&{hostname}.example.com` evaluates to `ds.example.com`. * Configuration token evaluation is recursive. For example, suppose `port` is set to `&{port.prefix}389`, and `port.prefix` is set to `2`. Then `&{port}` evaluates to `2389`. You can define *nested* properties (that is a property definition within another property definition) and you can combine system properties, boot properties, and environment variables. | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Property substitution is *not* available for any configuration not processed by the IDM backend, such as `ui-themeconfig` or any user-supplied configuration. | ## Expression resolvers At server startup, expression resolvers evaluate property values to determine the effective configuration. You must define expression values before you start the IDM server that uses them. When configuration tokens are resolved, the result is always a string. However, you can *coerce* the output type of the evaluated token to match the type that is required by the property. Ultimately, the expression must return the appropriate data type for the configuration property. For example, the `port` property takes an integer. If you set it using an expression, the result of the evaluated expression must be an integer. If the type is wrong, the server fails to start due to a syntax error. For more information about data type coercion, refer to [Transforming Data Types](#value-coercion). Expression resolvers can obtain values from the following sources: ### Environment variables You set an environment variable to hold the property value. For example: `export OPENIDM_PORT_HTTP=8080`. The environment variable name must be composed of uppercase characters and underscores. The name maps to the expression token as follows: * Uppercase characters are converted to lowercase. * Underscores (`_`) are replaced with `.` characters. In other words, the value of `OPENIDM_PORT_HTTP` replaces `&{openidm.port.http}` in the server configuration. ### Java system properties You set a Java system property to hold the value. Java system property names must match expression tokens exactly. In other words, the value of the `openidm.repo.port` system property replaces `&{openidm.repo.port}` in the server configuration. Java system properties can be set in a number of ways. One way of setting system properties for IDM servers is to pass them through the `OPENIDM_OPTS` environment variable. For example: `export OPENIDM_OPTS="-Dopenidm.repo.port=3306"` System properties can also be declared in your project's `conf/system.properties`. This example uses property value substitution with a standard system property. The example modifies the audit configuration, changing the `audit.json` file to redirect JSON audit logs to the user's home directory. The `user.home` property is a default Java System property: ```json "eventHandlers" : [ { "class" : "org.forgerock.audit.handlers.json.JsonAuditEventHandler", "config" : { "name" : "json", "logDirectory" : "&{user.home}/audit", ... } }, ... ] ``` ### Expression files You set a key in a `.json` or `.properties` file to hold the value. To use an expression file, set the `IDM_ENVCONFIG_DIRS` environment variable, or the `idm.envconfig.dirs` Java system property as described below. By default, IDM sets `idm.envconfig.dirs` to `&{idm.install.dir}/resolver/`. The default property resolver file in IDM is `resolver/boot.properties` but you can specify additional files that might hold property values. Keys in `.properties` files must match expression tokens exactly. In other words, the value of the `openidm.repo.port` key replaces `&{openidm.repo.port}` in the server configuration. The following example expression properties file sets the repository port: ```properties openidm.repo.port=1389 ``` JSON expression files can contain nested objects. JSON field names map to expression tokens as follows: * The JSON path name matches the expression token. * The `.` character serves as the JSON path separator character. The following example JSON expression file uses property value substitution to set the host in the LDAP connector configuration: ```json { "openidm" : { "provisioner" : { "ldap" : { "host" : "ds.example.com" } } } } } ``` To substitute this value in the configuration, the LDAP provisioner file would include the following: ```json { ... "configurationProperties" : { "host" : &{openidm.provisioner.ldap.host|localhost}, ... } } ``` If the server does not find a configuration token for the host name, it substitutes the default (`localhost`). To use expression files, set the environment variable, `IDM_ENVCONFIG_DIRS`, or the Java system property, `idm.envconfig.dirs`, to a comma-separated list of the directories containing the expression files. When reading these files, the server browses the directories in the order specified. It reads all the files with `.json` and `.properties` extensions, and attempts to use them to evaluate expression tokens. For example, if you define `idm.envconfig.dirs=/directory1,/directory2` and a configuration token is defined in both `directory1` and `directory2`, the resolved value will be the value defined in `directory1`. If the configuration token is defined only in `directory2`, the resolved value will be the value defined in `directory2`. | | | | - | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Using expression files are subject to the following constraints:- Although IDM scans the directories in a specified order, within a directory IDM scans the files in a nondeterministic order. - IDM does not scan subdirectories. - Do *not* define the same configuration token more than once in a file. If you define the same property twice in the same file, one definition will be used and the other will be ignored. The server will not throw an error, but because files are scanned in a nondeterministic order, you have no way of knowing which value will be used. - You cannot define the same configuration token in more than one file in a single directory. The server generates an error in this case. This constraint implies that you cannot have backup .properties and .json files, in a single directory if they define the same tokens. - If the same token occurs once in several files that are located in different directories, IDM uses the first value that is read. | ### Framework configuration properties You can use the `conf/config.properties` file to override values used by the OSGI framework. ### Configuration files All the properties declared in the `.json` files in your project's `conf/` directory. ## Evaluation order of precedence The following list displays the order of precedence, from greatest to least: 1. Environment variables override system properties, default token settings, and settings in expression files. 2. System properties override default token settings, and any settings in expression files. 3. Default token settings. 4. If `IDM_ENVCONFIG_DIRS` or `idm.envconfig.dirs` is set, the server uses the settings found in expression files. 5. Framework configuration properties. 6. Hardcoded property values. 7. Properties passed to the startup script [with options such as: -P, -w, and -s.](../install-guide/startup-configuration.html) ## Transforming data types When configuration tokens are resolved, the result is always a string. However, you can transform or *coerce* the output type of the evaluated token to match the type that is required by the property. You transform a property's data type by setting `$type` before the property value. The following coercion types are supported: * array (`$array`) * boolean (`$bool`) * decodeBase64 (`$base64:decode`) Transforms a base64-encoded string into a decoded string. * encodeBase64 (`$base64:encode`) Transforms a string into a base64-encoded string. * integer (`$int`) * list (`$list`) * number (`$number`) This type can coerce integers, doubles, longs, and floats. * object (`$object`) This type can coerce a JSON object such as an encrypted password. > **Collapse: Type Coercion to Integer** > > This example JSON expression file sets the value of the port in the LDAP connector configuration: > > ```json > { > "openidm" : { > "provisioner" : { > "ldap" : { > "port" : 6389 > } > } > } > } > ``` > > When the expression is evaluated, the port is evaluated as a `string` value, (which would cause an error). To coerce the port value to an integer, substitute the value in the LDAP provisioner file as follows: > > ```json > { > ... > "configurationProperties" : { > "port" : { > "$int" : "&{openidm.provisioner.ldap.port|1389}", > }, > ... > } > } > ``` > > With this configuration, the server evaluates the LDAP port property to the integer `6389`. If the server does not find a configuration token for the port, it substitutes the default (`1389`). > **Collapse: Type Coercion to Array** > > This example JSON expression file sets a value for the failover servers in an LDAP connector configuration: > > ```json > "openidm" : { > "provisioner" : { > "ldap" : { > "failover" : ["ldap://host1.domain.com:1389", "ldap://host2.domain.com:1389"] > } > } > } > ``` > > When the expression is evaluated, the URLs would be evaluated as a single `string`. To coerce the value to an array, substitute the value in the LDAP provisioner file as follows: > > ```json > { > ... > "configurationProperties" : { > "failover" : { > "$array":"&{openidm.provisioner.ldap.failover}" > }, > ... > } > } > ``` > > | | | > | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | > | | If you set the failover URLs in a `.properties` file, instead of in a `.json` file, you *must* escape the JSON object. This example sets the failover servers array in the `boot.properties` file:```properties > openidm.provisioner.ldap.failover=[\"ldap://host1.domain.com:1389\",\"ldap://host2.domain.com:1389\"] > ``` | > **Collapse: Type Coercion From List to Array** > > The `$list` function is similar to `$array`, but lets you specify values in a `.properties` file as a list of strings, separated by a comma (`,`). > > For example, you could list the LDAP failover servers in `boot.properties` as follows: > > ```properties > openidm.provisioner.ldap.failover=ldap://host1.domain.com:1389,ldap://host2.domain.com:1389 > ``` > > To coerce the value to an array, your property definition in the LDAP provisioner file would be: > > ```json > { > ... > "configurationProperties" : { > "failover" : { > "$list":"&{openidm.provisioner.ldap.failover}", > }, > ... > } > } > ``` > > This configuration would be converted to: > > ```json > "openidm" : { > "provisioner" : { > "ldap" : { > "failover" : ["ldap://host1.domain.com:1389", "ldap://host2.domain.com:1389"] > } > } > } > ``` ## Configuration property value storage The values of configuration properties that are set explicitly (in `conf/*.json` files) are stored in the repository. You can manage these configuration objects over REST or by using the JSON files themselves. Properties that use value substitution are stored in the repository as *variables*. You store the *value* of each variable in `.properties` files. You can use different `.properties` files to change the configuration for multiple nodes in a cluster. The following table shows how specific configuration properties can be set: **Configuration Property Variables** | Variable | Description | Environment Variables | System Variables | boot.properties | | -------------------- | ------------------------------------------------------------- | --------------------- | ---------------- | --------------- | | `idm.install.dir` | Directory of files from unpacked IDM binary | YES | YES | YES | | `idm.data.dir` | Working location directory | YES | YES | YES | | `idm.instance.dir` | Project directory with IDM configuration files | YES | YES | YES | | `idm.envconfig.dirs` | Directory with environment files, including `boot.properties` | YES | YES | | You can access configuration properties in scripts using `identityServer.getProperty()`. For more information, refer to [The identityServer Variable](../scripting-guide/script-variables-identity-server.html). ## Limitations of property value substitution Property value substitution is limited in the following areas: ### Admin UI limitations | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | | Fields configured to use property substitution are read-only and appear greyed out in the Admin UI. To make changes, you must use the API or edit the file directly.![greyed out field](_images/subs-field-UI.png) | Support for property value substitution in the admin UI is limited to the following categories: * String substitution, where `&{some.property|DefaultValue}` * Number and integer substitution, including: * `"$number" : "&{openidm.port|1234}"` * `"$int" : "&{openidm.port|5678}"` * Base64 substitution, such as: `"$base64:decode" : "&{some.property|YWRtaW4=}"` * Cryptographic substitution, where for passwords and client secrets, IDM substitutes `"********"` for `$crypto` ### Connector configuration limitations You cannot use property substitution for connector reference (`connectorRef`) properties. For example, the following configuration is *not* valid: ```json "connectorRef" : { "connectorName" : "&{connectorName}", "bundleName" : "org.forgerock.openicf.connectors.ldap-connector", "bundleVersion" : "&{LDAP.BundleVersion}" ... } ``` The `connectorName` must be the precise string from the connector configuration. To specify multiple connector version numbers, use a range of versions. For example: ```json "connectorRef" : { "connectorName" : "org.identityconnectors.ldap.LdapConnector", "bundleName" : "org.forgerock.openicf.connectors.ldap-connector", "bundleVersion" : "[1.5.0.0,2.0.0.0)", ... } ```