IDM 7.3.1

Correlate source objects with existing target objects

When a synchronization operation creates an object on a target system, it also creates a link between the source and target object. IDM then uses that link to determine the object’s synchronization situation during later synchronization operations. For a list of synchronization situations, refer to How IDM assesses synchronization situations.

Every synchronization operation can correlate existing source and target objects. Correlation matches source and target objects, based on the results of a query or script, and creates links between matched objects.

Correlation queries and correlation scripts are configured as part of the mapping. Each query or script is specific to the mapping for which it is configured.

Configure correlation using the admin UI

  1. From the navigation bar, click Configure > Mappings.

  2. From the Mappings page, click the mapping to correlate.

  3. From the Mapping Detail page, click the Association tab.

  4. Expand the Association Rules node, click the drop-down menu, and select one of the following:

    • Correlation Queries

    • Correlation Script

  5. Build and/or write your script or query, and click Save.

Correlation queries

IDM processes a correlation query by constructing a query map. The content of the query is generated dynamically, using values from the source object. For each source object, a new query is sent to the target system, using (possibly transformed) values from the source object for its execution.

Queries are run against target resources, either managed or system objects, depending on the mapping. Correlation queries on system objects access the connector, which executes the query on the external resource.

You express a correlation query using a query filter (_queryFilter). For more information about query filters, refer to Define and call data queries. The synchronization process executes the correlation query to search through the target system for objects that match the current source object.

To configure a correlation query, define a script whose source returns a query that uses the _queryFilter, for example:

{ "_queryFilter" : "uid eq \"" + source.userName + "\"" }

Use filtered queries to correlate objects

For filtered queries, the script that is defined or referenced in the correlationQuery property must return an object with the following elements:

  • The element that is being compared on the target object; for example, uid.

    The element on the target object is not necessarily a single attribute. Your query filter can be simple or complex; valid query filters range from a single operator to an entire boolean expression tree.

    If the target object is a system object, this attribute must be referred to by its IDM name rather than its ICF nativeName. For example, with the following provisioner configuration, the attribute to use in the correlation query would be uid and not +NAME`:

    ...
    "uid" : {
        "type" : "string",
        "nativeName" : "__NAME__",
        "required" : true,
        "nativeType" : "string"
    }
...

  • The value to search for in the query.

    This value is generally based on one or more values from the source object. However, it does not have to match the value of a single source object property. You can define how your script uses the values from the source object to find a matching record in the target system.

    You might use a transformation of a source object property, such as toUpperCase(). You can concatenate that output with other strings or properties. You can also use this value to call an external REST endpoint, and redirect the response to the final "value" portion of the query.

The following correlation query matches source and target objects if the value of the uid attribute on the target is the same as the userName attribute on the source:

"correlationQuery" : {
    "type" : "text/javascript",
    "source" : "var qry = {'_queryFilter': 'uid eq \"' + source.userName + '\"'}; qry"
},

The query can return zero or more objects. The situation assigned to the source object depends on the number of target objects that are returned, and on the presence of any link qualifiers in the query. For information about synchronization situations, refer to How Synchronization Situations Are Assessed. For information about link qualifiers, refer to Map a Single Source Object to Multiple Target Objects.

Create Correlation Queries Using the Expression Builder

The Expression Builder is a wizard that lets you quickly build expressions using drop-down menu options.

  1. From the navigation bar, click Configure > Mappings.

  2. On the Mappings page, click the mapping to correlate.

  3. From the Mapping Detail page, click the Association tab.

  4. Expand the Association Rules node, click the drop-down menu, and select Correlation Queries.

  5. Click Add Correlation Query.

  6. In the Correlation Query window, click the Link Qualifier drop-down menu, and select a link qualifier.

    If you do not need to correlate multiple potential target objects per source object, select the default link qualifier. For more information about linking to multiple target objects, refer to Map a Single Source Object to Multiple Target Objects.

  7. Select Expression Builder.

  8. To create an expression, use the drop-down menus to add and remove items, as necessary. List the fields to use for matching existing items in your source to items in your target.

    The following example displays an Expression Builder correlation query for a mapping from managed/user to system/ldap/accounts objects. The query creates a match between the source (managed) object and the target (LDAP) object if the value of the givenName or the telephoneNumber of those objects is the same.

    expression-builder

  9. After you finish building the expression, click Submit.

  10. On the Mapping Detail page, under the Association Rules node, click Save.

The correlation query displays as follows in the mapping:

"correlationQuery" : [
    {
        "linkQualifier" : "default",
        "expressionTree" : {
            "any" : [
                "givenName",
                "telephoneNumber"
            ]
        },
        "mapping" : "managedUser_systemLdapAccounts",
        "type" : "text/javascript",
        "file" : "ui/correlateTreeToQueryFilter.js"
    }
]

Correlation scripts

In general, a correlation query should meet the requirements of most deployments. However, if you need a more powerful correlation mechanism than a simple query can provide, you can write a correlation script with additional logic. Correlation scripts can be useful if your query needs extra processing, such as fuzzy-logic matching or out-of-band verification with a third-party service over REST. Correlation scripts are generally more complex than correlation queries, and impose no restrictions on the methods used to find matching objects.

A correlation script must execute a query and return the result of that query. The result of a correlation script is a list of maps, each of which contains a candidate _id value. If no match is found, the script returns a zero-length list. If exactly one match is found, the script returns a single-element list. If there are multiple ambiguous matches, the script returns a list with multiple elements. There is no assumption that the matching target record or records can be found by a simple query on the target system. All of the work required to find matching records is left to the script.

To invoke a correlation script, use one of the following properties:

correlationQuery

Returns a Map whose values specify the QueryFilter for the sync engine to execute.

correlationScript

Returns a List<Map> whose value is a list of correlated objects from the target.

You can invoke a correlation script inline:

"correlationScript" : {
    "type": "text/javascript",
    "source": " var resultData = openidm.query("system/ldap/account", myQuery); return resultData.result;"
}

You can also invoke a correlation script using a script file:

"correlationScript" : {
    "type": "text/javascript",
    "file": "myCustomCorrelationScript.js"
}

The following example shows a correlation script that uses link qualifiers. The script returns resultData.result—a list of maps, each of which has an _id entry. These entries will be the values that are used for correlation.

(function () {
    var query, resultData;
    switch (linkQualifier) {
        case "test":
            logger.info("linkQualifier = test");
	        query = {'_queryFilter': 'uid eq \"' + source.userName + '-test\"'};
            break;
        case "user":
            logger.info("linkQualifier = user");
	        query = {'_queryFilter': 'uid eq \"' + source.userName + '\"'};
            break;
        case "default":
            logger.info("linkQualifier = default");
	        query = {'_queryFilter': 'uid eq \"' + source.userName + '\"'};
            break;
        default:
            logger.info("No linkQualifier provided.");
	        break;
    }
    var resultData = openidm.query("system/ldap/account", query);
    logger.info("found " + resultData.result.length + " results for link qualifier " + linkQualifier)
    for (i=0;i<resultData.result.length;i++) {
        logger.info("found target: " + resultData.result[i]._id);
    }
    return resultData.result;
} ());

Configure a correlation script using the admin UI

  1. From the navigation bar, click Configure > Mappings.

  2. On the Mappings page, select the mapping to correlate.

  3. From the Mapping Detail page, click the Association tab.

  4. Expand the Association Rules node, click the drop-down menu, and select Correlation Script.

  5. From the Type drop-down menu, select JavaScript or Groovy.

  6. Enter the correlation script:

    • To use an inline script, select Inline Script, and type the script source.

    • To use a script file, select File Path, and enter the path to the script.

    To create a correlation script, use the details from the source object to find the matching record in the target system. If you are using link qualifiers to match a single source record to multiple target records, you must also use the value of the linkQualifier variable within your correlation script to find the target ID that applies for that qualifier.

  7. Click Save.