--- title: Manage reconciliation description: You can trigger, cancel, and monitor reconciliation operations over REST by using the openidm/recon REST endpoint or the admin UI. component: pingidm version: 8.1 page_id: pingidm:synchronization-guide:manage-recon canonical_url: https://docs.pingidentity.com/pingidm/8.1/synchronization-guide/manage-recon.html keywords: ["Synchronization", "Reconciliation"] section_ids: triggering-recons: Trigger a reconciliation cancel-recon: Cancel a reconciliation list-recons: List reconciliation history recon-association-storage: Reconciliation association storage recon-run-props: Reconciliation properties recon-details: Reconciliation details recon-details-ui: View reconciliation details using the admin UI recon-assoc: Reconciliation association details recon-delete: Purge reconciliation statistics --- # Manage reconciliation You can trigger, cancel, and monitor reconciliation operations over REST by using the `openidm/recon` REST endpoint or the admin UI. ## Trigger a reconciliation The following example triggers a reconciliation operation over REST based on the `systemLdapAccounts_managedUser` mapping: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser" ``` By default, a reconciliation run ID is returned when the reconciliation operation is initiated. Clients can make subsequent calls to the reconciliation service, using this reconciliation run ID to query its state and call operations on it. Learn more and find an example in [Reconciliation Details](#recon-details). It's also possible to override specific mapping configurations, such as `sourceQuery` or `targetQuery` for a single run by providing a JSON body with the POST request. Learn more in [Reconciliation operations](../rest-api-reference/endpoints/rest-recon.html). The reconciliation run initiated previously would return something similar to the following: ```json { "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144", "state": "ACTIVE" } ``` To have the request wait until the reconciliation operation completes before the reconciliation run ID is returned, set the `waitForCompletion` parameter to `true` when the reconciliation is initiated: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser&waitForCompletion=true" ``` | | | | - | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | The `waitForCompletion=true` parameter is **not** supported in clustered reconciliation environments. Attempting to use it in a cluster will result in an error. In such cases, poll the reconciliation status using the returned run ID. | | | | | - | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | To trigger this reconciliation using the admin UI, select Configure > Mappings, select a mapping, then click Reconcile.If you click Cancel Reconciliation before the reconciliation is complete, you'll need to restart it. | ## Cancel a reconciliation To cancel an in-progress reconciliation, specify the reconciliation run ID. The following REST call cancels the reconciliation run initiated in the previous section: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/recon/0890ad62-4738-4a3f-8b8e-f3c83bbf212e?_action=cancel" ``` The output for a reconciliation cancellation request is similar to the following: ```json { "status":"INITIATED", "action":"cancel", "_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e" } ``` If the reconciliation run is waiting for completion (`waitForCompletion=true`), obtain the reconciliation run ID from the list of active reconciliations, as described in the following section. | | | | - | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | To cancel a reconciliation run in progress using the admin UI, click Configure > Mappings, click the mapping reconciliation to cancel and click Cancel Reconciliation. | ## List reconciliation history Display a list of reconciliation processes that have completed and those in progress by performing a RESTful GET on `http://localhost:8080/openidm/recon`. The following example displays all reconciliation runs: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/recon" ``` > **Collapse: Example Output** > > The output is similar to the following, with one item for each reconciliation run: > > ```json > "reconciliations": [ > { > "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144", > "mapping": "systemLdapAccounts_managedUser", > "state": "SUCCESS", > "stage": "COMPLETED_SUCCESS", > "stageDescription": "reconciliation completed.", > "progress": { > "source": { > "existing": { > "processed": 2, > "total": "2" > } > }, > "target": { > "existing": { > "processed": 0, > "total": "0" > }, > "created": 2, > "unchanged": 0, > "updated": 0, > "deleted": 0 > }, > "links": { > "existing": { > "processed": 0, > "total": "0" > }, > "created": 2 > } > }, > "situationSummary": { > "SOURCE_IGNORED": 0, > "FOUND_ALREADY_LINKED": 0, > "UNQUALIFIED": 0, > "ABSENT": 2, > "TARGET_IGNORED": 0, > "MISSING": 0, > "ALL_GONE": 0, > "UNASSIGNED": 0, > "AMBIGUOUS": 0, > "CONFIRMED": 0, > "LINK_ONLY": 0, > "SOURCE_MISSING": 0, > "FOUND": 0 > }, > "statusSummary": { > "SUCCESS": 2, > "FAILURE": 0 > }, > "durationSummary": { > "sourceQuery": { > "min": 42, > "max": 42, > "mean": 42, > "count": 1, > "sum": 42, > "stdDev": 0 > }, > "auditLog": { > "min": 0, > "max": 1, > "mean": 0, > "count": 24, > "sum": 15, > "stdDev": 0 > }, > "linkQuery": { > "min": 5, > "max": 5, > "mean": 5, > "count": 1, > "sum": 5, > "stdDev": 0 > }, > "targetQuery": { > "min": 3, > "max": 3, > "mean": 3, > "count": 1, > "sum": 3, > "stdDev": 0 > }, > "targetPhase": { > "min": 0, > "max": 0, > "mean": 0, > "count": 1, > "sum": 0, > "stdDev": 0 > }, > "sourceObjectQuery": { > "min": 6, > "max": 34, > "mean": 21, > "count": 22, > "sum": 474, > "stdDev": 9 > }, > "postMappingScript": { > "min": 0, > "max": 1, > "mean": 0, > "count": 22, > "sum": 17, > "stdDev": 0 > }, > "onMappingScript": { > "min": 0, > "max": 4, > "mean": 2, > "count": 22, > "sum": 48, > "stdDev": 2 > }, > "sourcePhase": { > "min": 490, > "max": 490, > "mean": 490, > "count": 1, > "sum": 490, > "stdDev": 0 > } > }, > "parameters": { > "sourceQuery": { > "resourceName": "system/ldap/account", > "queryFilter": "true", > "_fields": "_id" > }, > "targetQuery": { > "resourceName": "managed/user", > "queryFilter": "true", > "_fields": "_id" > } > }, > "started": "2020-05-07T09:14:57.740Z", > "ended": "2020-05-07T09:14:58.325Z", > "duration": 585, > "sourceProcessedByNode": {} > } > ] > ``` ### Reconciliation association storage You can adjust the number of reconciliation runs stored in IDM by adding the `maxAnalysisRunsPerMapping` and `maxNonAnalysisRunsPerMapping` properties to a [mapping](mappings.html): ```json "reconAssociation" : { "maxAnalysisRunsPerMapping" : 1, "maxNonAnalysisRunsPerMapping" : 3 } ``` In this context, *analysis* refers to reconciliation runs triggered with the `analyze=true` parameter. These runs determine which actions *would* be performed in a real reconciliation but don't execute them. Non-analysis refers to a normal reconciliation. The default value for both properties is `1`. For analysis results to be stored and available for querying with the `recon/assoc` endpoints, the reconciliation must also be run with `persistAssociations=true`. In contrast, the admin UI displays the results of only the most recent reconciliation. Learn more in [View reconciliation details using the admin UI](#recon-details-ui). ### Reconciliation properties Each reconciliation run includes the following properties: * `_id` The ID of the reconciliation run. * `mapping` The name of the [mapping](mappings.html). * `state` The high-level state of the reconciliation run. Values can be as follows: * `ACTIVE` The reconciliation run is in progress. * `CANCELED` The reconciliation run was canceled. * `FAILED` The reconciliation run was terminated because of failure. * `SUCCESS` The reconciliation run completed successfully. * `stage` The current stage of the reconciliation run. Values can be as follows: * `ACTIVE_INITIALIZED` The initial stage, when a reconciliation run is first created. * `ACTIVE_QUERY_ENTRIES` Querying the source, target, and possibly link sets to reconcile. * `ACTIVE_RECONCILING_SOURCE` Reconciling the set of IDs retrieved from the mapping source. * `ACTIVE_RECONCILING_TARGET` Reconciling any remaining entries from the set of IDs retrieved from the mapping target, that were not matched or processed during the source phase. * `ACTIVE_LINK_CLEANUP` Checking whether any links are now unused and should be cleaned up. * `ACTIVE_PROCESSING_RESULTS` Post-processing of reconciliation results. * `ACTIVE_CANCELING` Attempting to abort a reconciliation run in progress. * `COMPLETED_SUCCESS` Successfully completed processing the reconciliation run. * `COMPLETED_CANCELED` Completed processing because the reconciliation run was aborted. * `COMPLETED_FAILED` Completed processing because of a failure. * `stageDescription` A description of the stages described previously. * `progress` The progress object has the following structure (annotated here with comments): ```json "progress":{ "source":{ // Progress on set of existing entries in the mapping source "existing":{ "processed":1001, "total":"1001" // Total number of entries in source set, if known, "?" otherwise } }, "target":{ // Progress on set of existing entries in the mapping target "existing":{ "processed":1001, "total":"1001" // Total number of entries in target set, if known, "?" otherwise }, "created":0 // New entries that were created }, "links":{ // Progress on set of existing links between source and target "existing":{ "processed":1001, "total":"1001" // Total number of existing links, if known, "?" otherwise }, "created":0 // Denotes new links that were created } }, ``` ## Reconciliation details To display the details of a specific reconciliation over REST, include the reconciliation run ID in the URL. The following call shows the details of the reconciliation run initiated in [Trigger a reconciliation](#triggering-recons). ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/recon/05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144" ``` > **Collapse: Example Output** > > ```json > { > "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144", > "mapping": "systemLdapAccounts_managedUser", > "state": "SUCCESS", > "stage": "COMPLETED_SUCCESS", > "stageDescription": "reconciliation completed.", > "progress": { > "source": { > "existing": { > "processed": 2, > "total": "2" > } > }, > "target": { > "existing": { > "processed": 0, > "total": "0" > }, > "created": 2, > "unchanged": 0, > "updated": 0, > "deleted": 0 > }, > "links": { > "existing": { > "processed": 0, > "total": "0" > }, > "created": 2 > } > }, > "situationSummary": { > "SOURCE_IGNORED": 0, > "FOUND_ALREADY_LINKED": 0, > "UNQUALIFIED": 0, > "ABSENT": 2, > "TARGET_IGNORED": 0, > "MISSING": 0, > "ALL_GONE": 0, > "UNASSIGNED": 0, > "AMBIGUOUS": 0, > "CONFIRMED": 0, > "LINK_ONLY": 0, > "SOURCE_MISSING": 0, > "FOUND": 0 > }, > "statusSummary": { > "SUCCESS": 2, > "FAILURE": 0 > }, > "durationSummary": { > "sourceQuery": { > "min": 42, > "max": 42, > "mean": 42, > "count": 1, > "sum": 42, > "stdDev": 0 > }, > "auditLog": { > "min": 0, > "max": 1, > "mean": 0, > "count": 24, > "sum": 15, > "stdDev": 0 > }, > "linkQuery": { > "min": 5, > "max": 5, > "mean": 5, > "count": 1, > "sum": 5, > "stdDev": 0 > }, > "targetQuery": { > "min": 3, > "max": 3, > "mean": 3, > "count": 1, > "sum": 3, > "stdDev": 0 > }, > "targetPhase": { > "min": 0, > "max": 0, > "mean": 0, > "count": 1, > "sum": 0, > "stdDev": 0 > }, > "sourceObjectQuery": { > "min": 6, > "max": 34, > "mean": 21, > "count": 22, > "sum": 474, > "stdDev": 9 > }, > "postMappingScript": { > "min": 0, > "max": 1, > "mean": 0, > "count": 22, > "sum": 17, > "stdDev": 0 > }, > "onMappingScript": { > "min": 0, > "max": 4, > "mean": 2, > "count": 22, > "sum": 48, > "stdDev": 2 > }, > "sourcePhase": { > "min": 490, > "max": 490, > "mean": 490, > "count": 1, > "sum": 490, > "stdDev": 0 > } > }, > "parameters": { > "sourceQuery": { > "resourceName": "system/ldap/account", > "queryFilter": "true", > "_fields": "_id" > }, > "targetQuery": { > "resourceName": "managed/user", > "queryFilter": "true", > "_fields": "_id" > } > }, > "started": "2020-05-07T09:14:57.740Z", > "ended": "2020-05-07T09:14:58.325Z", > "duration": 585, > "sourceProcessedByNode": {} > } > ``` ### View reconciliation details using the admin UI You can display the details of the most recent reconciliation in the admin UI. Select the mapping. The resulting page displays a message similar to the following: ``` Completed: Last reconciled November 20, 2019 15:28 ``` Clicking on the reconciliation run date displays the details of the reconciliation run. Click Reconciliation Results for additional information. If a reconciliation fails, select the Failure Summary tab for more information about the failure. To view reconciliation audit logs in the UI, add an [Audit widget](../audit-guide/audit-admin-ui.html) to the dashboard. The reconciliation Audit widget shows the same information retrieved over REST. | | | | - | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Widgets are deprecated and will be removed in a future release of IDM. For more information, refer to [Deprecation](../release-notes/deprecated-functionality.html#deprecation-widgets). | ### Reconciliation association details When performing a reconciliation run, information is reconciled between the source object and the target object. This creates an association between the two objects. This association can be recorded in IDM by including the `persistAssociations=true` parameter when triggering a reconciliation. This information can then be retrieved by querying the `recon/assoc` endpoint. | | | | - | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | The `persistAssociations` parameter is `false` by default. Setting it to `true` can cause performance issues during large reconciliations and, in extreme cases, might cause a system outage. | To get a list of stored recon associations, run the following query: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/recon/assoc?_queryFilter=true" { "result": [ { "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230", "_rev": "1", "mapping": "managedUser_systemLdapAccounts", "sourceResourceCollection": "managed/user", "targetResourceCollection": "system/ldap/account", "isAnalysis": "false", "finishTime": "2019-05-01T23:36:24.434153Z" }, { "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-99638", "_rev": "1", "mapping": "systemLdapAccounts_managedUser", "sourceResourceCollection": "system/ldap/account", "targetResourceCollection": "managed/user", "isAnalysis": "true", "finishTime": "2019-05-06T21:31:42.140066Z" } ], "resultCount": 2, "pagedResultsCookie": null, "totalPagedResultsPolicy": "NONE", "totalPagedResults": -1, "remainingPagedResults": -1 } ``` You can retrieve information for a specific reconciliation by querying the recon ID: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/recon/assoc/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230" { "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230", "_rev": "1", "mapping": "managedUser_systemLdapAccounts", "sourceResourceCollection": "managed/user", "targetResourceCollection": "system/ldap/account", "isAnalysis": "false", "finishTime": "2019-05-01T23:36:24.434153Z" } ``` It's possible to also get the specific association details of each entry in the reconciliation run by appending `/entry` to the query: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/recon/assoc/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230/entry?_queryFilter=true" { "result": [ { "_id": "400d40fd-da58-41f5-857b-71855eb97bd9", "_rev": "0", "mapping": "managedUser_systemLdapAccounts", "reconId": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230", "situation": "CONFIRMED", "action": "UPDATE", "linkQualifier": "default", "sourceObjectId": "07978ba5-b31d-4f8b-9f60-506c07f68495", "targetObjectId": "ca8abc7f-7b97-3e96-94fb-6b27b0ec5aed", "sourceResourceCollection": "managed/user", "targetResourceCollection": "system/ldap/account", "status": "SUCCESS", "exception": null, "message": null, "messageDetail": "null", "ambiguousTargetObjectIds": null }, ... { "_id": "08ec633c-744f-4092-b88d-fe253b1d8e52", "_rev": "0", "mapping": "managedUser_systemLdapAccounts", "reconId": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230", "situation": "CONFIRMED", "action": "UPDATE", "linkQualifier": "default", "sourceObjectId": "ee2449a8-01e6-4c0b-84d3-e65e25c3e38c", "targetObjectId": "67a6596e-ebfc-3542-a664-1ab1610e082a", "sourceResourceCollection": "managed/user", "targetResourceCollection": "system/ldap/account", "status": "SUCCESS", "exception": null, "message": null, "messageDetail": "null", "ambiguousTargetObjectIds": null } ], ... } ``` | | | | - | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | For large reconciliations, the results returned by the `/entry` endpoint can be substantial because they include details for every object reconciled. Use query filters to tune queries and return only the required subset of results. | ### Purge reconciliation statistics When the number of completed reconciliation runs for a given mapping reaches the number specified by `maxAnalysisRunsPerMapping` or `maxNonAnalysisRunsPerMapping`, statistics are purged automatically. Statistics and reconciliation run information (such as recon associations) are purged chronologically by mapping, with the oldest reconciliation run for that mapping purged first. You can also remove reconciliation statistics. To purge reconciliation statistics from the repository, perform a DELETE request on the reconciliation run ID. For example: ``` curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request DELETE \ "http://localhost:8080/openidm/recon/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230" ```