ICF 1.5.20.24

Scripted REST connector

The Scripted REST connector is an implementation of the Scripted Groovy connector toolkit. It can interact with any REST API, using Groovy scripts for the ICF operations. This connector type lets you develop a fully functional REST-based connector for in-house applications or any cloud-based application not yet supported with the standard set of connectors.

To use this connector, you must write a Groovy script for each operation that you want the connector to perform (create, read, update, delete, authenticate, and so on). No sample scripts are bundled with the connector, but IDM customers have access to the Scripted REST connector source code in the connectors-customers-ga repo. This repository includes sample scripts for all the ICF operations.

You cannot configure the Scripted REST connector through the UI. Configure the connector over REST, as described in Configure Connectors Over REST.

Alternatively, a sample connector configuration and scripts are provided in the /path/to/openidm/samples/scripted-rest-with-dj/ directory and described in Connect to DS with ScriptedREST. The scripts provided with this sample demonstrate how the connector can be used, but most likely cannot be used as is in your deployment. They are a good starting point from which to base your customization. For information about writing your own scripts, refer to Scripted connectors with Groovy.

Script custom behavior

The Scripted REST connector uses the Apache HTTP client library. Unlike the Scripted SQL connector, which uses JDBC drivers and a Tomcat JDBC connection pool, the Scripted REST connector includes a special script to customize the Apache HTTP client.

This customizer script lets you customize the Apache HTTP client connection pool, proxy, default headers, timeouts, and so on.

The customizer script is referenced in the connector configuration, in the CustomizerScriptFileName property:

{
...
   "configurationProperties": {
       ...
       "customizerScriptFileName": "CustomizerScript.groovy",
       ...
   }
}

The script can implement two predefined Groovy closures — init {} and decorate {}.

init {}

The Apache HTTP client provides an HTTPClientBuilder class, to build an instance of the HTTPClient. The Scripted REST connector injects this builder into the init closure when the connector is first instantiated. The init closure is the ideal place to customize the HTTP client with the builder.

You can customize the following elements of the client:

  • Connection pool

  • Connection timeouts

  • Proxy

  • Default HTTP headers

  • Certificate handling

Example init closure
/**
 * A customizer script defines the custom closures to interact with the default implementation and customize it.
 * Here, the {@link HttpClientBuilder} is passed to the customize closure. This is where the pooling, the headers,
 * the timeouts etc... should be defined.
 */
customize {
    init { HttpClientBuilder builder ->

        //SETUP: org.apache.http
        def c = delegate as ScriptedRESTConfiguration
        def httpHost = new HttpHost(c.serviceAddress?.host, c.serviceAddress?.port, c.serviceAddress?.scheme)

        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager()
        // Increase max total connection to 200
        cm.setMaxTotal(200)
        // Increase default max connection per route to 20
        cm.setDefaultMaxPerRoute(20)
        // Increase max connections for httpHost to 50
        cm.setMaxPerRoute(new HttpRoute(httpHost), 50)

        builder.setConnectionManager(cm)

        // configure timeout on the entire client
        RequestConfig requestConfig = RequestConfig.custom()/*
                                                             * .
                                                             * setConnectionRequestTimeout
                                                             * ( 50).
                                                             * setConnectTimeout
                                                             * (50)
                                                             * .setSocketTimeout
                                                             * (50)
                                                             */.build();
        builder.setDefaultRequestConfig(requestConfig)

        ...
    }
}

Call the builder methods to fit your requirements. The init{} closure does not need to return anything.

decorate {}

The init closure configures a Java instance of the HTTP client, which is injected into every CRUD script. In addition to the libraries provided by the Apache HTTP client, Groovy provides a number of libraries to deal with requests and responses.

The decorate closure lets you inject a "decorated" instance of the HTTP client into your scripts. For example, the sample scripts use the groovyx.net.http.RESTClient library.

This excerpt of a sample delete script shows the injection of the httpClient and connection variables into the script. The connection variable is the output of the decorate closure.

Example decorate closure
def operation = operation as OperationType
def configuration = configuration as ScriptedRESTConfiguration
def httpClient = connection as HttpClient
def connection = customizedConnection as RESTClient
def log = log as Log
def objectClass = objectClass as ObjectClass
def options = options as OperationOptions
def uid = uid as Uid

log.info("Entering " + operation + " Script");

switch (objectClass) {
    case ObjectClass.ACCOUNT:
        connection.delete(path: '/api/users/' + uid.uidValue);
        break
    case ObjectClass.GROUP:
        connection.delete(path: '/api/groups/' + uid.uidValue);
}

When you use the defaultRequestHeaders configuration property to set HTTP request headers, the syntax requires an = sign rather than a :. For example, to generate a request header such as "Authorization: Bearer rg1cwAeQJxEf", you must set the following value for defaultRequestHeaders in the connector configuration:

"defaultRequestHeaders" : [ "Authorization = Bearer rg1cwAeQJxEf" ]
Example OAuth2 Authentication Implementation

This example shows how to use the customizer script to implement OAuth2 authentication in the Scripted REST connector.

Although grant types are largely standardized across OAuth2 authentication providers, the way in which different providers handle flows, headers, attribute names, and so on, often differs. This makes it difficult to include a single implementation of OAuth2 authentication in the Scripted REST connector. To make sure that OAuth2 authentication works in your specific use case, you use the customizer script, which can be adapted without requiring a new version of the connector itself.

The Scripted REST connector includes a simple implementation of the OAuth2 Client Credentials grant type. The connector needs to get an access token, using the Client ID and the Client Secret, cache it, and renew it when it expires or when the server revokes it. The Apache client provides interceptors for requests and responses. These interceptors can be used in the customizer script to manage the access token:

  • In the request: If the access token is absent or expired, renew the token and cache it in the Scripted REST connector property bag.

  • In the response: If the server returns a 401 error, delete the Access Token from the connector property bag. This will ensure that the next connector request gets a new access token. The HTTP POST query to get the access token is also handled by the customizer script.

This example shows a complete customizer script for the OAuth2 implementation:

init { HttpClientBuilder builder ->

        switch (ScriptedRESTConfiguration.AuthMethod.valueOf(c.defaultAuthMethod)) {
 // ......
            case ScriptedRESTConfiguration.AuthMethod.OAUTH:
                // define a request interceptor to set the Authorization header if absent or expired
                HttpRequestInterceptor requestInterceptor = { HttpRequest request, HttpContext context ->
                    if (null == context.getAttribute("oauth-request")) {
                        def exp = c.propertyBag.tokenExpiration as Long
                        if (c.propertyBag.accessToken == null || exp < System.currentTimeMillis() / 1000) {
                            new NewAccessToken(c).clientCredentials()
                        }
                        request.addHeader(new BasicHeader(HttpHeaders.AUTHORIZATION, "Bearer " + c.propertyBag.accessToken))
                    }
                }

                // define a response interceptor to catch a 401 response code and delete access token from cache
                HttpResponseInterceptor responseInterceptor = { HttpResponse response, HttpContext context ->
                    if (HttpStatus.SC_UNAUTHORIZED == response.statusLine.statusCode) {
                        if (c.propertyBag.accessToken != null) {
                            c.propertyBag.remove("accessToken")
                            Log.getLog(ScriptedRESTConnector.class).info("Code 401 - accessToken removed")
                        }
                    }
                }

                builder.addInterceptorLast(requestInterceptor)
                builder.addInterceptorLast(responseInterceptor)
                break

            default:
                throw new IllegalArgumentException()
        }
    }

class NewAccessToken {

    static final String  GRANT_TYPE = "grant_type"
    static final String  REFRESH_TOKEN = "refresh_token"
    static final String  CLIENT_CREDENTIALS = "client_credentials"
    static final String  CLIENT_ID = "client_id"
    static final String  CLIENT_SECRET = "client_secret"
    static final String  OAUTH_REQUEST = "oauth-request"

    Log logger = Log.getLog(NewAccessToken.class)
    ScriptedRESTConfiguration c = null
    final CloseableHttpClient client = null
    final HttpPost post = null

    NewAccessToken(ScriptedRESTConfiguration conf) {
        this.c = conf
        this.client = c.getHttpClient()
        this.post = new HttpPost(c.getOAuthTokenEndpoint())
        post.setHeader(HttpHeaders.CONTENT_TYPE, "application/x-www-form-urlencoded")
        post.setHeader(HttpHeaders.ACCEPT, "application/json")
    }

    @Synchronized
    void clientCredentials() {
        boolean expired = (c.propertyBag.tokenExpiration as Long) < System.currentTimeMillis() / 1000
        if (c.propertyBag.accessToken == null || expired ) {
            if (c.propertyBag.tokenExpiration != null && expired) {
                logger.info("Token expired!")
            }
            logger.info("Getting new access token...")

            final List<NameValuePair> pairs = new ArrayList<>()
            pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS))
            pairs.add(new BasicNameValuePair(CLIENT_ID, c.getOAuthClientId()))
            pairs.add(new BasicNameValuePair(CLIENT_SECRET, SecurityUtil.decrypt(c.getOAuthClientSecret())))
            post.setEntity(new UrlEncodedFormEntity(pairs))

            CloseableHttpResponse response = null
            try {
                HttpClientContext ctx = HttpClientContext.create()
                ctx.setAttribute(OAUTH_REQUEST, true)
                response = client.execute(post, ctx)
                int statusCode = response.getStatusLine().getStatusCode()
                if (HttpStatus.SC_OK == statusCode) {
                    def jsonSlurper = new JsonSlurper()
                    def oauthResponse = jsonSlurper.parseText(EntityUtils.toString(response.getEntity()))
                    c.propertyBag.accessToken = oauthResponse.access_token
                    c.propertyBag.tokenExpiration = System.currentTimeMillis() / 1000 + oauthResponse.expires_in as Long
                } else {
                    throw new InvalidCredentialException("Retrieve Access Token failed with code: " + statusCode)
                }
            } catch (ClientProtocolException ex) {
                logger.info("Trace: {0}", ex.getMessage())
                throw new ConnectorException(ex)
            } catch (IOException ex) {
                logger.info("Trace: {0}", ex.getMessage())
                throw new ConnectionFailedException(ex)
            } finally {
                try {
                    if (response != null) {
                        response.close()
                    }
                } catch (IOException e) {
                    logger.info("Can't close HttpResponse")
                }
            }
        }
    }
}

Using the Scripted REST connector with a proxy server

If the IDM server is hosted behind a firewall and requests to the resource are routed through a proxy, you must specify the proxy host and port in the connector configuration.

To specify the proxy server details, set the proxyAddress property in the connector configuration. For example:

"configurationProperties": {
    ...
    "proxyAddress": "http://myproxy:8080",
    ...
}

Run scripts through the connector

Groovy toolkit connectors have two operations that allow you to run arbitrary script actions: runScriptOnConnector and runScriptOnResource. runScriptOnConnector is an operation that sends the script action to the connector to be compiled and executed. runScriptOnResource is an operation that sends the script to another script to be handled.

runScriptOnConnector

The runScriptOnConnector script lets you run an arbitrary script action through the connector. This script takes the following variables as input:

configuration

A handler to the connector’s configuration object.

options

A handler to the Operation Options.

operation

The operation type that corresponds to the action (RUNSCRIPTONCONNECTOR in this case).

log

A handler to the connector’s log.

To run an arbitrary script on a Groovy toolkit connector, define the script in the systemActions property of your provisioner file:

"systemActions" : [
    {
        "scriptId" : "MyScript",
        "actions" : [
            {
                "systemType" : ".*ScriptedConnector",
                "actionType" : "groovy",
                "actionFile" : "path/to/<script-name>.groovy"
            }
        ]
    }
]

If you want to define your script in the provisioner file itself rather than in a separate file, you can use the actionSource property instead of the actionFile one. A simple example follows:

"systemActions" : [
    {
        "scriptId" : "MyScript",
        "actions" : [
            {
                "systemType" : ".*ScriptedConnector",
                "actionType" : "groovy",
                "actionSource" : "2 * 2"
            }
        ]
    }
]
It is optional to prepend the last script statement in actionSource with return.

Running MyScript will return:

{
  "actions" : [
    {
      "result": 4
    }
  ]
}

If your script accepts parameters, you may supply them in the request body or the query string. For example:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
--data-raw '{"param1":"value1"}'
"http://localhost:8080/openidm/system/groovy?_action=script&scriptId=MyScript&param2=value2"**

You can also call it through the IDM script engine. Note that the system can accept arbitrary parameters, as demonstrated here:

openidm.action("/system/groovy", "script", {"contentParameter": "value"}, {"scriptId": "MyScript", "additionalParameter1": "value1", "additionalParameter2": "value2"})

runScriptOnResource

To run an arbitrary script using runScriptOnResource, you must add some configuration details to your provisioner file. These details include a scriptOnResourceScriptFileName which references a script file located in a path contained in the scriptRoots array.

Define these properties in your provisioner file as follows:

"configurationProperties": {
  "scriptRoots": [
    "path/to/scripts"
  ],
  "scriptOnResourceScriptFileName": "ScriptOnResourceScript.groovy"
},
"systemActions" : [
    {
        "scriptId" : "script-1",
        "actions" : [
            {
                "systemType" : ".*ScriptedConnector",
                "actionType" : "groovy",
                "actionFile" : "path/to/<script-name>.groovy"
            }
        ]
    }
]

When you have defined the script, you can call it over REST on the system endpoint, as follows:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/system/groovy?_action=script&scriptId=scriptOnResourceScript&scriptExecuteMode=resource"

Implemented interfaces

This table lists the ICF interfaces that are implemented for the scripted REST connector:

OpenICF Interfaces Implemented by the Scripted REST Connector

The Scripted REST Connector implements the following OpenICF interfaces. For additional details, see ICF interfaces:

Authenticate

Provides simple authentication with two parameters, presumed to be a user name and password.

Create

Creates an object and its uid.

Delete

Deletes an object, referenced by its uid.

Resolve Username

Resolves an object by its username and returns the uid of the object.

Schema

Describes the object types, operations, and options that the connector supports.

Script on Connector

Enables an application to run a script in the context of the connector.

Any script that runs on the connector has the following characteristics:

  • The script runs in the same execution environment as the connector and has access to all the classes to which the connector has access.

  • The script has access to a connector variable that is equivalent to an initialized instance of the connector. At a minimum, the script can access the connector configuration.

  • The script has access to any script arguments passed in by the application.

Script on Resource

Runs a script on the target resource that is managed by this connector.

Search

Searches the target resource for all objects that match the specified object class and filter.

Sync

Polls the target resource for synchronization events, that is, native changes to objects on the target resource.

Test

Tests the connector configuration.

Testing a configuration checks all elements of the environment that are referred to by the configuration are available. For example, the connector might make a physical connection to a host that is specified in the configuration to verify that it exists and that the credentials that are specified in the configuration are valid.

This operation might need to connect to a resource, and, as such, might take some time. Do not invoke this operation too often, such as before every provisioning operation. The test operation is not intended to check that the connector is alive (that is, that its physical connection to the resource has not timed out).

You can invoke the test operation before a connector configuration has been validated.

Update

Updates (modifies or replaces) objects on a target resource.

Scripted REST Connector Configuration

The Scripted REST Connector has the following configurable properties:

Basic Configuration Properties

Property Type Default Encrypted(1) Required(2)

username

String

null

No

The Remote user to authenticate with.

password

GuardedString

null

Yes

No

The Password to authenticate with.

serviceAddress

URI

null

Yes

The service URI (example: http://myservice.com/api).

proxyAddress

URI

null

No

The optional Proxy server URI (example: http://myproxy:8080).

proxyUsername

String

null

No

The username to authenticate with the proxy server.

proxyPassword

GuardedString

null

Yes

No

The password to authenticate with the proxy server.

defaultAuthMethod

String

BASIC

No

Authentication method used. Can be: BASIC, BASIC_PREEMPTIVE, OAUTH or NONE.

defaultContentType

String

application/json

No

Default HTTP request content type. Can be: JSON, TEXT, XML, HTML, URLENC, BINARY.

defaultRequestHeaders

String[]

null

No

Placeholder for default HTTP request headers.

OAuthTokenEndpoint

URI

null

No

When using OAUTH, this property defines the endpoint where a new access token should be queried for (https://myserver.com/oauth2/token).

OAuthClientId

String

null

No

The client identifier.

OAuthClientSecret

GuardedString

null

Yes

No

Secure client secret for OAUTH.

OAuthRefreshToken

GuardedString

null

Yes

No

The refresh token used to renew the access token for the refresh_token grant type.

OAuthScope

String

null

No

The optional scope.

OAuthGrantType

String

client_credentials

No

The grant type to use. Can be: client_credentials or any grant type supported by the customizer script.

readRateLimit

String

null

No

Defines throttling for read operations either per seconds ("30/sec") or per minute ("100/min").

writeRateLimit

String

null

No

Defines throttling for write operations (create/update/delete) either per second ("30/sec") or per minute ("100/min").

(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.

(2) A list of operations in this column indicates that the property is required for those operations.

Groovy Engine configuration

Property Type Default Encrypted(1) Required(2)

scriptRoots

String[]

null

Yes

The root folder to load the scripts from. If the value is null or empty the classpath value is used.

classpath

String[]

[]

No

Classpath for use during compilation.

debug

boolean

false

No

If true, debugging code should be activated.

disabledGlobalASTTransformations

String[]

null

No

Sets a list of global AST transformations which should not be loaded even if they are defined in META-INF/org.codehaus.groovy.transform.ASTTransformation files. By default, none is disabled.

minimumRecompilationInterval

int

100

No

Sets the minimum of time after a script can be recompiled.

recompileGroovySource

boolean

false

No

If set to true recompilation is enabled.

scriptBaseClass

String

null

No

Base class name for scripts (must derive from Script).

scriptExtensions

String[]

['groovy']

No

Gets the extensions used to find groovy files.

sourceEncoding

String

UTF-8

No

Encoding for source files.

targetDirectory

File

null

No

Directory into which to write classes.

tolerance

int

10

No

The error tolerance, which is the number of non-fatal errors (per unit) that should be tolerated before compilation is aborted.

verbose

boolean

false

No

If true, the compiler should produce action information.

warningLevel

int

1

No

Warning Level of the compiler.

customConfiguration

String

null

No

Custom Configuration script for Groovy ConfigSlurper.

customSensitiveConfiguration

GuardedString

null

Yes

No

Custom Sensitive Configuration script for Groovy ConfigSlurper.

(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.

(2) A list of operations in this column indicates that the property is required for those operations.

Operation Script Files

Property Type Default Encrypted(1) Required(2)

authenticateScriptFileName

String

null

The name of the file used to perform the AUTHENTICATE operation.

createScriptFileName

String

null

The name of the file used to perform the CREATE operation.

customizerScriptFileName

String

null

No

The script used to customize some function of the connector. Read the documentation for more details.

deleteScriptFileName

String

null

The name of the file used to perform the DELETE operation.

resolveUsernameScriptFileName

String

null

The name of the file used to perform the RESOLVE_USERNAME operation.

schemaScriptFileName

String

null

The name of the file used to perform the SCHEMA operation.

scriptOnResourceScriptFileName

String

null

The name of the file used to perform the RUNSCRIPTONRESOURCE operation.

searchScriptFileName

String

null

The name of the file used to perform the SEARCH operation.

syncScriptFileName

String

null

The name of the file used to perform the SYNC operation.

testScriptFileName

String

null

The name of the file used to perform the TEST operation.

updateScriptFileName

String

null

The name of the file used to perform the UPDATE operation.

(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.

(2) A list of operations in this column indicates that the property is required for those operations.