ICF 1.5.20.25

Database Table connector

The Database Table connector lets you provision to a single table in a JDBC database.

Install the Database Table connector

If you are looking for the Advanced Identity Cloud application for this connector, refer to:

You can download any connector from Backstage, but some are included in the default deployment for Advanced Identity Cloud, IDM, or RCS. When using an included connector, you can skip installing it and move directly to configuration.

Connector included in default deployment
Connector IDM RCS

Yes

Yes

Download the connector .jar file from Backstage.

  • If you are running the connector locally, place it in the /path/to/openidm/connectors directory, for example:

    mv ~/Downloads/databasetable-connector-1.5.20.23.jar /path/to/openidm/connectors/
  • If you are using a remote connector server (RCS), place it in the /path/to/openicf/connectors directory on the RCS.

Configure the Database Table connector

Create a connector configuration using the IDM admin UI:

  1. From the navigation bar, click Configure > Connectors.

  2. On the Connectors page, click New Connector.

  3. On the New Connector page, type a Connector Name.

  4. From the Connector Type drop-down list, select Database Table Connector - 1.5.20.23.

  5. Complete the Base Connector Details.

    For a list of all configuration properties, refer to Database Table Connector Configuration
  6. Click Save.

When your connector is configured correctly, the connector displays as Active in the admin UI.

Refer to this procedure to create a connector configuration over REST.

Alternatively, use the sample connector configuration for the Database Table connector in samples/example-configurations/provisioners/provisioner.openicf-contractordb.json. The corresponding data definition language file is provided in samples/example-configurations/provisioners/provisioner.openicf-contractordb.sql.

The following excerpt shows a sample Database Table connector configuration:

"configurationProperties" : {
    "url" : "jdbc:mysql://localhost:3306/contractordb?serverTimezone=UTC",
    "driverClassName" : "com.mysql.cj.jdbc.Driver",
    "username" : "root",
    "password" : "password",
    "table" : "people",
    "keyColumn" : "EMAIL",
    "passwordColumn" : "",
    "changeLogColumn" : "CHANGE_TIMESTAMP",
    "disablePaging" : false,
    "enableEmptyString" : false,
    "quoting" : "",
    "rethrowAllSQLExceptions" : true,
    "nativeTimestamps" : false,
    "allNative" : false,
    "suppressPassword" : true,
    "validationQueryTimeout" : -1,
    "validationQuery" : "SELECT 1 FROM DUAL",
    "validationInterval" : 3000,
    "initialSize" : 10,
    "maxIdle" : 100,
    "minIdle" : 10,
    "maxWait" : 30000,
    "maxActive" : 100,
    "maxAge" : 0,
    "minEvictableIdleTimeMillis" : 60000,
    "timeBetweenEvictionRunsMillis" : 5000,
    "testWhileIdle" : false,
    "testOnBorrow" : true
}

The mandatory configurable properties are as follows:

url

The JDBC database address that contains the table to which you are provisioning. The format of the url will change depending on the type of database, such as jdbc:mysql://localhost:3306/contractordb?serverTimezone=UTC, or jdbc:oracle:thin:@//localhost:3306/contractordb. Note that the address includes the name of the database you are connecting to.

driverClassName

The class name of the driver you are using to connect to a database. The name varies depending on the type of database you are using, such as oracle.jdbc.OracleDriver, or com.mysql.cj.jdbc.Driver.

table

The name of the table in the JDBC database that contains the user accounts.

keyColumn

The column value that is used as the unique identifier for rows in the table.

If you want to map NAME or UID to an attribute in IDM, change the keyColumn to a column in the SQL schema that does not match any of the target properties in your mapping; otherwise, a conflict occurs and IDM does not create the account. Previously, this column was UNIQUE_ID.

Unless the database is configured to not need authentication, username and password are also required.

Tomcat JDBC connection pool

The Database Table connector uses the Apache Tomcat JDBC Connection Pool. Additional configurable properties and information are available in the Apache Tomcat documentation.

Database Table remote connector

If you want to run this connector outside of PingOne Advanced Identity Cloud or IDM, you can configure the Database Table connector as a remote connector. Java Connectors installed remotely on a Java Connector Server function identically to those bundled locally within PingOne Advanced Identity Cloud or installed locally on IDM.

You can download the Database Table connector from here.

Refer to Remote connectors for configuring the Database Table remote connector.

Implementation specifics

  • To use this connector for liveSync, add a changelog type column to the database and provide the name of this column in the changeLogColumn property. Note that the Database Table connector supports liveSync for create and update operations only. To detect deletes in the database you must run a full reconciliation.

  • For PATCH requests, a connector can potentially add, remove, or replace an attribute value. The Database Table connector does not implement the add or remove operations, so a PATCH request always replaces the entire attribute value with the new value.

  • The Database Table connector supports paged reconciliation queries only for the following databases:

    • MySQL

    • PostgreSQL

    • Oracle Database 11gR2 and later versions

    • Microsoft SQL Server 2012 and later versions

      Paging is enabled by default. If you are connecting to a database for which paging is not supported, you must disable it by setting "disablePaging" : true in the connector configuration.

    For more information about configuring paged reconciliation queries, refer to Paging Reconciliation Query Results.

  • If your database does not support precise (nanosecond) timestamps, you can use the inclusiveSync configuration property to ensure that modified entries are not missed in liveSync operations. If inclusiveSync is set to true, the connector synchronizes all entries whose change timestamp is greater than or equal to the syncToken. Be aware that if you set this property to true, the activity log creates a new entry every time liveSync occurs, even if entries are changed. This can lead to rapid growth of the activity audit log.

OpenICF Interfaces Implemented by the Database Table Connector

The Database Table Connector implements the following OpenICF interfaces. For additional details, see ICF interfaces:

Authenticate

Provides simple authentication with two parameters, presumed to be a user name and password.

Create

Creates an object and its uid.

Delete

Deletes an object, referenced by its uid.

Resolve Username

Resolves an object by its username and returns the uid of the object.

Schema

Describes the object types, operations, and options that the connector supports.

Script on Connector

Enables an application to run a script in the context of the connector.

Any script that runs on the connector has the following characteristics:

  • The script runs in the same execution environment as the connector and has access to all the classes to which the connector has access.

  • The script has access to a connector variable that is equivalent to an initialized instance of the connector. At a minimum, the script can access the connector configuration.

  • The script has access to any script arguments passed in by the application.

Search

Searches the target resource for all objects that match the specified object class and filter.

Sync

Polls the target resource for synchronization events, that is, native changes to objects on the target resource.

Test

Tests the connector configuration.

Testing a configuration checks all elements of the environment that are referred to by the configuration are available. For example, the connector might make a physical connection to a host that is specified in the configuration to verify that it exists and that the credentials that are specified in the configuration are valid.

This operation might need to connect to a resource, and, as such, might take some time. Do not invoke this operation too often, such as before every provisioning operation. The test operation is not intended to check that the connector is alive (that is, that its physical connection to the resource has not timed out).

You can invoke the test operation before a connector configuration has been validated.

Update

Updates (modifies or replaces) objects on a target resource.

Database Table Connector Configuration

The Database Table Connector has the following configurable properties:

Configuration properties

Property Type Default Encrypted(1) Required(2)

connectionProperties

String

null

No

The connection properties that will be sent to our JDBC driver when establishing new connections. Format of the string must be [propertyName=property;]* NOTE - The "user" and "password" properties will be passed explicitly, so they do not need to be included here. The default value is null.

propagateInterruptState

boolean

false

No

Set this to true to propagate the interrupt state for a thread that has been interrupted (not clearing the interrupt state). Default value is false for backwards compatibility.

useDisposableConnectionFacade

boolean

true

No

Set this to true if you wish to put a facade on your connection so that it cannot be reused after it has been closed. This prevents a thread holding on to a reference of a connection it has already called closed on, to execute queries on it.

defaultCatalog

String

null

No

The default catalog of connections created by this pool.

validationInterval

long

3000

No

To avoid excess validation, run validation at most at this frequency (in milliseconds). If a connection is due for validation, but was validated within this interval, it will not be validated again. The default value is 3000 (3 seconds).

ignoreExceptionOnPreLoad

boolean

false

No

Flag whether ignore error of connection creation while initializing the pool. Set to true if you want to ignore error of connection creation while initializing the pool. Set to false if you want to fail the initialization of the pool by throwing exception.

jmxEnabled

boolean

true

No

Register the pool with JMX or not. The default value is true.

commitOnReturn

boolean

false

No

If autoCommit==false then the pool can complete the transaction by calling commit on the connection as it is returned to the pool If rollbackOnReturn==true then this attribute is ignored. Default value is false.

logAbandoned

boolean

false

No

Flag to log stack traces for application code which abandoned a Connection. Logging of abandoned Connections adds overhead for every Connection borrow because a stack trace has to be generated. The default value is false.

maxIdle

int

100

No

The maximum number of connections that should be kept in the pool at all times. Idle connections are checked periodically (if enabled) and connections that have been idle for longer than minEvictableIdleTimeMillis are released. The default value is derived from maxActive:100. (Also see testWhileIdle).

testWhileIdle

boolean

false

No

The indication of whether objects will be validated by the idle object evictor (if any). If an object fails to validate, it will be dropped from the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. The default value is false and this property has to be set in order for the pool cleaner/test thread is to run (also see timeBetweenEvictionRunsMillis).

removeAbandoned

boolean

false

No

Flag to remove abandoned connections if they exceed the removeAbandonedTimeout. If set to true a connection is considered abandoned and eligible for removal if it has been in use longer than the removeAbandonedTimeout Setting this to true can recover db connections from applications that fail to close a connection. See also logAbandoned The default value is false.

abandonWhenPercentageFull

int

0

No

Connections that have been abandoned (timed out) wont get closed and reported up unless the number of connections in use are above the percentage defined by abandonWhenPercentageFull. The value should be between 0-100. The default value is 0, which implies that connections are eligible for closure as soon as removeAbandonedTimeout has been reached.

minIdle

int

10

No

The minimum number of established connections that should be kept in the pool at all times. The connection pool can shrink below this number if validation queries fail. The default value is derived from initialSize:10. (Also see testWhileIdle).

defaultReadOnly

Boolean

null

No

The default read-only state of connections created by this pool. If not set then the setReadOnly method will not be called. (Some drivers dont support read only mode, ex: Informix)

maxWait

int

30000

No

The maximum number of milliseconds that the pool will wait (when there are no available connections) for a connection to be returned before throwing an exception. Default value is 30000 (30 seconds).

logValidationErrors

boolean

false

No

Set this to true to log errors during the validation phase to the log file. If set to true, errors will be logged as SEVERE. Default value is false for backwards compatibility.

driverClassName

String

null

No

The fully qualified Java class name of the JDBC driver to be used. The driver has to be accessible from the same classloader as tomcat-jdbc.jar.

name

String

Tomcat Connection Pool[1-2629388]

No

Returns the name of the connection pool. By default a JVM unique random name is assigned.

useStatementFacade

boolean

true

No

Returns true if this connection pool is configured to wrap statements in order to enable equals() and hashCode() methods to be called on the closed statements if any statement proxy is set.

initSQL

String

null

No

A custom query to be run when a connection is first created. The default value is null.

validationQueryTimeout

int

-1

No

The timeout in seconds before a connection validation queries fail. This works by calling java.test_sample.Statement.setQueryTimeout(seconds) on the statement that executes the validationQuery. The pool itself doesnt timeout the query, it is still up to the JDBC driver to enforce query timeouts. A value less than or equal to zero will disable this feature. The default value is -1.

validationQuery

String

null

No

The SQL query that will be used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just cant throw a SQLException. The default value is null. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server).

rollbackOnReturn

boolean

false

No

If autoCommit==false then the pool can terminate the transaction by calling rollback on the connection as it is returned to the pool Default value is false.

alternateUsernameAllowed

boolean

false

No

By default, the jdbc-pool will ignore the DataSource.getConnection(username,password) call, and simply return a previously pooled connection under the globally configured properties username and password, for performance reasons. The pool can however be configured to allow use of different credentials each time a connection is requested. To enable the functionality described in the DataSource.getConnection(username,password) call, simply set the property alternateUsernameAllowed to true. Should you request a connection with the credentials user1/password1 and the connection was previously connected using different user2/password2, the connection will be closed, and reopened with the requested credentials. This way, the pool size is still managed on a global level, and not on a per schema level.

validatorClassName

String

null

No

The name of a class which implements the org.apache.tomcat.jdbc.pool.Validator interface and provides a no-arg constructor (may be implicit). If specified, the class will be used to create a Validator instance which is then used instead of any validation query to validate connections. The default value is null. An example value is com.mycompany.project.SimpleValidator.

suspectTimeout

int

0

No

Timeout value in seconds. Similar to to the removeAbandonedTimeout value but instead of treating the connection as abandoned, and potentially closing the connection, this simply logs the warning if logAbandoned is set to true. If this value is equal or less than 0, no suspect checking will be performed. Suspect checking only takes place if the timeout value is larger than 0 and the connection was not abandoned or if abandon check is disabled. If a connection is suspect a WARN message gets logged and a JMX notification gets sent once.

useEquals

boolean

true

No

Set to true if you wish the ProxyConnection class to use String.equals and set to false when you wish to use == when comparing method names. This property does not apply to added interceptors as those are configured individually. The default value is true.

removeAbandonedTimeout

int

60

No

Timeout in seconds before an abandoned(in use) connection can be removed. The default value is 60 (60 seconds). The value should be set to the longest running query your applications might have.

defaultAutoCommit

Boolean

null

No

The default auto-commit state of connections created by this pool. If not set, default is JDBC driver default (If not set then the setAutoCommit method will not be called).

testOnConnect

boolean

false

No

Returns true if we should run the validation query when connecting to the database for the first time on a connection. Normally this is always set to false, unless one wants to use the validationQuery as an init query.

jdbcInterceptors

String

null

No

A semicolon separated list of classnames extending org.apache.tomcat.jdbc.pool.JdbcInterceptor class. See Configuring JDBC interceptors below for more detailed description of syntaz and examples. These interceptors will be inserted as an interceptor into the chain of operations on a java.test_sample.Connection object. The default value is null.

initialSize

int

10

No

The initial number of connections that are created when the pool is started. Default value is 10.

defaultTransactionIsolation

int

-1

No

The default TransactionIsolation state of connections created by this pool. One of the following: NONE, READ_COMMITTED, READ_UNCOMMITTED, REPEATABLE_READ, SERIALIZABLE If not set, the method will not be called and it defaults to the JDBC driver.

numTestsPerEvictionRun

int

0

No

Property not used in tomcat-jdbc-pool.

url

String

null

No

The URL used to connect to the database.

testOnBorrow

boolean

false

No

The indication of whether objects will be validated before being borrowed from the pool. If the object fails to validate, it will be dropped from the pool, and we will attempt to borrow another. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. In order to have a more efficient validation, see validationInterval. Default value is false.

fairQueue

boolean

true

No

Set to true if you wish that calls to getConnection should be treated fairly in a true FIFO fashion. This uses the org.apache.tomcat.jdbc.pool.FairBlockingQueue implementation for the list of the idle connections. The default value is true. This flag is required when you want to use asynchronous connection retrieval. Setting this flag ensures that threads receive connections in the order they arrive. During performance tests, there is a very large difference in how locks and lock waiting is implemented. When fairQueue=true there is a decision making process based on what operating system the system is running. If the system is running on Linux (property os.name=Linux. To disable this Linux specific behavior and still use the fair queue, simply add the property org.apache.tomcat.jdbc.pool.FairBlockingQueue.ignoreOS=true to your system properties before the connection pool classes are loaded.

accessToUnderlyingConnectionAllowed

boolean

true

No

Property not used. Access can be achieved by calling unwrap on the pooled connection. see javax.test_sample.DataSource interface, or call getConnection through reflection or cast the object as javax.test_sample.PooledConnection

maxAge

long

0

No

Time in milliseconds to keep this connection. When a connection is returned to the pool, the pool will check to see if the now - time-when-connected > maxAge has been reached, and if so, it closes the connection rather than returning it to the pool. The default value is 0, which implies that connections will be left open and no age check will be done upon returning the connection to the pool.

minEvictableIdleTimeMillis

int

60000

No

The minimum amount of time an object may sit idle in the pool before it is eligible for eviction. The default value is 60000 (60 seconds).

timeBetweenEvictionRunsMillis

int

5000

No

The number of milliseconds to sleep between runs of the idle connection validation/cleaner thread. This value should not be set under 1 second. It dictates how often we check for idle, abandoned connections, and how often we validate idle connections. The default value is 5000 (5 seconds).

testOnReturn

boolean

false

No

The indication of whether objects will be validated before being returned to the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. The default value is false.

useLock

boolean

false

No

Return true if a lock should be used when operations are performed on the connection object. Should be set to false unless you plan to have a background thread of your own doing idle and abandon checking such as JMX clients. If the pool sweeper is enabled, then the lock will automatically be used regardless of this setting.

maxActive

int

100

No

The maximum number of active connections that can be allocated from this pool at the same time. The default value is 100.

username

String

null

No

The connection username to be passed to our JDBC driver to establish a connection. Note that method DataSource.getConnection(username,password) by default will not use credentials passed into the method, but will use the ones configured here. See alternateUsernameAllowed property for more details.

table

String

TABLE_NAME

Yes

Enter the name of the table in the database that contains the accounts.

(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.

(2) A list of operations in this column indicates that the property is required for those operations.

Basic Configuration Properties

Property Type Default Encrypted(1) Required(2)

password

String

null

Yes

Yes

The connection password to be passed to the JDBC driver to establish a connection. Note that method DataSource.getConnection(username,password) by default will not use credentials passed into the method, but will use the ones configured here. See alternateUsernameAllowed property for more details.

quoting

String

NONE

No

Select whether database column names for this resource should be quoted, and the quoting characters. By default, database column names are not quoted (None). For other selections (Single, Double, Back, or Brackets), column names will appear between single quotes, double quotes, back quotes, or brackets in the SQL generated to access the database.

keyColumn

String

KEY_COLUMN

Yes

This mandatory column value will be used as the unique identifier for rows in the table.

passwordColumn

String

null

No

Enter the name of the column in the table that will hold the password values. If empty, no validation is done on resources and passwords.

disablePaging

boolean

false

Yes

If true, optional paging in a query will be ignored by the connector. Defaults to false.

enableEmptyString

boolean

false

No

Select to enable support for writing an empty string, instead of a NULL value, in character based columns defined as not-null in the table schema. This option does not influence the way strings are written for Oracle based tables. By default empty strings are written as a NULL value.

rethrowAllSQLExceptions

boolean

true

No

If this is not checked, SQL statements which throw SQLExceptions with a 0 ErrorCode will be have the exception caught and suppressed. Check it to have exceptions with 0 ErrorCodes rethrown.

nativeTimestamps

boolean

false

No

Select to retrieve Timestamp data type of the columns in java.sql.Timestamp format from the database table.

allNative

boolean

false

No

Select to retrieve all data types of columns in native format from the database table.

changeLogColumn

String

null

The change log column stores the latest change time. Providing this value the Sync capabilities are activated.

suppressPassword

boolean

true

No

If set to true then the password will not be returned. Never. Even though it is explicitly requested. If set to false then the password will be returned if it is explicitly requested.

inclusiveSync

boolean

false

No

If true, the SyncOp will query for ChangeLogColumn >= syncToken. One record will always be returned from the database in this case and be handled by the connector. If set to false, the SyncOp will query for ChangeLogColumn > syncToken. Defaults to false.

returnGeneratedKeys

boolean

true

No

If set to true then the JDBC Statement.RETURN_GENERATED_KEYS is used for prepared statement. It may need to be set to false with Oracle JDBC driver for create() operation. Defaults to tue.

(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.

(2) A list of operations in this column indicates that the property is required for those operations.