PingDirectory

Configure a Kafka sync destination

Use the dsconfig command or the administrative console to configure PingDataSync to synchronize changes to an Apache Kafka environment.

PingDataSync supports synchronization of single and multivalued attributes to Kafka. You can reuse existing Ping Identity sync sources that were created for other Sync Pipes.

To view an example configuration, see the file located at <server-root>/config/sample-dsconfig-batch-files/reference-sync-pingdirectory-to-kafka.dsconfig.

To configure Kerberos authentication for a Kafka sync destination, supply the producer-property attribute with the appropriate values according to the Apache Kafka documentation.

The following objects are required to configure a Kafka sync destination:

  • Kafka cluster external server – Defines the procedure for connecting to a Kafka cluster. The Kafka cluster external server can be referenced from multiple Kafka sync destination configuration objects. The only required property is bootstrap-server, which identifies some of the Kafka brokers in the environment.

    When use-ssl is set to true, the following configuration changes are made:

    • A trust-manager-provider is configured to validate the Kafka broker’s SSL certificate.

    • A key-manager-provider is configured to let the Kafka broker authenticate the PingDataSync Kafka producer.

  • Kafka sync destination – References the Kafka cluster external server. The Kafka sync destination must specify the name of the topic to use for publishing messages.

    To adjust Kafka messages beyond the mapping, attribute filtering, and other configuration changes that PingDataSync makes, reference one or more of the KafkaSyncDestinationPlugin extension points that are implemented by using the Server SDK.

Run the prepare-endpoint-server command for the PingDirectory sync source.

Obscuring sensitive producer property values

About this task

When configuring a PingDataSync Kafka producer, you might add producer properties that contain sensitive values such as keys or passwords. To prevent storing these sensitive values in plain text, you can use the sensitive-kafka-producer-property configuration property.

You create a sensitive-kafka-producer-property using the following required arguments:

--property-name

Specifies the name of the sensitive Kafka producer property.

--set sensitive-producer-key:<key>

Specifies the name of the valid property key that contains a sensitive value.

--set sensitive-producer-value:<value>

Specifies the sensitive value associated with the producer key.

Steps

  • Create one or more sensitive Kafka producer properties using dsconfig create-sensitive-kafka-producer-property.

    Example:

    $ bin/dsconfig create-sensitive-kafka-producer-property \
  --property-name saslConfig \
  --set "sensitive-producer-key:sasl.jaas.config" \
  --set "sensitive-producer-value:org.apache.kafka.common.security.scram.ScramLoginModule" \
    required username="username" password="password";

    Result:

    Perform an ldapsearch for the sensitive property:

    ldapsearch --baseDN "cn=saslConfig,cn=Sensitive Kafka Producer Property,cn=config" "(objectclass=*)"

    The sensitive value is now obscured.

    dn: cn=saslConfig,cn=Sensitive Kafka Producer Property,cn=config
objectClass: top
objectClass: ds-cfg-sensitive-kafka-producer-property
cn: saslConfig
ds-cfg-sensitive-producer-key: sasl.jaas.config
ds-cfg-sensitive-producer-value: AADu9yRP8DyrLndvqqDzeQEK9aqqLvDBZZhgHAZbh++KgovN+kUthhyn9+1o9+AqExDmigO14YQnwakqOpTAB4LnbsvwBJos6PZzYlWMNjFNXsDtOUeBsFhVi/nErPJT+cmQijC5P1EUsKWPvjDVauBe

    The config-audit.log file that contains the dsconfig change you made to create the sensitive property also obscures the value.

  • Optional: Delete one or more sensitive Kafka producer properties using dsconfig delete-sensitive-kafka-producer-property.

    Example:

    $ bin/dsconfig delete-sensitive-kafka-producer-property \
--property-name saslConfig

SSL configuration

The following table identifies the trust-manager-provider and key-manager-provider properties of the Kafka cluster external server configuration object, as well as the Kafka configuration properties to which they map.

Configuration Object Type Configuration Property Kafka Configuration Property

File-based Trust Manager Provider

trust-store-file

ssl.truststore.location

File-based Trust Manager Provider

trust-store-pin, trust-store-pin-property, trust-store-pin-environment-variable, or trust-store-pin-file

ssl.truststore.password

File-based Key Manager Provider

key-store-file

ssl.keystore.location

File-based Key Manager Provider

key-store-pin, key-store-pin-property, key-store-pin-environment-variable, or key-store-pin-file

ssl.keystore.password

File-based Key Manager Provider

private-key-pin, private-key-pin-property, private-key-pin-environment-variable, or private-key-pin-file

ssl.key.password