By default, in the config/hosts file, Kafka and Zookeeper are configured to be installed as part of the deployment framework:
# this option can be used if there is an existing kafka installation that can be used
# set it to false if kafka need not be installed
  • To disable the Kafka and Zookeeper installation, set install_kafka=false.
  • Edit the parameters in config/kafka-defaults.yml to change the default installation settings for Kafka and Zookeeper.
     # Define ports for the Kafka brokers
     # These ports remain same for all brokers
     ssl_port: 9094
     #Port to be used for Communication with abs and dataengine
     sasl_port: 9093
     #kafka jks password
     jks_password: changeme
     #Enable Delete topics in kafka
     delete_topic: false
     ssl_key_alias: pingidentity
     wait_time_before_clean: 30
     startup_timeout: 120
       # Define ports for the zookeeper brokers
       # These ports remain same for all zookeeper
       ssl_port: 2182
    Kafka variables
    Variable Description
    ssl_port SSL port for Kafka.

    default: 9094

    sasl_port SASL port that is also used by the data engine and ABS to communicate with Kafka

    default: 9093

    jks_password JKS password.

    If a custom truststore and keystore is provided, you can configure the password here.

    delete_topic Enables topic deletion in Kafka.
    wait_time_before_clean Waiting time before cleaning existing data in Kafka, for new installation.
    startup_timeout Waiting time for Kafka to start.
    Zookeeper variables
    ssl_port SSL port for Zookeeper, also used for communication to Kafka via Kafka's SSL port..

    default :9093

Changing default Kafka configurations in config/hosts

The config/hosts file has a Kafka configuration, for example:
[kafka] zookeeper_id=1
You can install Kafka and Zookeeper as a cluster by providing multiple IPs or hosts in the host file, and zookeeper_id according to the number of nodes to install.
  • zookeeper_id should start from 1.
  • Each new node's zookeeper_id should increase by 1.
  • The nodes should be listed in ascending zookeeper_id order.
  • The number of nodes should only be either one or three, for example:
    • One node:
      <IP address> zookeeper_id=1
    • Three nodes:
      <IP address 1> zookeeper_id=1
      <IP address 2> zookeeper_id=2
      <IP address 3> zookeeper_id=3
You can provide custom crt and key files in the certs folder location:
kafka_private_key_location: "{{ cert_dir }}/kafka/kafka.key"
kafka_cert_location: "{{ cert_dir }}/kafka/kafka.crt"

Configure Kafka keystore password

You can configure the keystore password in config/kafka-defaults.yml. The keystore and truststore will be generated dynamically and will be used for zookeeper-kafka and kafka-client communication.