By default, in the config/hosts file, Kafka and Zookeeper are configured to be installed as part of the deployment framework: 
# this option can be used if there is an existing kafka installation that can be used
# set it to false if kafka need not be installed
install_kafka=true
  • To disable the Kafka and Zookeeper installation, set install_kafka=false.
  • Edit the parameters in config/kafka-defaults.yml to change the default installation settings for Kafka and Zookeeper.
    kafka:
 # Define ports for the Kafka brokers
 # These ports remain same for all brokers
 ssl_port: 9094
 #Port to be used for Communication with abs and dataengine
 sasl_port: 9093
 
 #kafka jks password
 jks_password: changeme
 
 
 #Enable Delete topics in kafka
 delete_topic: false
 
 ssl_key_alias: pingidentity
 wait_time_before_clean: 30
 startup_timeout: 120
 
zookeeper:
 
   # Define ports for the zookeeper brokers
   # These ports remain same for all zookeeper
   ssl_port: 2182
    Kafka variables
    Variable Description
    ssl_port SSL port for Kafka.

    default: 9094
    sasl_port SASL port that is also used by the data engine and ABS to communicate with Kafka

    default: 9093
    jks_password JKS password.

    If a custom truststore and keystore is provided, you can configure the password here.
    delete_topic Enables topic deletion in Kafka.
    wait_time_before_clean Waiting time before cleaning existing data in Kafka, for new installation.
    startup_timeout Waiting time for Kafka to start.
    Zookeeper variables
    ssl_port SSL port for Zookeeper, also used for communication to Kafka via Kafka's SSL port..

    default :9093

Changing default Kafka configurations in config/hosts

The config/hosts file has a Kafka configuration, for example:
[kafka]
172.16.40.81 zookeeper_id=1
You can install Kafka and Zookeeper as a cluster by providing multiple IPs or hosts in the host file, and zookeeper_id according to the number of nodes to install.
Note:
  • zookeeper_id should start from 1.
  • Each new node's zookeeper_id should increase by 1.
  • The nodes should be listed in ascending zookeeper_id order.
  • The number of nodes should only be either one or three, for example:
    • One node:
      [kafka]
<IP address> zookeeper_id=1
    • Three nodes:
      [kafka]
<IP address 1> zookeeper_id=1
<IP address 2> zookeeper_id=2
<IP address 3> zookeeper_id=3
You can provide custom crt and key files in the certs folder location:
kafka_private_key_location: "{{ cert_dir }}/kafka/kafka.key"
kafka_cert_location: "{{ cert_dir }}/kafka/kafka.crt"

Configure Kafka keystore password

You can configure the keystore password in config/kafka-defaults.yml. The keystore and truststore will be generated dynamically and will be used for zookeeper-kafka and kafka-client communication.