Setup and configure Zookeper and Kafka on Linux

by

We would need java runtime environment to install and operate the kafka and zookeeper programs on our linux environment as a dependency which uses the java runtime environment.
So lets quickly check the current java version on the system with java -version.
If its not present Lets now begin with the setup of Java and quickly download the java stable version from the epel repository.

# yum install java-1.8.0-openjdk

Once the java version is installed we verify it with the java -version command.
Creating the kafka user account.

# useradd kafka -m -d /opt/kafka

Adding the kafka password

# echo "kafka" | passwd kafka --stdin
usermod -aG wheel kafka
# su -l kafka

Now login to the system as kafka user.

# cd /opt

Navigate to the url : https://downloads.apache.org/kafka/ and download the latest kafka version

# curl https://downloads.apache.org/kafka/2.5.0/kafka_2.13-2.5.0.tgz -o /opt/kafka.tgz
# tar -xvzf ~/opt/kafka.tgz --strip 1
# cd /opt/kafka
# cp -a /opt/kafka/config/server.properties ~/opt/kafka/config/server.properties.bkp
# echo -e "\ndelete.topic.enable = true\nauto.create.topics.enable = true" >> /opt/kafka/config/server.properties

Adding the Kafka start/stop scripts to systemctl controller daemon services

# sudo vim /etc/systemd/system/zookeeper.service
Add the following lines to /etc/systemd/system/zookeeper.service
[Unit]
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
User=kafka
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

Now adding the Kafka control scripts to systemd service

# sudo vim /etc/systemd/system/kafka.service
Add the following lines to /etc/systemd/system/kafka.service
[Unit]
Requires=zookeeper.service
After=zookeeper.service

[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties > /opt/kafka/kafka.log 2>&1'
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target
# sudo chown kafka.kafka -R /opt/kafka/logs
# sudo mkdir /var/log/kafka-logs
# sudo chown kafka.kafka -R /var/log/kafka-logs

Now start the zookeeper and kafka services as follows:

# sudo systemctl enable zookeeper --now

# sudo systemctl enable kafka --now

Leave a Comment