Kafdrop is a popular Web UI for Kafka that allows you to monitor your cluster, view topics and consumer groups with integration into Schema Registry, with support for Avro, JSON and Protobuf!
Note: As of version 3.0, Aiven for Apache Kafka no longer supports Confluent Schema Registry. For more information, see this article that describes the replacement, Karapace.
The second version has been
hard-forked into Kafdrop 3 that has significant changes; most notable is that it does not require access to Zookeeper in order to run.
This means it is easy for you to monitor your Aiven for Apache Kafka cluster without having a terminal open or squinting at hundreds of lines of plain text output.
Aiven for Apache Kafka Connection
Kafdrop has support for SASL and SSL, in this article we will be using SSL to connect to our cluster. In the details page for your cluster, you will need to download:
- Access Key
- Access Certificate
- CA Certificate
and make a note of your:
- Service URI
You did not think you would get away with zero command line effort, did you?
Setting up your keystores and truststores
To create your keystore (assuming the Access Key and Certificate are called
service.cert respectively) you can run:
openssl pkcs12 -export -inkey service.key -in service.cert -out kafka.keystore.p12 -name demo_kafka_key
And for the truststore, you can run:
keytool -import -file ca.pem -alias KafkaCA -keystore kafka-client.truststore.jks
Make a note of the passwords set and we will create a
kafka.properties file that looks like this:
You can also add:
if you set a password when creating the truststore.
The easiest method to run Kafdrop is using Docker/Podman and we can do this with a one-liner:
docker run -p 9000:9000 -e KAFKA_BROKERCONNECT=$SERVICEURI -e KAFKA_PROPERTIES="$(cat kafka.properties | base64)" -e KAFKA_TRUSTSTORE="$(cat kafka-client.truststore.jks | base64)" -e KAFKA_KEYSTORE="$(cat kafka.keystore.p12 | base64)" obsidiandynamics/kafdrop
We do not put the location of the files in
kafka.properties as we pass the content directly here, as well as the URI of your broker.
If you want to use Schema Registry to deserialize avro messages in Kafdrop then you will need to configure it with two additional environment variables:
You should see a lot of output as Kafdrop starts up (if you see any SSL errors, make sure the port is right, the supplied password is right and that the names of your stores are correct and in this directory).
Open a browser window and point to localhost:9000 where you will now see your cluster reporting!
The GIF above shows a basic click through of the UI but let's list some of what you can do here:
* View and Search Topics
* Create and Delete Topics
* View Brokers
* View Messages
All without having to open a terminal. A nice way to see an overview of your cluster but a powerful tool to inspect topics and messages in great detail. For more information, you can check out the repository here.
Having troubles setting Kafdrop up or other issues with your Aiven for Apache Kafka? Drop us a line at firstname.lastname@example.org or using the chat widget on the right.