First of all you need to confirm that you have a Datadog endpoint configured and enabled on the Kafka instance, instructions on how to do that can be found from the Getting started with Datadog article.
After installing the Aiven command-line client and having the integration enabled, you can list the enabled integrations with
avn service integration-list <myservice>. You should have one integration with the integration type
datadog, you need the service integration ID of that line.
For example to enable sending the
kafka.log.log_size metric to Datadog, you can use
avn service integration-update -c kafka_custom_metrics=kafka.log.log_size <integration-id>. You can also send a combination of custom metrics by separating them with a comma:
avn service integration-update -c kafka_custom_metrics=kafka.log.log_size,kafka.log.log_end_offset <integration-id>
The following custom metrics are currently supported:
Finally, after the update goes through and Kafka has started collecting some logs you should be able to see the data on your Datadog explorer:
Each of these metrics are also tagged with
partition to be able to monitor each Kafka topic and partition independent of each other.
Got here by accident? Learn how Aiven simplifies working with Apache Kafka: