First of all you need to confirm that you have a Datadog endpoint configured and enabled on the Kafka instance, instructions on how to do that can be found from the Getting started with Datadog article.
After installing the Aiven command-line client and having the integration enabled, you can list the enabled integrations with
avn service integration-list <myservice>. You should have one integration with the integration type
datadog, you need the service integration ID of that line.
For example to enable sending the
kafka.log.log_size metric to Datadog, you can use
avn service integration-update -c kafka_custom_metrics=kafka.log.log_size <integration-id>. You can also send a combination of custom metrics by separating them with a comma:
avn service integration-update -c kafka_custom_metrics=kafka.log.log_size,kafka.log.log_end_offset <integration-id>
The following custom metrics are currently supported:
Finally, after the update goes through and Kafka has started collecting some logs you should be able to see the data on your Datadog explorer:
Each of these metrics are also tagged with
partition to be able to monitor each Kafka topic and partition independent of each other.