Based on the Datadog Kafka integration documentation you would expect to see a full dashboard of metrics after enabling this service integration for your Aiven for Kafka service.

The service integration does indeed support sending all broker-side metrics to Datadog, and even allows for sending custom metrics too.

It is important to note, however, that all the metrics prefixed with kafka.producer.* or kafka.consumer.* are client-side metrics and would actually require a connection from the Datadog agent to the Java producer or consumer processes.

This setup (where the Datadog agent running on the Kafka nodes polls the application code processes) would make sense in an environment where you are hosting Kafka yourself and your producers and consumers were available at specific IP addresses. But for Kafka running in a managed service it does not make sense from a security perspective for the Datadog agent to reach out to the client processes to poll for metrics.

In short, Aiven does not support gathering client-side metrics with the service integration. If you would like to send these additional metrics to Datadog then you should send them directly using DogStatsD as recommended in the Datadog documentation.

Did this answer your question?