Kafka Users and Access Control

Aiven Kafka supports Access Control Lists for allowing and limiting Produce and Consumer rights on the topic level for your users. You can manage both users and ACL entries on the Users pane on the Service Details page on the Aiven Console.

Important Note: By default, an admin account with wildcard (*) permissions is added to every new service. When you create your own ACLs to restrict access, please remove this account.

Terraform Note: When using the Aiven Terraform Provider, you can add the default_acl key to your resource and set it to false if you would not like the wildcard admin user to be created.

Users can be added by clicking the Add Service User button on the top right corner.

Each user has their individual Access Key and Certificate that can be downloaded from the Users pane.

ACLs are defined as an user or a wildcard mask of users, the grant to produce and/or consume and finally a topic or a wildcard mask of topics that the grant is applied to. By default, the access is allowed for all configured users to both produce and consume on all topics.

You can add new grants using the Add an ACL entry... button on the right.

Since the rules are additive, you probably want to delete the default rule once you start using Access Control Lists. To do so, you can use the Remove button next to each rule.

Please note the ACL restrictions do not currently apply to Kafka REST. We're working on extending the same restrictions there.

Permission Mapping

Admin

• Read
• Write
• CreateTopics
• Describe
• Describe_Configs
• Alter
• AlterConfigs
• Delete

Important Note: When giving a user the admin permission, they will be able to create a topic with any name as the CreateTopics permission is applied at the cluster level. All other permissions related to a topic (alter, delete) will only apply to the topics matching the pattern specified.

Consume and Produce

• Write
• Read
• Describe
• Describe_Configs

Produce

• Write
• Describe
• Describe_Configs

Consume

• Read
• Describe
• Describe_Configs

These mappings are subject to change and this article will be updated when that happens.

For Kafka, by default, number of users per service is limited to 50. In case this needs to be adjusted, don't hesitate to contact us.

Monitoring and Alerting on Logs for Denied ACL Checks

You can use the following logs patterns to setup alerts on failed authentication and ACL evaluation.

Failed Producer

HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:35:33,509] INFO [DENY] Auth request Write on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)

Failed Consumer

HOSTNAME: kafka-pi-3141592-74
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:43:09,712] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)

Valid Cert with Invalid Key

HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:54:10,781] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by Invalid CN=delete-user,OU=u6l6y9h1,O=kafka-pi-3141592 (io.aiven.kafka.auth.AivenAclAuthorizer)

Got here by accident? Learn how Aiven simplifies working with Apache Kafka:

• Managed Kafka as a Service