Aiven Kafka supports Access Control Lists for allowing and limiting Produce and Consumer rights on the topic level for your users. You can manage both users and ACL entries on the Users pane on the Service Details page on the Aiven Console.
Important Note: By default, an admin account with wildcard (*) permissions is added to every new service. When you create your own ACLs to restrict access, please remove this account.
Terraform Note: When using the Aiven Terraform Provider, you can add the default_acl
key to your resource
and set it to false
if you would not like the wildcard admin user to be created.
Users can be added by clicking the Add Service User button on the top right corner.
Each user has their individual Access Key and Certificate that can be downloaded from the Users pane.
ACLs are defined as an user or a wildcard mask of users, the grant to produce and/or consume and finally a topic or a wildcard mask of topics that the grant is applied to. By default, the access is allowed for all configured users to both produce and consume on all topics.
You can add new grants using the Add an ACL entry... button on the right.
Since the rules are additive, you probably want to delete the default rule once you start using Access Control Lists. To do so, you can use the Remove button next to each rule.
Please note the ACL restrictions do not currently apply to Kafka REST. We're working on extending the same restrictions there.
Permission Mapping
Admin
Read
Write
CreateTopics
Describe
Describe_Configs
Alter
AlterConfigs
Delete
Important Note: When giving a user the admin
permission, they will be able to create a topic with any name as the CreateTopics
permission is applied at the cluster level. All other permissions related to a topic (alter, delete) will only apply to the topics matching the pattern specified.
Consume and Produce
Write
Read
Describe
Describe_Configs
Produce
Write
Describe
Describe_Configs
Consume
Read
Describe
Describe_Configs
These mappings are subject to change and this article will be updated when that happens.
For Kafka, by default, number of users per service is limited to 50. In case this needs to be adjusted, don't hesitate to contact us.
Monitoring and Alerting on Logs for Denied ACL Checks
You can use the following logs patterns to setup alerts on failed authentication and ACL evaluation.
Failed Producer
HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:35:33,509] INFO [DENY] Auth request Write on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)
Failed Consumer
HOSTNAME: kafka-pi-3141592-74
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:43:09,712] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)
Valid Cert with Invalid Key
HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:54:10,781] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by Invalid CN=delete-user,OU=u6l6y9h1,O=kafka-pi-3141592 (io.aiven.kafka.auth.AivenAclAuthorizer)
Got here by accident? Learn how Aiven simplifies working with Apache Kafka: