Aiven for Apache Kafka supports access control lists (ACL) for allowing and limiting producer and consumer rights on the topic level for your users. You can manage users and ACL entries in the corresponding tabs of the service page in the Aiven web console.

Important: By default, Aiven adds an Admin account with wildcard (*) permissions to every new service. When you create your own ACLs to restrict access, remove this account.

Note: When using the Aiven Terraform Provider, you can add the default_acl key to your resource and set it to false if you do not want to create the admin user with wildcard permissions.

ACLs are defined as a user or a wildcard mask of users, the grant to produce and/or consume, and finally a topic or a wildcard mask of topics that the grant is applied to. By default, access is allowed for all configured users to both produce and consume on all topics.

To add a new user:

  1. Log in to the Aiven web console and select your service.

  2. Click the Users tab.

  3. Enter a name for the new user and then click Add service user.

    The new user appears on the Users page, with links to the user-specific access key and certificate.

  4. To add a new grant:

    1. Click the ACL tab.

    2. Enter the username and topic that the grant applies to.

    3. Select the Permission that you want to apply.
      Details for the permissions of each option are described later in this article.

    4. Click Add ACL entry.

  5. Once you start using ACLs, click the Remove icon for the default rule to delete it.
    This is not mandatory, but it is recommended, since the rules are additive.

Note: ACL restrictions currently do not apply to Kafka REST, but we are working on extending the same restrictions there. Rules are applied based on the username and topic names, but there are no restrictions on consumer group names.

Permission Mapping

Admin

For the cluster:

  • CreateTopics

For consumer groups:

  • Delete

  • Describe

  • Read

For topics:

  • Read

  • Write

  • Describe

  • Describe_Configs

  • Alter

  • AlterConfigs

  • Delete

For transactions:

  • Describe

  • Write

Important: When you give a user the admin permission, they can create a topic with any name, as the CreateTopics permission is applied at the cluster level. All other permissions related to a topic (alter, delete) only apply to the topics matching the pattern that you specify.

Consume and Produce

For consumer groups:

  • Delete

  • Describe

  • Read

For topics:

  • Write

  • Read

  • Describe

  • DescribeConfigs

For transactions:

  • Describe

  • Write

Produce

For topics:

  • Write

  • Describe

  • DescribeConfigs

For transactions:

  • Describe

  • Write

Consume

For consumer groups:

  • Delete

  • Describe

  • Read

For topics:

  • Read

  • Describe

  • Describe_Configs

These mappings are subject to change and this article will be updated when that happens.

By default, the number of users per service is limited to 50 in Kafka. Contact Aiven support if you need more users.

Monitoring and alerting on logs for denied ACL checks

You can use the following log patterns to set up alerts for failed authentication and ACL evaluation.

Failed producer

HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:35:33,509] INFO [DENY] Auth request Write on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)

Failed consumer

HOSTNAME: kafka-pi-3141592-74
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:43:09,712] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by User test-kuser (io.aiven.kafka.auth.AivenAclAuthorizer)

Valid certificate with invalid key

HOSTNAME: kafka-pi-3141592-75
SYSTEMD_UNIT: kafka.service
MESSAGE: [2020-09-04 06:54:10,781] INFO [DENY] Auth request Describe on Topic:nodejs-quickstart-kafka-topic by Invalid CN=delete-user,OU=u6l6y9h1,O=kafka-pi-3141592 (io.aiven.kafka.auth.AivenAclAuthorizer)

Learn how Aiven simplifies working with Apache Kafka:

Did this answer your question?