A long time ago, it was uncommon to see padlocks and `https` all over your browser. These times have changed now the Internet is not mostly made up of personal pages, blogs and MySpace pages.
We use the Internet daily to send sensitive information, whether it is an email, a financial transaction or a mean message about a coworker. Sending this data in
PLAINTEXT is the old way (and a method that Kafka also still supports).
Kafka has evolved with the times and now has support for more than just unencrypted credentials. In this article, we will look at the more secure options that Aiven uses in our Kafka offering:
TLS (aka SSL)
Transport Layer Security (TLS, aka SSL - Secure Sockets Layer) is now moving to be the de facto for Internet traffic and involves a certificate for your domain to be provided by a Certificate Authority (we love, letsencrypt.org for this). With this certificate, and the right technical setup, you will be able to utilise
https://mykafkabroker.mydomain.com and traffic to your service will be encrypted.
Are you wondering how to get this set up for your Kafka service? Wonder no more, Aiven does this by default for all Kafka services and we handle the certificate application, renewal and configuration!
This is already a great start but TLS can handle the encryption and authentication but there are 2 ways it can be used with Kafka:
TLS Encryption - This is a 1 way street where your Kafka client validates the certificate for your Kafka broker.
TLS Authentication - This is a 2 way street where your Kafka client validates the certificate for your Kafka broker AND your Kafka broker validates the certificate for your Kafka client.
SASL (Simple Authentication and Security Layer)
SASL acts as a pluggable layer to use alternative login methods for your service. For example, if you use Active Directory for authentication, SASL supports a Kerberos login method that can provide access.
In this article, we will focus on SASL/PLAIN and SASL/SCRAM.
This method is the easiest to enable and handles a simple
username/password login method over a TLS connection. What this means is that your traffic is encrypted, if you use SASL/PLAIN without TLS then your credentials will be sent for anyone to read.
SCRAM stands for Salted Challenge Response Authentication Mechanism. It is a mechanism which allows a client to identify itself to a server without sending a plaintext password. A key benefit is that it will not reveal the password to servers which do not already have it, for example if a client connects to the wrong server even if that server has a valid TLS certificate.
A brief explanation of this is that a random
salt is created, which is then used to create an
Identity that holds:
The number of iterations to use (default is 4096)
StoredKey(the hash of the Client's Key)
Identity is then stored in Zookeeper by default.
Enabling in Aiven
Navigate to your Kafka service and scroll down to
Advanced Configuration, there you can select
kafka.authentication_methods.sasl to enable.
Connecting from your clients can be tough so we have some example code in this help article to get you started.