Querying Aiven for Apache Kafka with Lenses.io

Get started with Lenses. Set it up with Docker and connect to your Aiven-managed Kafka with the standard configuration.

1. Get your Lenses access key

Sign up with Lenses and download your access key.

2. Sign up for Aiven and create a Kafka

Aiven for Apache Kafka is a fully managed streaming platform, deployable in the cloud of your choice.

Start by signing up for an Aiven account . Follow the in-product instructions to launch Kafka clusters within the Aiven user interface.

Then, enable the additional Kafka features to get the most of it.

Enable Schema Registry

Note: As of version 3.0, Aiven for Apache Kafka no longer supports Confluent Schema Registry. For more information, see this article that describes the replacement, Karapace.

Enable Kafka Connect

Add Jolokia integration to enable JMX Metrics (click Service Integrations in the left menu)

Gather 4 connection points

1. Kafka endpoint

2. Schema Registry endpoint

3. Kafka Connect endpoint

4. Jolokia endpoint

Download 3 Aiven certificates

3. Get Docker

You will use docker-compose to spin up Lenses.

Start by signing up with Docker . Then install docker-compose .

4. Setup Lenses docker-compose

Use this docker-compose.yml file to spin up Lenses and connect to your Aiven for Apache Kafka.

Just 2 steps:

1. Fill in the connection details:

- Kafka endpoint

- Schema Registry

- Kafka Connect

- Jolokia endpoint

2. Share the authentication files with your docker. Copy these files in the same directory as your docker-compose.yml file and reference them:Your Lenses license key license.jsonThe 3 Aiven certificate files

The file:

version: '3'
services:
  lenses:
    image: lensesio/lenses
    environment:
      # Admin user credentials
      LENSES_SECURITY_USER: admin
      LENSES_SECURITY_PASSWORD: admin
      LENSES_PORT: 80
      
      # === Set your values in this section ===
      
      # ===Step 1=== Put your Aiven for Apache Kafka 'Service URI' here
      # e.g. 
      # LENSES_KAFKA_BROKERS: SSL://lenses-aiven-kafka-lenses-56f3.aivencloud.com:23381
      # Kafka Endpoint
      LENSES_KAFKA_BROKERS: SSL://[AIVEN_HOST]:[AIVEN_PORT]
      # Aiven SSL configuration
      LENSES_KAFKA_SETTINGS_CLIENT_SECURITY_PROTOCOL: SSL
      
      # ===Step 2=== Put the Aiven Schema Registry endpoint here
      # Aiven Schema Registry (Karapace)
      LENSES_SCHEMA_REGISTRY_URLS: '[{url: "https://[REGISTRY_HOST]:[REGISTRY_PORT]"}]'
      LENSES_SCHEMA_REGISTRY_AUTH: USER_INFO
      LENSES_SCHEMA_REGISTRY_USERNAME: [USER]
      LENSES_SCHEMA_REGISTRY_PASSWORD: [PASSWORD]
      LENSES_KAFKA_SETTINGS_CLIENT_BASIC_AUTH_CREDENTIALS_SOURCE: USER_INFO
      LENSES_KAFKA_SETTINGS_CLIENT_BASIC_AUTH_USER_INFO: [USER]:[PASSWORD]
      # ===Step 3=== Put the Kafka Connect endpoint here
      # Kafka Connect
      LENSES_CONNECT_CLUSTERS: |
        [{
          name:"aiven_connect",
          urls: [{url:"https://[CONNECT_HOST]:[CONNECT_PORT]"}],
          auth = "USER_INFO",
          username = "[USER]",
          password = "[PASSWORD]",
          statuses:"__connect_status",
          configs:"__connect_configs",
          offsets:"__connect_offsets"
        }]
      # ===Step 4=== Put your Aiven Jolokia REST endpoint here
      # Kafka JMX metrics with Aiven Jolokia
      LENSES_KAFKA_METRICS: |
        {
          type: "JOLOKIAP",
          default.port: "[CLIENT_PORT]"
          user: "[BASIC_AUTH_USERNAME]",
          password: "[BASIC_AUTH_PASSWORD]",
          ssl: true
        }
      
    ports:
      - 80:80
    volumes:
      # ===Step 5=== Share your authentication files here
      # Map the Lenses license key file here to activate Lenses
      - ./license.json:/license.json
      # Map the Aiven files here to enable Lenses to communicate with Aiven via SSL
      - ./service.key:/mnt/secrets/FILECONTENT_SSL_KEY_PEM
      - ./service.cert:/mnt/secrets/FILECONTENT_SSL_CERT_PEM
      - ./ca.pem:/mnt/secrets/FILECONTENT_SSL_CACERT_PEM

4. Start Lenses up

docker-compose up

5. Login

User:     admin
Password: admin

6. Eureka!

7. Next steps

Learn the basics of Lenses.io.