Kafka includes a kafka-consumer-groups.sh tool that allows you to view and manipulate the state of consumer groups. This article shows you how to set up the tool, use it, and reset the offset for a consumer group.

Setting up kafka-consumer-groups.sh

  1. Create the Java keystore and truststore for your Aiven for Apache Kafka service.
    For details, see this article.

  2. Create a properties file that points to the keystore and truststore files.
    For example:

    security.protocol=SSL
    ssl.keystore.type=PKCS12
    ssl.keystore.location=client.keystore.p12
    ssl.keystore.password=changeit
    ssl.key.password=changeit
    ssl.truststore.location=client.truststore.jks
    ssl.truststore.password=changeit

Using kafka-consumer-groups.sh

  • To list the currently active groups, run the following command:

    kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.htn-aiven-demo.aivencloud.com:17072 \
    --command-config consumer.properties \
    --list

  • To view the details for a single group, run the following command:

    kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.htn-aiven-demo.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group test-group \
    --describe

  • To list the current members of a group, run the following command:

    kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.htn-aiven-demo.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group test-group \
    --describe \
    --members

Resetting the offset for a consumer group

Note: The consumer group must be inactive when you make offset changes.

Run the following command:

kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.htn-aiven-demo.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group test-group \
    --topic test-topic \
    --reset-offsets \
    --to-earliest \
    --execute

Options:

  • --to-earliest: Reset to the beginning of the data.

  • --to-lastest: Reset to the end of the topic.

  • --to-offset: Reset to a known, fixed offset.

  • --shift-by: Perform relative changes using the given integer. Use positive values to skip forward or negative values to move backwards.

  • --to-datetime <YYYY-MM-DDTHH:mm:SS.sss>: Reset to the given timestamp.

  • --topic <topicname>:<partition>: Apply the change to a specific partition, for example --topic test-topic:0. By default, the --topic argument applies to all partitions.

Learn how Aiven simplifies working with Apache Kafka:

Did this answer your question?