Fortunately Aiven takes care of the upgrade for you, but you may notice that immediately following an upgrade these values remain unchanged.
Consider a topic created on a service running Kafka
When you upgrade the service to a new version of Kafka, new nodes join the cluster and topics are synced from the old nodes until they can be removed entirely. For the duration of the upgrade the cluster contains a mixture of nodes running Kafka
To maintain compatibility, Aiven ensures the
log.message.format.version values are set to the minimum Kafka version present in the cluster.
This means that following the upgrade these configuration values remain equal to the previous version of Kafka running in the cluster. Although it would be safe to increase the
inter.broker.protocol.version after the upgrade, it is not safe to increase the
log.message.format.version because there will be old messages in the old format on disk.
This is not usually a problem thanks to Kafka's bidirectional client compatibility unless you are looking to take advantage of new features allowed by a higher
log.message.format.version. The solution here is another rolling update which is usually triggered by applying maintenance updates, or upgrading or downgrading the plan.
Therefore when you trigger a maintenance update/upgrade after completing a major version upgrade, then the new nodes that join the cluster only contain the nodes running Kafka
The minimum Kafka version present is now also
2.6 and the
log.message.format.version values are increased. This means that the messages of topics are finally created on the new nodes with the latest message format.
If you have any questions, please feel free to reach out to our Support and let us know.