Maintenance: Remove dependency on Kafka internal ByteUtils for kafka-clients 4.4.0 #2710

Summary

kafka-clients 4.4.0 moved org.apache.kafka.common.utils.ByteUtils into the internal package org.apache.kafka.common.utils.internals (KAFKA-20297, apache/kafka#21856). KafkaProtobufDeserializer imports ByteUtils to call readVarint, so powertools-kafka no longer compiles against kafka-clients 4.4.0. This blocks Dependabot PR #2704.

Kafka has no public replacement. KIP-1247 makes only Bytes and Time public.

Why is this needed?

• powertools-kafka cannot move to kafka-clients 4.4.0 or later until it stops using ByteUtils.
• Depending on Kafka internal API can break the build on any kafka-clients release.

There is no customer-facing behavior change.

Which area does this relate to?

Other

Solution

Replace ByteUtils.readVarint with a local package-private readVarint(ByteBuffer) in KafkaProtobufDeserializer. It reads an unsigned varint of at most 5 bytes and applies zigzag decoding. It matches Kafka's previous behavior: same values, IllegalArgumentException after 5 bytes, and BufferUnderflowException on truncated input.

Protobuf's CodedInputStream.readSInt32() is not a drop-in replacement because it accepts varints of up to 10 bytes.

Follow-up: examples/powertools-examples-kafka/tools/.../GenerateProtobufSamples.java also imports ByteUtils. It pins kafka-clients 4.0.2, so it builds today, but it will break on 4.4.0 or later.

Acknowledgment

• This request meets Powertools for AWS Lambda (Java) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, TypeScript