NIFI-16375 Add Lag and Record Metrics to ConsumeKafka

[exceptionfactory]

Summary

NIFI-16375 Adds the following Counter and Gauge metrics to the ConsumeKafka Processor:

• kafka.bytes.consumed
  • Type: Counter
• kafka.records.consumed
  • Type: Counter
• kafka.records.consumer.lag
  • Type: Gauge
• kafka.records.parsed.errors
  • Type: Counter

This metrics include the kafka.topic and kafka.partition values in the recently-added Map of attributes for each Counter and Gauge recorded.

The implementation uses the polled results to determine the Topics and Partitions that will be recorded.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
• Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21
  • JDK 25

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for the review @pvillard31, I pushed an update refactoring the approach to capture the original byte length from the Kafka Records

[pvillard31]

Latest LGTM, will merge if all checks are green