You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
OpenMetrics 2.0: option to keep _total and unit suffixes, so switching from OM1 doesn't rename series #2518
This means OpenMetrics 2.0 does not apply OM1 suffix behavior such as appending `_total` or unit
suffixes, while the legacy OpenMetrics 1.0 and Prometheus text formats keep that existing suffix
behavior.
, e.g. Counter("req").unit(BYTES) is req_bytes_total in OM1 but req in OM2.
The problem is that switching a target from OM1 to OM2 renames every counter, and every metric with a unit, that was instrumented the way client_java has always recommended (Counter.builder().name("events")). Prometheus then stores them as new series, so existing queries, dashboards and alerts silently stop matching.
Note
I hit this with a demo that scrapes the same client_java app with OM1 and OM2 and diffs the stored series: the JVM metrics match (they already have the suffixes in their names), but http_requests_total (OM1) became http_requests (OM2), and http_request_size_bytes_total became http_request_size. Demo: https://35-204-166-191.sslip.io/, code: bwplotka/prometheus#7.
add a property, e.g. io.prometheus.openmetrics2.suffixes (or a naming mode), that keeps the OM1 suffix behaviour in the OM2 writer: append _total to counters and the unit suffix where missing, as expositionBaseName already does for OM1.
I'd argue it should default to on, so that negotiating OM2 doesn't change any series names, and users who want names exactly as written (e.g. for OTel-style names) can opt out.
For comparison, client_golang keeps the names users register (which by convention include _total and the unit), so there OM1 and OM2 produce the same series.
With
io.prometheus.openmetrics2.enabled, the OM2 writer exposes names exactly as passed to the builder. It never appends_totalor the unit suffix:client_java/prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/OpenMetrics2TextFormatWriter.java
Lines 174 to 179 in 9e9deb6
This is documented in
client_java/docs/content/exporters/openmetrics2.md
Lines 57 to 77 in 9e9deb6
Counter("req").unit(BYTES)isreq_bytes_totalin OM1 butreqin OM2.The problem is that switching a target from OM1 to OM2 renames every counter, and every metric with a unit, that was instrumented the way client_java has always recommended (
Counter.builder().name("events")). Prometheus then stores them as new series, so existing queries, dashboards and alerts silently stop matching.Note
I hit this with a demo that scrapes the same client_java app with OM1 and OM2 and diffs the stored series: the JVM metrics match (they already have the suffixes in their names), but
http_requests_total(OM1) becamehttp_requests(OM2), andhttp_request_size_bytes_totalbecamehttp_request_size. Demo: https://35-204-166-191.sslip.io/, code: bwplotka/prometheus#7.The OM2 spec relaxed
_totaland the unit suffix from MUST to SHOULD, mainly for OpenTelemetry compatibility (https://github.com/prometheus/docs/blob/605cf81fefc2e8e91f8ba89bb1555ae52a43a318/docs/guides/open_metrics_2_0_migration.md?plain=1#L140). Both are still recommended though (https://github.com/prometheus/docs/blob/605cf81fefc2e8e91f8ba89bb1555ae52a43a318/docs/specs/om/open_metrics_spec_2_0.md?plain=1#L220 and https://github.com/prometheus/docs/blob/605cf81fefc2e8e91f8ba89bb1555ae52a43a318/docs/specs/om/open_metrics_spec_2_0.md?plain=1#L664). Today there is no way to get them in OM2 other than renaming every metric in code;OpenMetrics2Propertieshas no option for it.Proposal:
io.prometheus.openmetrics2.suffixes(or a naming mode), that keeps the OM1 suffix behaviour in the OM2 writer: append_totalto counters and the unit suffix where missing, asexpositionBaseNamealready does for OM1.For comparison, client_golang keeps the names users register (which by convention include
_totaland the unit), so there OM1 and OM2 produce the same series.