[feat][fn] Expose the Prometheus metrics registry to Go functions via FunctionContext - #26458
Open
kritharth2005 wants to merge 1 commit into
Open
[feat][fn] Expose the Prometheus metrics registry to Go functions via FunctionContext#26458kritharth2005 wants to merge 1 commit into
kritharth2005 wants to merge 1 commit into
Conversation
… FunctionContext Go functions have exactly one way to emit a custom metric: FunctionContext.RecordMetric(name, value), which funnels every value into a single fixed-shape SummaryVec (pulsar_function_user_metric). There's no way to register a Counter, Gauge, Histogram, or any other prometheus.Collector, and the registry the SDK already runs and serves on the metrics port has no exported accessor. This adds FunctionContext.GetMetricsRegistry() prometheus.Registerer, returning the existing package-level registry typed as the Registerer interface (Register/MustRegister/Unregister, not Gather), so a function can register its own collectors alongside the SDK's. RecordMetric and userMetricSummary are unchanged; typed convenience helpers (Option B from the issue) are left as a follow-on. Covered by two new tests in stats_test.go: registering and scraping a custom Counter, and the collision behavior on a name colliding with a built-in metric (Register errors, MustRegister panics). Fixes apache#26403
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #26403
Motivation
The Go Functions SDK gives user code exactly one way to emit a custom metric:
FunctionContext.RecordMetric(name, value), which funnels every value into asingle fixed-shape
SummaryVec(pulsar_function_user_metric, with quantileobjectives 0.5/0.9/0.99/0.999 baked in at
pulsar-function-go/pf/stats.go). AGo function cannot register a plain counter, a gauge, a histogram with its own
buckets, or any other
prometheus.Collector.The SDK already runs a Prometheus registry (
reg, an unexported*prometheus.Registryinpf/stats.go) and serves it over the function'smetrics port via
NewMetricsServicer. There is currently no exported accessorfor it, so user code outside package
pfhas no way to reach it.Python functions get this for free: the Python SDK relies on
prometheus_client's process-global registry, so a Python function can justinstantiate a
prometheus_client.Counter(...)and have it scraped on the sameendpoint. This PR brings the Go SDK to parity by exposing the registry it
already maintains.
Modifications
pulsar-function-go/pf/context.go: addedfunc (c *FunctionContext) GetMetricsRegistry() prometheus.Registerer,which returns the package-level
reg. Placed immediately afterRecordMetric(and next toGetMetricsPort) so the metrics-relatedmethods stay grouped. The return type is the
prometheus.Registererinterface rather than the concrete
*prometheus.Registry: callers getRegister/MustRegister/Unregisterbut notGather, keeping thescrape/collection path internal to the SDK. The doc comment states that
pulsar_function_-prefixed names are reserved for SDK metrics and that acolliding fully-qualified name fails registration (
Registerreturns anerror,
MustRegisterpanics).RecordMetricand theuserMetricSummarydefinition are left unchanged.pulsar-function-go/pf/stats_test.go: added two tests next to theexisting
TestMetricsServer/TestUserMetrics/TestInstanceControlMetrics:TestGetMetricsRegistry_CustomCollector— starts agoInstanceand itsMetricsServicer, registers aprometheus.Counter(a shape theuser_metricSummary cannot express) throughgi.context.GetMetricsRegistry().Register(...), adds42, scrapes/metricsonGetMetricsPort(), and asserts the response body containspulsar_function_go_test_custom_counter_total 42. The collector isunregistered on cleanup so the process-global
regis not left mutated.TestGetMetricsRegistry_NameCollision— registers a counter namedpulsar_function_received_total(colliding with a built-in SDK metric)and asserts
Register()returns a non-nil error, then assertsMustRegister()panics on the same collision.Design note (open to review) — prefix-collision handling
The issue thread raised whether the SDK should actively reject user
collectors that use the reserved
pulsar_function_prefix (e.g. a wrappertype around the registry). This PR intentionally does not do that: it
documents the reserved prefix in the method's doc comment and relies on
prometheus.Registerer.Registeralready returning a non-panicking errorwhen a fully-qualified name collides with an existing metric — which is the
reason the method returns the
Registererinterface. This keeps thesurface minimal and matches how the underlying library behaves. If
reviewers would prefer a stricter guard, that can be added.
Out of scope
Option B from the issue — typed convenience helpers such as
NewUserCounter/
NewUserGauge— is intentionally not included here. Exposing the registryis the smaller, lower-risk change; the typed helpers can follow in a separate
PR once this lands.
Verifying this change
This change added tests and can be verified as follows:
TestGetMetricsRegistry_CustomCollectorregisters a custom counter andconfirms it is exposed on the function's
/metricsendpoint.TestGetMetricsRegistry_NameCollisionconfirms a name colliding with abuilt-in SDK metric is rejected by
Register()(error, no panic) andpanics under
MustRegister().Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
FunctionContext.GetMetricsRegistry()to the GoFunctions SDK. Additive only; no existing signatures change.
collectors, which are scraped on the existing function metrics endpoint.