[Proposal] Add log observability and metric-trace-log correlation

[dodjdnh]

Background

Dubbo Admin already provides application, instance, and service information, together with metric and trace Grafana dashboards. However, log observability is not currently exposed as a unified capability in the Admin console. When investigating an RPC failure, users need to switch between different tools and manually connect the trace, related logs, and metric changes.

This issue proposes adding log observability and cross-signal correlation to Dubbo Admin.

Current status

The current codebase already contains several extension points:

• pkg/config/observability provides Prometheus and Grafana configuration.
• pkg/console/handler/observability.go builds Grafana dashboard URLs.
• pkg/console/router/router.go exposes metric and trace dashboard routes for applications, instances, and services.
• ui-vue3/src/components/GrafanaPage.vue provides the reusable Grafana iframe container.
• Instance metric collection already reads Prometheus exposition data from the application metrics endpoint.

The missing parts are log data source adapters, log dashboard routes, a common correlation context, and metric compatibility handling for dubbogo.

Proposed scope

1. Add configurable Loki and Elasticsearch log providers.
2. Introduce a unified log query and result model in the Go backend.
3. Add application, instance, and service level log dashboard configuration and routes.
4. Pass a common context containing service, application, instance, method, trace ID, and time range between metric, trace, and log panels.
5. Support trace-to-log lookup and opening the related metric dashboard.
6. Add metric label/profile mapping so dubbogo and dubbojava metrics can use the same display model where possible.
7. Add unit tests, provider mock tests, frontend tests, fixture data, and configuration documentation.

Implementation direction

The first implementation should reuse the existing Grafana integration and add a small backend abstraction instead of introducing a new observability platform.

Resource detail page
        |
        v
CorrelationContext
        |
  +-----+------+
  |            |
Log query   Grafana URL builder
  |       metric / trace / log
  v
LokiAdapter or ElasticsearchAdapter

The backend should translate the unified query model into LogQL or Elasticsearch Query DSL. The frontend should only call Dubbo Admin APIs and should not directly expose data source credentials or arbitrary downstream URLs.

Acceptance criteria

• Loki can be configured and queried from Dubbo Admin.
• Elasticsearch can be configured and queried from Dubbo Admin.
• Applications, instances, and services have clear log dashboard entry points.
• A trace ID and time range can be used to find related logs.
• The same resource and time context can be used to open the related metric dashboard.
• Existing metric and trace features remain available when no log provider is configured.
• dubbogo and dubbojava metric fixtures pass the normalization/compatibility tests.
• The implementation includes tests and documentation that can be reproduced by contributors.

Non-goals for the first version

• Replacing Loki, Elasticsearch, Prometheus, or Grafana.
• Building a separate log storage or alerting platform.
• Implementing advanced log aggregation, anomaly detection, or long-term trace storage.

Questions for discussion

• Should log queries be proxied through Dubbo Admin, or should the first version only generate Grafana dashboard URLs?
• Which log fields and Grafana variables are considered stable across current Dubbo Java and dubbogo deployments?
• Should the first version prioritize trace ID correlation, or also include resource/time-window correlation when trace IDs are unavailable?

Related project

This proposal corresponds to the OSPP project “增强Dubbo Admin的可观测能力”. The implementation would use Go for the backend and TypeScript for the frontend, with RISC-V64 compatibility included in the build verification.

[dodjdnh]

@robocanic Hello, I am preparing an OSPP application for this project and
drafted this proposal based on the current Dubbo Admin codebase.

Could you please review whether the proposed scope and implementation direction
are appropriate? In particular, I would appreciate your feedback on the log
query API boundary, metric-trace-log correlation, and dubbogo metric
compatibility.

Any suggestions about priorities or existing modules/issues to study would also
be very helpful. Thank you!

[robocanic]

I’m going through all the proposals and will need some time.

[Xinbeok]

Has this issue been resolved? I want to claim this issue.

[dodjdnh]

Has this issue been resolved? I want to claim this issue.

Hi @Xinbeok ,I'm currently working on this issue for the OSPP project and will submit a PR shortly. Thanks for checking in!