diff --git a/docs/Collecting Metrics/AWS CloudWatch Profile Format.mdx b/docs/Collecting Metrics/AWS CloudWatch Profile Format.mdx new file mode 100644 index 000000000..ac24e9c3a --- /dev/null +++ b/docs/Collecting Metrics/AWS CloudWatch Profile Format.mdx @@ -0,0 +1,200 @@ +--- +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/cloudwatch/profile-format.md" +sidebar_label: "AWS CloudWatch Profile Format" +learn_status: "Published" +toc_max_heading_level: "6" +toc_collapsible: "true" +learn_rel_path: "Collecting Metrics" +keywords: [aws, cloudwatch, profiles, collectors, metrics] +description: "Define CloudWatch namespaces, supported regions, resource dimensions, metrics, statistics, and dynamic charts in reusable profiles." +sidebar_position: "80" +learn_link: "https://learn.netdata.cloud/docs/collecting-metrics/aws-cloudwatch-profile-format" +slug: "/collecting-metrics/aws-cloudwatch-profile-format" +--- + +# AWS CloudWatch profile format + +CloudWatch profiles define which metrics the collector queries and how those +metrics become Netdata charts. Each profile covers one CloudWatch namespace and +one exact resource-dimension grain. + +Stock profiles are installed under +`/usr/lib/netdata/conf.d/go.d/cloudwatch.profiles/default/`. To customize one, +copy it to `/etc/netdata/go.d/cloudwatch.profiles/` and keep the same filename. +A user profile fully replaces the stock profile with that basename. Restart the +go.d process after changing profiles because the catalog is cached process-wide. + +## Complete example + +```yaml +version: v1 +display_name: AWS Example Service +namespace: AWS/Example +supported_regions: [us-east-1] # optional; omit when unrestricted +disabled: true # optional; exclude from the default-enabled set +period: 300 + +instance: + dimensions: + - name: ResourceId + label: resource_id + - name: Scope + constant: Global + +metrics: + - id: requests + metric_name: RequestCount + statistics: [sum] + rate: true + - id: latency + metric_name: Latency + statistics: [average, p99] + period: 60 # optional metric-level override + nil_as_zero: false # optional no-datapoint policy + +template: + family: Example Service + context_namespace: example + chart_defaults: + instances: + by_labels: [account_id, region, resource_id] + charts: + - id: aws_cloudwatch_example_requests + context: requests + title: Example Requests + family: Traffic + units: requests/s + algorithm: absolute + dimensions: + - selector: requests_sum + name: requests + - id: aws_cloudwatch_example_latency + context: latency + title: Example Latency + family: Latency + units: milliseconds + algorithm: absolute + dimensions: + - selector: latency_average + name: average + - selector: latency_p99 + name: p99 +``` + +The filename without `.yaml` is the profile name. In the example, a file named +`example.yaml` is selected as `example`, exports series beginning with +`example.`, and contributes chart contexts under `cloudwatch.example.*`. + +## Top-level fields + +| Field | Required | Description | +|:------|:--------:|:------------| +| `version` | yes | Profile schema version. The supported value is `v1`. | +| `display_name` | yes | Human-readable service name. | +| `namespace` | yes | CloudWatch namespace, such as `AWS/EC2`. | +| `supported_regions` | no | Non-empty list of canonical lowercase region codes where this namespace publishes metrics. Omit for unrestricted regional services. | +| `disabled` | no | When `true`, collection rules do not select the profile through their default set. A rule can still name it explicitly in `profiles.include`. | +| `period` | yes | Default CloudWatch period in seconds. It must be a positive multiple of 60, up to 86400. | +| `instance` | yes | Exact dimension set that identifies one collected resource. | +| `metrics` | yes | Metrics and statistics queried for every discovered instance. | +| `template` | yes | Dynamic chart template populated from the exported series. | + +### Supported regions + +Use `supported_regions` only when AWS publishes a service's CloudWatch metrics +in a fixed subset of regions. CloudFront, for example, is a global service whose +metrics are published in `us-east-1`: + +```yaml +supported_regions: [us-east-1] +``` + +The collection-rule compiler intersects `rules[].regions` with this list: + +- A default-selected profile with no intersection is skipped and reported once + during startup. +- A profile explicitly named in `profiles.include` with no intersection is a + configuration error. +- An omitted `supported_regions` field means that every rule region is allowed. + +An explicitly empty list is invalid; omit the field for unrestricted profiles. + +## Instance dimensions + +`instance.dimensions` is the exact CloudWatch dimension-name set accepted during +discovery. Metrics with missing or extra dimensions do not match. This prevents +different CloudWatch aggregation grains from collapsing into one resource. + +Each dimension sets exactly one of: + +- `label`: emit the dimension value as a Netdata identity label. Labels use + lowercase letters, digits, and underscores, start with a letter, and cannot be + `account_id` or `region`. +- `constant`: require an exact CloudWatch dimension value, include it in the + query, but do not emit it as a label. Use this for a fixed qualifier such as + CloudFront's `Region=Global`. + +Dimension names and constant values are matched verbatim and must not have +leading or trailing whitespace. Names and emitted labels must be unique within +the profile. + +## Metrics + +Every `metrics` entry supports: + +| Field | Required | Description | +|:------|:--------:|:------------| +| `id` | yes | Stable lowercase identifier used in exported series names and chart selectors. | +| `metric_name` | yes | CloudWatch metric name, matched verbatim. | +| `statistics` | yes | One or more of `average`, `minimum`, `maximum`, `sum`, `sample_count`, or a percentile such as `p90` or `p99.9`. | +| `rate` | no | Divide a per-period `sum` or `sample_count` by the effective period and present it per second. Requires one of those total statistics. | +| `period` | no | Metric-specific period override, with the same validation as the profile period. | +| `nil_as_zero` | no | Record a clean no-datapoint response as `0` (`true`) or a gap (`false`). When omitted, rate totals default to zero and other statistics default to gaps. | + +Metric IDs and CloudWatch metric names must be unique within a profile. The +exported series name is: + +```text +._ +``` + +Chart selectors may use the shorthand `_` shown in the +example; the loader qualifies it with the profile name. + +## Chart template rules + +The `template` uses Netdata's dynamic chart-template format. See the complete +[Chart Template Format](/docs/collecting-metrics/chart-template-format) for every +group, chart, instance, label-promotion, lifecycle, dimension, presentation, +and selector field. CloudWatch profiles add these restrictions: + +- Every chart must set `algorithm: absolute`. CloudWatch returns per-period + aggregates, not cumulative counters. +- Every chart's effective `instances.by_labels` must include `account_id`, + `region`, and every identifying dimension `label`. Constant dimensions are not + included. +- Do not author `template.metrics`; the collector derives the visible series + from `metrics`. +- Every dimension selector must resolve to a series exported by the profile. +- Chart IDs must be unique across the complete loaded profile catalog. + +The collector injects the period divisor for rate metrics and marks all emitted +CloudWatch metric families as floating-point values. + +## Authoring workflow + +1. Inspect the service's CloudWatch namespace, metric names, and exact dimension + sets with `aws cloudwatch list-metrics`. +2. Start from the closest stock profile and choose one resource grain. +3. Keep the default metric set focused; CloudWatch bills `GetMetricData` per + requested metric. Leave expensive or high-cardinality additions commented + with matching chart definitions when appropriate. +4. Run the collector tests and profile-catalog validation: + + ```bash + cd src/go + go test -count=1 ./plugin/go.d/collector/cloudwatch/... + ``` + +5. Restart go.d and verify discovery, exported labels, chart identity, gaps, and + the expected CloudWatch API volume in a test account. diff --git a/docs/Collecting Metrics/Chart Template Format.mdx b/docs/Collecting Metrics/Chart Template Format.mdx new file mode 100644 index 000000000..f45394942 --- /dev/null +++ b/docs/Collecting Metrics/Chart Template Format.mdx @@ -0,0 +1,1114 @@ +--- +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/framework/charttpl/README.md" +sidebar_label: "Chart Template Format" +learn_status: "Published" +toc_max_heading_level: "6" +toc_collapsible: "true" +learn_rel_path: "Collecting Metrics" +keywords: [charts, templates, collectors, dimensions, labels] +description: "Define dynamic Netdata chart groups, instances, labels, dimensions, lifecycle, and presentation using declarative templates." +sidebar_position: "70" +learn_link: "https://learn.netdata.cloud/docs/collecting-metrics/chart-template-format" +slug: "/collecting-metrics/chart-template-format" +--- + +# Chart Template Format + +## Overview + +A **chart template** defines _how a collector's metrics are organized into charts_ in the Netdata dashboard. + +> [!NOTE] +> Chart templates are declarative YAML files. You describe **what** to chart, and the engine handles creating, updating, and removing chart instances at runtime. + +It tells the chart engine: + +- which **metrics** to include +- how to **group** them into charts and families +- how to **name** and **scale** dimensions +- how to create **per-instance charts** (e.g., one chart per host, per disk, per user) + +Each collector has a single `charts.yaml` file that describes all its charts. + +> [!TIP] +> Groups can be nested to **any depth**. Family paths, context namespaces, and metric scopes compose automatically as you nest — no need to repeat prefixes. See [groups](#4-groups) for the full composition rules and examples. + +### How Chart Templates Work + +When a collector runs, the chart engine: + +1. Reads the collector's `charts.yaml` file. +2. Compiles it into an immutable program (validates, resolves defaults, infers algorithms). +3. On each collection cycle, matches incoming metrics against dimension selectors. +4. Creates chart instances dynamically based on instance identity labels. +5. Updates dimension values every cycle; removes stale instances based on lifecycle policy. + +**Template Lifecycle** + +```text + charts.yaml + | + v + ┌─────────────────────┐ + │ Decode & Validate │ strict YAML parse + semantic checks + └──────────┬──────────┘ + v + ┌─────────────────────┐ + │ Compile (engine) │ selector parsing, algorithm inference, + │ │ context/family/ID composition + └──────────┬──────────┘ + v + ┌─────────────────────┐ + │ Runtime (per cycle)│ match series → create/update/remove + │ │ charts and dimensions + └─────────────────────┘ +``` + +### Example: Complete Chart Template + +The example below shows a single template that covers all common metric kinds: +gauge, counter, histogram, summary, and stateset. Inline comments explain each field. + +```yaml +version: v1 # schema version (only "v1" supported) +context_namespace: myapp # prefix for all chart contexts → myapp.. + +groups: + # ── Gauge: point-in-time values (algorithm: absolute) ────────────── + - family: Resources + metrics: # metrics visible to dimension selectors in this group + - memory_used_bytes + - memory_total_bytes + - cpu_usage_percent + charts: + - id: memory_usage + title: Memory Usage + context: memory_usage # final context: myapp.memory_usage + units: bytes + type: stacked # line (default), area, stacked, heatmap + algorithm: absolute # absolute = raw value; incremental = rate (value - prev) / interval + instances: + by_labels: [host] # one chart per unique "host" label value + dimensions: + - selector: memory_used_bytes + name: used # static dimension name + - selector: memory_total_bytes + name: total + options: + hidden: true # collected but not drawn (useful for % calculations) + + - id: cpu_usage + title: CPU Usage + context: cpu_usage + units: percentage + instances: + by_labels: [host] + dimensions: + - selector: cpu_usage_percent + name: used + options: + divisor: 100 # raw value in basis points → divide by 100 for percent + float: true # use floating-point precision + + # ── Counter: monotonically increasing values (algorithm: incremental) ── + - family: Traffic + metrics: + - http_requests_total + - bytes_received + - bytes_sent + charts: + - id: http_requests + title: HTTP Requests + context: http_requests + units: requests/s + algorithm: incremental # engine computes rate: (current - previous) / interval + instances: + by_labels: [host] + dimensions: + - selector: http_requests_total + name_from_label: method # dynamic name: each unique label value becomes a dimension + # e.g., method="GET" → dim "GET", method="POST" → dim "POST" + + - id: bandwidth + title: Network Bandwidth + context: bandwidth + units: kilobits/s + type: area + algorithm: incremental + instances: + by_labels: [host] + dimensions: + - selector: bytes_received + name: in + options: + multiplier: 8 # bytes → bits + divisor: 1000 # bits → kilobits + - selector: bytes_sent + name: out + options: + multiplier: -8 # negative = drawn below zero line (bidirectional chart) + divisor: 1000 + + # ── Histogram: bucketed distribution (flattened into _bucket, _count, _sum) ── + - family: Latency + metrics: + - request_duration_seconds_bucket + - request_duration_seconds_count + - request_duration_seconds_sum + charts: + - id: request_duration_buckets + title: Request Duration Buckets + context: request_duration_buckets + units: observations/s + type: heatmap # histogram bucket charts are forced to heatmap + algorithm: incremental # histogram range buckets are counter-like totals + instances: + by_labels: [host] + dimensions: + - selector: request_duration_seconds_bucket + # no name, no name_from_label → engine infers dimension names + # from the "le" (less-than-or-equal) label automatically: + # le="0.005" → dim "0.005", le="0.01" → dim "0.01", etc. + + - id: request_rate + title: Request Rate + context: request_rate + units: requests/s + algorithm: incremental + instances: + by_labels: [host] + dimensions: + - selector: request_duration_seconds_count + name: requests + + # ── Summary: quantile distribution (flattened into quantile values, _count, _sum) ── + - family: Response Time + metrics: + - response_time_seconds + - response_time_seconds_count + - response_time_seconds_sum + charts: + - id: response_time_quantiles + title: Response Time Quantiles + context: response_time_quantiles + units: seconds + algorithm: absolute # quantile values are gauges, not counters + instances: + by_labels: [host] + dimensions: + - selector: response_time_seconds + # no name, no name_from_label → engine infers dimension names + # from the "quantile" label automatically: + # quantile="0.5" → dim "0.5", quantile="0.99" → dim "0.99", etc. + options: + float: true + + # ── StateSet: named boolean states (exactly one active at a time) ── + - family: Health + metrics: + - service_status + charts: + - id: service_health + title: Service Health Status + context: service_health + units: state + instances: + by_labels: [host] + dimensions: + - selector: service_status + # no name, no name_from_label → engine infers dimension names + # from the metric-name label (service_status=) automatically: + # service_status="ready" → dim "ready", service_status="degraded" → dim "degraded", etc. +``` + +**What this template produces** — if the collector reports metrics for 2 hosts (`host="web-1"`, `host="web-2"`), the engine creates **2 instances of every chart** (one per host). The histogram bucket chart is a heatmap with one non-overlapping range dimension per `le` boundary, the summary chart gets one per quantile, and the stateset chart gets one per state — all named automatically by the engine. + +## Template Structure + +Every `charts.yaml` follows this structure: + +```yaml +version: +context_namespace: +engine: +groups: + - family: + context_namespace: + metrics: + chart_defaults: + charts: + groups: +``` + +| Section | Purpose | +|-----------------------------------------------|----------------------------------------------------| +| [**version**](#1-version) | Schema version (must be `v1`). | +| [**context_namespace**](#2-context_namespace) | Top-level prefix for chart context paths. | +| [**engine**](#3-engine) | Engine-level policy (selectors, autogeneration). | +| [**groups**](#4-groups) | Recursive chart groups — the core of the template. | + +--- + +## Field Reference + +### 1. version + +Schema version. Currently only `v1` is supported. Defaults to `v1` if omitted. + +```yaml +version: v1 +``` + +### 2. context_namespace + +Top-level prefix for all chart context paths in the template. Combined with group-level `context_namespace` and chart `context` to form the final context. + +```yaml +context_namespace: mysql +``` + +**Context composition** — the final chart context is built by joining all context parts with `.`: + +``` +.... +``` + +For example: + +| Level | Value | +|-------------------------------|---------------------| +| Top-level `context_namespace` | `mysql` | +| Group `context_namespace` | _(empty)_ | +| Chart `context` | `queries` | +| **Resulting context** | **`mysql.queries`** | + +**Autogen charts** — the **top-level** `context_namespace` also prefixes the contexts of charts +created by `engine.autogen` (metrics not matched by any template dimension), joined with the same +`.`. Group-level `context_namespace` does not apply to autogen, since unmatched series belong to +no group. For example, with top-level `context_namespace: nagios`, an unmatched metric +`check_load` autogenerates the context `nagios.check_load`. + +The autogen context is `context_namespace` joined with the **full metric name**, and a metric's name +includes any `SnapshotMeter("")` prefix (`.`). So a non-empty meter prefix +**stacks after** `context_namespace` — e.g. `context_namespace: app` with `SnapshotMeter("app")` and +instrument `foo` yields `app.app.foo`. When you set `context_namespace`, write metrics with +`SnapshotMeter("")` so the namespace has a single source; do not also encode it in the meter prefix. + +### 3. engine + +Template-level policy that controls metric filtering and autogeneration. + +```yaml +engine: + selector: + allow: ["cpu_*", "memory_*"] + deny: ["cpu_guest_*"] + autogen: + enabled: true + expire_after_success_cycles: 50 +``` + +| Field | Type | Default | Description | +|---------------------------------------|---------------|-------------|----------------------------------------------------------------------------------------| +| `selector.allow` | array[string] | _(empty)_ | Include only metrics matching these patterns (simple patterns: `*` and `?` wildcards). | +| `selector.deny` | array[string] | _(empty)_ | Exclude metrics matching these patterns (simple patterns: `*` and `?` wildcards). | +| `autogen.enabled` | bool | `false` | Create charts for metrics not matched by any template dimension. | +| `autogen.max_type_id_len` | int | `0` (=1200) | Max full `type.id` length. Must be `0` or `>= 4`. | +| `autogen.expire_after_success_cycles` | uint64 | `0` | Remove autogenerated charts not seen for N successful cycles (`0` = never). | + +**When to use autogen**: For collectors like Nagios plugins where the set of metrics is unpredictable and user-defined. The engine creates a chart for every unmatched metric automatically. + +**Example: Nagios collector with autogeneration** + +```yaml +version: v1 +context_namespace: nagios +engine: + autogen: + enabled: true + expire_after_success_cycles: 50 +groups: + - family: Job + context_namespace: job + groups: + - family: Execution + metrics: + - nagios.job.execution_state + - nagios.job.execution_duration + charts: + - id: job_execution_state + title: Job Execution State + context: execution_state + units: state + instances: + by_labels: [nagios_job] + dimensions: + - selector: nagios.job.execution_state + - id: job_execution_duration + title: Execution Duration + context: execution_duration + units: seconds + instances: + by_labels: [nagios_job] + dimensions: + - selector: nagios.job.execution_duration + name: duration + options: + float: true +``` + +Explicitly defined charts (like `execution_state`) use the template. Any _other_ metrics the Nagios plugin emits get auto-charted by the engine. + +### 4. groups + +Groups organize charts into a hierarchy that can be nested to **any depth**. Each group defines a **family** segment, can declare **metrics** in scope, and contains **charts** and/or nested **groups**. + +Nesting serves three purposes: + +1. **Family composition** — each level's `family` is joined with `/`, producing Netdata's hierarchical family structure automatically (the UI renders `/`-separated families as navigable levels). +2. **Context composition** — each level's `context_namespace` is joined with `.`, so you write short context leaves instead of long prefixed strings. +3. **Metric scoping** — metrics declared in a group are inherited by all descendants, so you declare once at the appropriate level. + +```yaml +groups: + - family: + context_namespace: + metrics: + - + chart_defaults: + label_promotion: [