Skip to content

feat: add vendor extensions metric with per-extension counts - #3021

Open
n0rahh wants to merge 24 commits into
mainfrom
feat/stats-for-vendor-extensions
Open

feat: add vendor extensions metric with per-extension counts#3021
n0rahh wants to merge 24 commits into
mainfrom
feat/stats-for-vendor-extensions

Conversation

@n0rahh

@n0rahh n0rahh commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What/Why/How?

Adds a Vendor Extensions metric to the stats command. It reports how many distinct x- extensions a document uses and how many times each one occurs, shown in the stylish, json, and markdown output. Works across OpenAPI and AsyncAPI.

The stats visitors collect extensions through a single SpecExtension entrypoint. Two walker fixes make that possible:

  • SpecExtension nodes now dedupe by location, so every occurrence is visited — previously occurrences with equal scalar values (e.g. x-internal: true on many operations) were visited only once. This also means visitors and configurable rules targeting SpecExtension now fire per occurrence.
  • Fixed a pre-existing bug where x- properties with a declared type (e.g. x-codeSamples) were walked twice, which inflated other metrics.

For the stats walk only, ensureSpecExtensionDispatch adjusts the command's normalized types so every x- key dispatches as SpecExtension — including natively-typed extensions and AsyncAPI types that don't declare extensionsPrefix. The structural extensions x-webhooks and x-query keep their declared types so the webhooks/operations/tags metrics still traverse their subtrees; the visitors count those two explicitly. Lint and bundle behavior is unchanged.

The collector also gathers per-extension prop names and value samples for the portal's stats collector (telemetry) via the accumulator — the CLI prints only totals and counts. Samples are bounded (20 props / 20 values per extension), long strings become a <string:N> marker, and credential-like keys and values are masked.

Fixed the stats command always reporting Parameters: 0 for AsyncAPI 2.x and 3.x descriptions. Channel parameters are keyed by name rather than carrying a name property, so none of them were counted.

Reference

Testing

Covered with unit and e2e tests.
Published snapshot and tested cli commands in terminal.

Screenshots (optional)

image image image

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

Note

Medium Risk
Walker changes affect how often SpecExtension visitors and rules fire per occurrence, though dispatch tweaks are scoped to the stats command; stats metrics and AsyncAPI parameter totals will change for existing AsyncAPI fixtures.

Overview
Adds a Vendor Extensions metric to stats for OpenAPI and AsyncAPI: total distinct x- keys plus per-extension occurrence counts in stylish, JSON (counts), and markdown (breakdown table) output.

Stats collection routes x- keys through a unified SpecExtension visitor after ensureSpecExtensionDispatch adjusts normalized types for the stats walk only (lint/bundle unchanged). x-webhooks and x-query stay structurally typed and are counted explicitly. The walker now dedupes SpecExtension visits by document location so repeated scalars (e.g. many x-internal: true) are counted correctly, and skips duplicate x- props already declared on the type tree.

AsyncAPI Parameters now use the channel parameter map key instead of parameter.name, fixing a constant 0 count on 2.x/3.x.

Extension sampling for telemetry (masked/bounded in the accumulator) is collected in core; CLI output shows counts only.

Reviewed by Cursor Bugbot for commit 66b983c. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 66b983c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@redocly/openapi-core Minor
@redocly/cli Minor
@redocly/client-generator Patch
@redocly/respect-core Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 77.23% (🎯 77%) 11707 / 15158
🔵 Statements 77.33% (🎯 77%) 12537 / 16212
🔵 Functions 81.16% (🎯 81%) 2361 / 2909
🔵 Branches 70.95% (🎯 70%) 8628 / 12160
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/commands/stats/index.ts 0% 0% 0% 0% 28-71
packages/cli/src/commands/stats/visitor-and-accumulator-resolver.ts 0% 0% 0% 0% 13-57
packages/cli/src/commands/stats/print-stats/json.ts 0% 0% 0% 0% 8-19
packages/cli/src/commands/stats/print-stats/markdown.ts 0% 0% 0% 0% 10-25
packages/cli/src/commands/stats/print-stats/stylish.ts 0% 0% 0% 0% 11-17
packages/core/src/walk.ts 98.24% 97.51% 85% 98.75% 220, 235, 459
packages/core/src/rules/other/stats.ts 4.91% 0% 0% 4.91% 23-31, 36-119, 126-180, 187-243
packages/core/src/utils/spec-extensions.ts 75.92% 70.37% 75% 76.08% 29-41, 71, 81
Generated in workflow #11289 for commit 66b983c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
cli-latest ▓ 1.00x (Fastest) ▓ 1.00x (Fastest) ▓ 1.00x ± 0.01
cli-next ▓ 1.01x ± 0.01 ▓ 1.01x ± 0.01 ▓ 1.00x (Fastest)

@n0rahh n0rahh added the snapshot Create experimental release PR label Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📦 A new experimental 🧪 version v0.0.0-snapshot.1785934483 of Redocly CLI has been published for testing.

Install with NPM:

npm install @redocly/cli@0.0.0-snapshot.1785934483

⚠️ Note: This is a development build and may contain unstable features.

@n0rahh
n0rahh marked this pull request as ready for review August 5, 2026 13:45
@n0rahh
n0rahh requested review from a team as code owners August 5, 2026 13:45
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
… to $ref and ignore map keys starting with x-
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread .changeset/seven-waves-create.md Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
@n0rahh n0rahh added snapshot Create experimental release PR and removed snapshot Create experimental release PR labels Aug 6, 2026
@n0rahh
n0rahh requested a review from DmitryAnansky August 6, 2026 10:07
Comment thread .changeset/seven-waves-create.md Outdated
Comment thread docs/@v2/commands/stats.md
Comment thread docs/@v2/commands/stats.md
Comment thread tests/e2e/stats/stats-async2-json/snapshot.txt
@n0rahh
n0rahh requested a review from DmitryAnansky August 6, 2026 14:14
Comment thread packages/core/src/utils/spec-extensions.ts Outdated
Comment thread packages/core/src/rules/other/stats.ts Outdated
Comment thread packages/core/src/typings/common.ts Outdated
Comment thread packages/core/src/utils/spec-extensions.ts Outdated
Comment thread packages/core/src/utils/spec-extensions.ts Outdated
Comment thread packages/core/src/utils/__tests__/spec-extensions.test.ts Outdated
@adamaltman

Copy link
Copy Markdown
Member

I like this idea

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 561ede2. Configure here.

Comment thread packages/core/src/rules/other/stats.ts Outdated
@n0rahh n0rahh added snapshot Create experimental release PR and removed snapshot Create experimental release PR labels Aug 7, 2026
@n0rahh
n0rahh requested a review from tatomyr August 7, 2026 10:49
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

📦 A new experimental 🧪 version v0.0.0-snapshot.1786099785 of Redocly CLI has been published for testing.

Install with NPM:

npm install @redocly/cli@0.0.0-snapshot.1786099785

⚠️ Note: This is a development build and may contain unstable features.

@adamaltman

Copy link
Copy Markdown
Member

I found things I didn't know existed in there. Thank you.

Document: ./website/api-definitions/openapi/openapi.yaml stats:

🚗 References: 972
📦 External Documents: 0
📈 Schemas: 838
👉 Parameters: 61
🔗 Links: 0
🔀 Path Items: 382
🎣 Webhooks: 148
👷 Operations: 613
🔖 Tags: 88
🧩 Vendor Extensions: 16
  - x-badges: 151
  - x-basic: 92
  - x-codeSamples: 261
  - x-currency-field: 16
  - x-enumDescriptions: 79
  - x-is-free-form: 1
  - x-label: 26
  - x-logo: 1
  - x-multiline: 31
  - x-products: 761
  - x-sdk-operation-name: 613
  - x-sdk-resource: 43
  - x-sdk-skip: 5
  - x-sortable: 155
  - x-tagGroups: 1
  - x-type: 138

./website/api-definitions/openapi/openapi.yaml: stats processed in 143ms

Comment thread docs/@v2/commands/stats.md Outdated
@n0rahh
n0rahh requested a review from RomanHotsiy August 10, 2026 07:17
webhooks: { metric: '🎣 Webhooks', total: 0, color: 'green' },
operations: { metric: '👷 Operations', total: 0, color: 'yellow' },
tags: { metric: '🔖 Tags', total: 0, color: 'white', items: new Set() },
xExtensions: { metric: '🧩 Vendor Extensions', total: 0, color: 'cyan' },

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saw we have items inside the tags accumulator (one line above). What if we arrange extensions similarly to tags? Is there something else we are going to show in the output except of the extension items and their count?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need a Map (a Set can't do keyed count++ per occurrence), which turns items into Set | Map<string, {count, props}> and pushes type-narrowing into every consumer — plus a Map isn't JSON-serializable for the JSON printer and the portal collector. Keeping items as "distinct names, size = total" and putting the per-extension payload in details seemed cleaner for me

Operation: {
enter(operation: Oas3Operation, ctx: UserContext) {
if (ctx.key === 'x-query') {
collectSpecExtension(extensions, 'x-query', operation);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to treat x-query differently? Isn't it already a SpecExtension?

I see, it has its own type.

SharedResponse:
description: ok
schemas:
x-MySchema:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one (as well as x-trace-id) shouldn't be counted as an extension, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it just counts as a schema name

Comment thread packages/core/src/walk.ts

currentLocation = resolvedLocation;
const isNodeSeen = seenNodesPerType[type.name]?.has?.(resolvedNode);
const seenKey = type === SpecExtension ? location.absolutePointer : resolvedNode;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this change for?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SpecExtension nodes are often equal scalars (true, "internal"), and deduping them by node value made the walker visit only the first occurrence document-wide — deduping by location visits every occurrence exactly once (while a $ref-shared extension still counts once), which the stats need to count extensions correctly.

export function ensureSpecExtensionDispatch(types: Record<string, NormalizedNodeType>) {
for (const type of Object.values(types)) {
if (type === SpecExtension) continue;
type.extensionsPrefix ??= EXTENSION_PREFIX;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is still needed after using the SpecExtension visitor.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is needed: the walker only dispatches an x- key to SpecExtension when the type declares extensionsPrefix, and most AsyncAPI types (everything except SecurityScheme) plus Paths in OAS never declare it. Removing this line drops the AsyncAPI fixture's extension count — only the additionalProperties: {} path survives.

Adding extensionsPrefix to the AsyncAPI type definitions themselves would make extension values walked (and their $refs bundled/linted) for every consumer, which is a behavior change beyond stats — could be a follow-up if we want it.

},
},
WebhooksMap: {
enter(node: unknown, ctx: UserContext) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also have other known extensions, like the following:

    'x-servers': 'XServerList',
    'x-tagGroups': 'TagGroups',
    'x-ignoredHeaderParameters': { type: 'array', items: { type: 'string' } },

Have you covered them?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are covered. Added e2e tests to make it clear

}
}
const extensionNames = Object.keys(extensions).sort();
statsAccumulator.xExtensions.total = extensionNames.length;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the same approach with Set like in other rows?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover, maybe we can simply calculate the items' size in place instead of assigning the total?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Set question already answered above.

Four consumers (three printers and the portal collector) read .total uniformly for every row, so deriving the size at read time would push an items ? items.size : total check into each of them. This one-time assignment at Root.leave also predates the PR — the loop just replaces the four per-row copies that main already had.

const entryType = type.additionalProperties;
// An untyped catch-all (`additionalProperties: {}`) swallows x- keys before the extensions fallback.
if (isPlainObject(entryType) && !isNamedType(entryType) && entryType.type === undefined) {
type.additionalProperties = (_value, key: string) =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a good idea to modify types. What are you're trying to achieve?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This modifies only the stats command's own copy of the types — lint keeps the original ones where the declarations matter for validation. Without it, typed extensions (x-codeSamples, x-logo) and most AsyncAPI nodes never reach the SpecExtension visitor, since the walker dispatches them to their declared types instead. The only alternative is going back to key-scanning in the any hook — the approach you wanted to replace with SpecExtension

Comment thread .changeset/olive-donkeys-shave.md Outdated
'@redocly/cli': patch
---

Fixed the `stats` command always reporting `Parameters: 0` for AsyncAPI 2.x and 3.x descriptions.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Fixed the `stats` command always reporting `Parameters: 0` for AsyncAPI 2.x and 3.x descriptions.
Fixed the `stats` command reporting wrong parameter count for AsyncAPI descriptions.

Comment thread packages/core/src/walk.ts

currentLocation = resolvedLocation;
const isNodeSeen = seenNodesPerType[type.name]?.has?.(resolvedNode);
const seenKey = type === SpecExtension ? location.absolutePointer : resolvedNode;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n0rahh
why scope the fix to SpecExtension only? can other primitive nodes hit the same problem?
If they can, using resolvedLocation.absolutePointer as a key instead of checking type === SpecExtension would cover them all in one place

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only SpecExtension holds arbitrary user data, so equal scalar values like true are its normal case — other node types hold objects, which never collide as keys. Using the location as the key for all types isn't safe either: a YAML anchor puts the same object in many places, and every rule would then visit and report it once per place instead of once

@n0rahh
n0rahh requested review from tatomyr and vadyvas August 11, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

snapshot Create experimental release PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants