Skip to content

fix(connectors): make selector dropdowns resolve options the drain has not reached - #6027

Merged
waleedlatif1 merged 8 commits into
stagingfrom
confluence-spaces-ordering
Jul 29, 2026
Merged

fix(connectors): make selector dropdowns resolve options the drain has not reached#6027
waleedlatif1 merged 8 commits into
stagingfrom
confluence-spaces-ordering

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Connector selector option lists fill by draining pages in the background and are filtered client-side, so an option that has not arrived yet is indistinguishable from one that does not exist. Production logs for the reporting workspace show a 24-page, 38-second drain at the exact minute of the report.

1. The empty state lied. The field ignored the drain state the shared hook already exposes (hasMore / isFetchingMore / truncated / error), so a still-filling list rendered "No spaces found" — which reads as this space does not exist and sends users off to enter the value by hand. It now distinguishes still-loading, failed-to-load, and too-many-to-list. error is checked first: a failed page halts the drain but leaves hasMore set, which would otherwise claim to be loading forever.

2. An exact key could not be resolved without waiting for the drain. confluence.spaces.fetchById fetched the first page and scanned it, so a space sorting beyond page 1 never resolved — on a ~6,000-space site, nearly all of them. It now uses the documented v2 keys filter. Both current and archived are queried explicitly and concurrently: status takes a single value on /spaces and Atlassian documents no default there (unlike /pages, where it is an array defaulting to current,archived), and archived spaces are selectable and sync against.

The typed value and the selected value are resolved through fetchById and merged into the option list, so an exact key is selectable immediately and a just-picked option does not vanish when the search box resets.

3. useSelectorOptionDetail could throw. A caller-supplied enabled replaced the guard checking that a definition declares fetchById, so queryFn's non-null assertion threw for the ~12 selectors without one. A caller's enabled is now narrowed by that hard precondition — but deliberately not by definition.enabled, which describes when a list can be fetched and gates on context (credential, domain, region) that resolving one known id may not need. cloudwatch.* echoes an id back without calling AWS, so gating on the list predicate would blank the log-group name on collapsed canvas blocks.

Adds onSearchChange to the shared Combobox, read through a ref so its identity stays stable for the handlers that capture it without declaring it, and deduped because several reset paths fire redundantly.

Deliberately not done

  • Draining confluence.pages. An earlier revision did this and it was reverted: with no search term title is unset, so opening the dropdown walked the entire site — up to 50 sequential requests and 2,500 options where there had been one request and 50 — the route forwards no abort signal upstream so superseded drains still bill the tenant's rate limit, and the same fan-out reached loadAllSelectorOptions. Its justification also rested on title match semantics Atlassian does not publish. The 50-item cap is a real gap that needs confirmed server-side search, not brute force.
  • sort=name on /spaces. Atlassian documents sort's allowed values only on the different /data-policies/spaces endpoint. A rejected value 400s, our route maps that to 502, and the dropdown breaks entirely.
  • CQL substring search. Response shape unverified, and space.title~ can false-match on the creator's username.

Type of Change

  • Bug fix

Testing

Typecheck, lint:check (19/19), check:api-validation, and 497 tests pass. Root cause established from CloudWatch (/ecs/sim-production/us-east-1/app), not inferred.

Reviewed by the eight /cleanup passes and then by six adversarial verification passes against the Atlassian OpenAPI spec and the surrounding plumbing. Between them they found five real defects in this diff — the enabled guard, a vanishing selected option, a combobox stale closure, an unverified status default, and the pages-drain regression — all fixed or reverted here.

Not runtime-verified. Nothing here has executed against a live Confluence instance, and there is no test file for these routes. Remaining unknowns, all needing a live call: title match semantics, whether /spaces silently clamps limit=250, and keys case-sensitivity.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 29, 2026 1:13am

Request Review

@cursor

cursor Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches shared selector hooks and Combobox used broadly; Confluence OAuth API calls change behavior for large-space tenants but are scoped to selector routes and opt-in speculative lookup.

Overview
Fixes connector selector dropdowns where background page draining plus client-side filtering made options look missing or unselectable on large tenants (e.g. Confluence with thousands of spaces).

Connector selector UI now uses drain state (hasMore, isFetchingMore, truncated, error) for empty copy instead of a flat “not found”, merges labels for saved/selected values via useSelectorOptionDetails, and—when a selector sets resolvesUnknownIds—debounced fetchById on search text so exact keys (Confluence space keys) appear before their page loads. Shared Combobox gains onSearchChange (deduped, stable via ref) to drive that path.

Selector hooks: useSelectorOptionDetail always requires fetchById before running (fixes throws when callers passed enabled alone); adds useSelectorOptionDetails with shared detail cache keys. resolvesUnknownIds is documented on selector definitions; Confluence spaces opts in.

Confluence spaces API accepts optional spaceKey, runs concurrent keys lookups for current and archived status (partial failure tolerated), and refactors listing through a shared fetch helper. fetchById for spaces uses that lookup instead of scanning page one. confluence.pages stays a single-request list (explicit comment against draining the whole site).

Reviewed by Cursor Bugbot for commit 10bb508. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes connector selector dropdowns so options still loading via background page drain can be resolved and empty states no longer claim a miss while the list is filling.

  • Confluence spaces API supports exact spaceKey lookup via concurrent current/archived keys filter.
  • Connector selector field merges fetchById results for selected and typed values; empty copy reflects drain/error/truncated state.
  • useSelectorOptionDetail(s) keeps fetchById as a hard enable precondition; Combobox adds stable, deduped onSearchChange.

Confidence Score: 5/5

This follow-up pass found no remaining blocking failures tied to prior Greptile threads, so the PR appears safe to merge from that scope.

No blocking failure remains from prior Greptile findings or eligible follow-up issues on the current head.

Important Files Changed

Filename Overview
apps/sim/app/api/tools/confluence/selector-spaces/route.ts Adds exact-key space lookup with concurrent current/archived queries and shared response mapping.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/connector-selector-field/connector-selector-field.tsx Wires detail resolution and drain-aware empty messages into the connector selector UI.
apps/sim/hooks/selectors/use-selector-query.ts Hardens detail-query enablement and adds multi-id detail resolution sharing the detail cache.
packages/emcn/src/components/combobox/combobox.tsx Adds ref-backed, deduped onSearchChange for searchable combobox consumers.

Reviews (8): Last reviewed commit: "refactor(connectors): resolve selected o..." | Re-trigger Greptile

@waleedlatif1
waleedlatif1 force-pushed the confluence-spaces-ordering branch from ac07b0c to 030df2b Compare July 28, 2026 23:23
@waleedlatif1 waleedlatif1 changed the title fix(connectors): order Confluence spaces by name and stop implying a space is missing fix(connectors): stop the connector selector implying an option does not exist Jul 28, 2026
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@waleedlatif1
waleedlatif1 force-pushed the confluence-spaces-ordering branch from 030df2b to 3f145fc Compare July 28, 2026 23:33
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3f145fc. Configure here.

…not exist

The connector selector field ignored the drain state the shared selector hook
already exposes. Paginated selectors fill in the background and the combobox
filters client-side, so a not-yet-drained option is genuinely absent from the
list — and the dropdown said "No spaces found", which reads as "this space does
not exist" and sends users off to enter the value by hand. That is what happened
with a Confluence space on a site with thousands of personal spaces.

It now reports that the list is still filling, surfaces a failed drain instead of
claiming to load forever, and is honest when the drain stops at its page cap.
…ng on the drain

The connector selector fills by draining pages in the background and filters
client-side, so an option is only findable once its page has arrived. On a large
Confluence site that is ~38 seconds, during which searching for a real space
returned nothing.

Resolves the typed value and the selected value directly through the selector's
`fetchById`, merging both into the option list, so an exact key is selectable
immediately regardless of drain progress. `confluence.spaces.fetchById` now uses
the documented v2 `keys` filter instead of scanning only the first page — which
never resolved a space sorting beyond it.

Adds `onSearchChange` to the shared Combobox so a consumer can observe the search
box, held in a ref so its identity stays stable for the handlers that capture it
without declaring it.

Also fixes a pre-existing hazard in useSelectorOptionDetail: a caller-supplied
`enabled` replaced the guard that checks a definition declares `fetchById`, so
`queryFn`'s non-null assertion would throw for the ~12 selectors without one.
…turning one page

`/api/tools/confluence/pages` issued a single request and discarded the
`_links.next` cursor Confluence returns, so the picker showed only the first
`limit` pages of however many exist — a search for a real page found nothing,
with no error and no truncation signal.

Threads the documented opaque cursor and moves the selector to `fetchPage`, so
the option list drains like `confluence.spaces` and reports real `hasMore` /
`truncated` state. The `title` server-side filter is unchanged and now paginates
too. `limit` is deliberately left at its existing default: the endpoint's maximum
is not confirmable from Atlassian's published reference, and raising it is not
needed for correctness.

Also guards `data.results`, which threw when the response omitted it.
@waleedlatif1 waleedlatif1 changed the title fix(connectors): stop the connector selector implying an option does not exist fix(connectors): make selector option lists complete and honest Jul 29, 2026
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 58fb25a. Configure here.

…okup

The exact-key lookup omitted `status`, assuming that matched a space whether it
was current or archived. Atlassian documents a `current,archived` default for
`/pages` but documents no default for `/spaces`, where `status` takes a single
value rather than an array — so the assumption was unverified, and resolving
only current spaces would silently miss archived ones. Archived spaces are
reachable through the paged path and sync works against them.

Queries `current` first and falls back to `archived` only when it finds nothing,
so the common case stays one request and the behaviour no longer depends on an
undocumented default.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 63603b5. Configure here.

…en key lookup

Six adversarial verification passes against the provider spec and the surrounding
plumbing found three defects in the prior commits.

Reverts the Confluence pages selector to a single request. Draining it was not
strictly better: with no search term `title` is unset, so opening the dropdown
walked the entire site — up to 50 sequential requests and 2,500 options where
there had been one request and 50 — and the route forwards no abort signal
upstream, so superseded drains still bill the tenant's rate limit. The same
fan-out reached `loadAllSelectorOptions`, and the justification rested on `title`
semantics Atlassian does not publish. The list cap is a real gap, but it needs
confirmed server-side search, not brute force. Keeps the `results` guard.

Restores selector display names for CloudWatch blocks. Narrowing a caller's
`enabled` against `definition.enabled` disabled a detail query those selectors
satisfy without AWS context, so collapsed blocks rendered "-" instead of the log
group name. A caller that opts in is now narrowed only by the hard precondition
that `fetchById` exists, which is what `queryFn` actually asserts.

Queries both space statuses concurrently rather than sequentially: the key is
user-typed text, so a miss dominates while typing and paid two round-trips. Each
row now falls back to the status its own call requested.

Also drops the "enter the value directly" empty states — the combobox is not
editable, so there is no such affordance — and dedupes `onSearchChange`, which
several reset paths fire redundantly.
@waleedlatif1 waleedlatif1 changed the title fix(connectors): make selector option lists complete and honest fix(connectors): make selector dropdowns resolve options the drain has not reached Jul 29, 2026
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/tools/confluence/selector-spaces/route.ts Outdated
…d key lookup

Addresses three review findings.

A key lookup failed entirely when either status leg errored, discarding a match
the other leg had found. Only a total failure is fatal now.

Resolved option labels are remembered for the lifetime of the field. Both lookups
key on values that change — the search box clears on select and close, and a
multi-select field resolves no id at all — so the label for a just-picked option
vanished a debounce later and the trigger fell back to a raw id. This covers the
multi-select case, which is what the Confluence space field uses.

A failed exact-value lookup now reads differently from a failed list load, rather
than being reported as "still loading" or "none found".
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread packages/emcn/src/components/combobox/combobox.tsx
…ext changes

Resolved ids are only meaningful within one selector context, so switching
credential, domain, or a dependency has to discard what was remembered under the
old one — the queries re-key, but the remembered labels would linger and mislabel
until the field remounted.

Keyed on the serialized context rather than its identity: the context memo also
depends on `sourceConfig`, so its identity changes on unrelated field edits and
would clear the cache far more often than intended.
Replaces the remembered-label map with per-id queries over the selected values,
following the pattern the knowledge-base selector already uses. The map only ever
held ids searched for in the current session, so a multi-select field restored
from saved config still rendered raw ids — the case that actually matters. It
also needed two effects and a serialized-context key to avoid leaking labels
across a context change, all of which disappear: queries key on the context, so a
label cannot outlive it.

Gates the speculative lookup of typed text on a new `resolvesUnknownIds` flag,
set only where `fetchById` returns null for an id that does not exist. Most
implementations resolve a record by id, so every partial keystroke was a failed
upstream request, retried once, and its error made "could not check that exact
value" the normal empty state on those selectors.

Also: pass handlers straight to the combobox rather than through identity
wrappers that defeated its memo, move the latest-callback ref write into an
effect, rename the raw search setter so the deduping write path cannot be
bypassed, and correct the prop and staleTime docs.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 10bb508. Configure here.

@waleedlatif1
waleedlatif1 merged commit 48d59ca into staging Jul 29, 2026
14 of 15 checks passed
@waleedlatif1
waleedlatif1 deleted the confluence-spaces-ordering branch July 29, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant