spec(content-drive): Title / All Fields search scope + literal-text search terms (#37479, #37532)

[KevinDavilaDotCMS]

Spec-Kit PR 1 of 2. Carries spec.md alone. Review it as a spec, not as code: is this the right problem, scoped right, with criteria a reviewer can tell "done" from? Approval here — not merge — unblocks /speckit-plan.

Refs #37479

What it specifies

A scope dropdown beside the Content Drive search input, with two options — Title and All Content — so an author who knows the name of what they want can say so, instead of getting back every document whose body or Story Block happens to mention the word. It is also the cheap query path: the all-content gate is +(catchall:<value>*^10 OR title_dotraw:*<value>*^2), an all-fields aggregate plus a leading wildcard, and in the drive a broad match multiplies DB hydration and permission filtering downstream, not just index time.

The four open decisions, settled

The issue marks these as needing a call before implementation. Settled by the issue owner on 2026-09-11; approving this PR is the record of that sign-off.

#	Decision	Settled as	Why
1	Default scope	All Content	No-regression. Doing nothing gives you exactly today's results; the fast path is opt-in.
2	Title-mode fields	Contentlet title only	Keeps the promise the label makes and the query on one field. File assets stay findable by name in practice — dotCMS keeps title in step with the file name.
3	Stickiness	URL only, per search	Behaves like every other Content Drive filter. No new preference storage.
4	Sorting	Unchanged	Void as asked — see correction 1 below.

Premises checked against main — four corrections

These are the part most worth a second pair of eyes, because three of them shrink the work and the fourth adds a rule the issue's file list has no place for.

1. Nothing sorts by score today. Decision 4 asks to confirm score-desc holds in Title mode. There is no score sort: DEFAULT_SORT is modDate:DESC (shared/constants.ts:51), the store sends it unconditionally (dot-content-drive.store.ts:157), and the server default is SORT_BY = "modDate" (AbstractDriveRequestForm.java:81). The only trace is a stale comment at dot-content-drive.store.ts:481 sitting above code that does nothing of the kind. → Sorting out of scope; comment corrected in passing.

2. Folders and links are already matched on name only, in both scopes. They never reach Elasticsearch — loaded from the DB and narrowed in Java by substring (BrowserAPIImpl.java:3026 for folders, :2908-2913 for links). → The selector governs the contentlet clause alone; folder/link matching must not change.

3. buildPureESQuery is unreachable for the drive under shipped configuration. doPureESQuery runs only when BROWSE_API_HEURISTIC_TYPE=PURE_ES; the default is HYBRID_SINGLE_CHUNKED_QUERY_ES (BrowserAPIImpl.java:701-709). → Explicitly out of scope rather than changed unverified.

4. A scope stored as a filter would offer "Clear all" on an unfiltered drive. The issue routes the scope through filter state — which is what carries it into the URL — but stops there. That state also feeds the chip bar, and hasNonDefaultFilters (utils/functions.ts:334-355) counts every key but sharedAssets and languageId as a non-default filter, which is exactly what shows "Clear all" (dot-filter-bar.component.html:7). Picking All Content — the default — would light it up on a drive with nothing filtered. → FR-021: the scope counts as filter state only while it differs from the default, mirroring how the search term already deletes its own key when it goes empty.

Scope boundaries

• In: the Content Drive search box, its filter/URL state, the drive's search request contract, and the title-scoped query.
• Out: buildPureESQuery; the all-content strategy shared with the Search portlet and the Relationships dialog, which keeps serving them unmodified — Title is a sibling path, not a branch inside it; the Asset Picker, which reuses the same search box and must stay untouched unless it opts in; sorting; any DB, index-mapping or content-model change.

UI

Deliberately thin. The issue's ASCII diagram is the only design input — no mock image exists — so the spec fixes which components are on screen (input, scope dropdown beside it, placeholder that follows the scope) and stops there. No dimensions, spacing or layout rules.

What a reviewer should push back on

• Decision 1 if you think Title-by-default is worth the behavior change for existing users.
• Decision 2 if excluding fileName / metadata.name will bite in practice — the spec names the gap (a file whose title was edited away from its name) and calls widening it a follow-up rather than a silent change.
• SC-003 carries no fixed latency threshold. No baseline exists yet; producing the dataset and the before/after comparison is part of the work and gets recorded on the issue.

🤖 Generated with Claude Code

[zJaaal]

One naming ask, because our two specs are about to collide in prose and then in the same request object.

This one introduces a scope that says which fields a search reads. #37487 introduces one that says where in the site you are browsing: All, the site root, or System Host. Bare "scope" now names either, and a reader landing on either spec cannot tell which is meant. Worse, both end up as fields on the same Content Drive search request, so the ambiguity outlives the specs.

I have already renamed mine to spell browse scope everywhere, with a note in Key Entities pointing at yours so the two cannot be conflated. Could you do the matching half here: say search scope in the prose rather than plain "scope", and name the field searchScope on the wire?

Then the pair reads unambiguously wherever they meet: searchScope says how a search reads what is there, browseScope says where "there" is. Neither name is load-bearing on its own, but together they stop a future reader guessing.

Unrelated to the naming, and only because it touches the same sentence I was reading: your spec's "All Content" label is the one I flagged above as really meaning all fields. If that wording changes, the field values change with it, so it is worth settling before the name goes on the wire.

Comment written by Claude on @zJaaal's behalf.

[zJaaal]

One more on the contract, following the naming note above: I think the search scope belongs inside filters, next to text, rather than as another top-level field.

AbstractQueryFilters today is { text, filterFolders }, and filterFolders's own Javadoc says "when text is provided". Both members exist to qualify the text search, which is exactly what the search scope does: it says which fields text reads, and it means nothing without text. Putting it beside text makes them travel together, and a request carrying a scope with no text becomes visibly nonsense instead of a validation rule someone has to remember later. So filters.searchScope.

To be clear that this is not a general "scopes go in filters" rule: the browse scope in #37487 stays at the top level, for two reasons. It qualifies assetPath rather than text. And the filter bar's "Clear all" resets the filters object, so a browse scope living there would let clearing your filters navigate you out of System Host. A filter chip narrows what you see where you are; it must not be able to move you somewhere else.

Related, since it came up while looking at this: we also considered letting the path alone carry the browse scope, with no field at all, mirroring how the Content Drive URL now encodes it in one value. It does not survive contact with this endpoint, and for a reason that matters to your spec too: assetPath: "//site/" already means "the whole site at any depth", and the Asset Picker sends exactly that. Redefining it would silently change what the picker asks for. The general rule both our features are leaning on is the same one, that an existing payload has to keep meaning what it means today, and it constrains where new meanings can be expressed.

Worth flagging so nobody invokes consistency in either direction: filters is not currently the home for filters. contentTypes, baseTypes, language, workflow, status and userSearchable all sit at the top level and all of them genuinely are chips in the filter bar. Moving those six in is only part of a real cleanup though, so I would not treat it as the benchmark. The contract also lets a caller say one thing twice, archived: true and status: ["ARCHIVED"], which ContentDriveHelper then has to reconcile, and it carries live as a boolean when it actually selects which version you get rather than filtering anything. That is a contract PR of its own, worth doing while this endpoint still has exactly two callers, and not something either of our feature PRs should take on.

Comment written by Claude on @zJaaal's behalf.

[ihoffmann-dot]

Mid: possible conflict with ADR-0018 (Database-First Search for Content Drive)

ADR-0018's routing table states explicitly:

Title → DB ∪ Index (contentlet.title column ∪ index records)

This union exists specifically to preserve read-your-writes: a just-saved/renamed item that the index hasn't caught up to yet must still be findable via the DB title column.

FR-008/FR-010 define Title scope purely as a narrower Elasticsearch query (drop catchall, drop the leading wildcard, gate on title). Nothing in the FR/SC list references consulting the contentlet.title DB column, and no acceptance scenario or success criterion covers "a document created/renamed just before search, not yet indexed, searched by title."

The ADR does note the DB ∪ Index union is gated on a separate title-persistence effort and may not be fully implemented in the current codebase either, so this may already be a pre-existing gap rather than something this feature introduces. It's worth an explicit call before /speckit-plan

No other findings. No other ADRs in platform-adrs were found that this spec contradicts.

[KevinDavilaDotCMS]

@zJaaal — naming and placement both taken, in full.

Naming. "Search scope" everywhere in prose, never bare "scope", and searchScope on the wire. Recorded as Review Decision 5 with your reasoning, and Key Entities now points at the browse scope the way yours points here, so the pair reads unambiguously from either side: searchScope says how a search reads what is there, browseScope says where "there" is.

Placement — filters.searchScope, agreed. I checked AbstractQueryFilters and it is exactly as you describe: { text, filterFolders }, with filterFolders' own Javadoc reading "When text is provided". Both members exist to qualify the text search, and the search scope is a third of the same kind — it says which fields text reads and means nothing without text. Beside text they travel together. I took your "visibly nonsense rather than a validation rule someone has to remember" one step further and wrote it down as FR-025: a request carrying a search scope with no text is a contract error, not a silently-ignored field.

Your asymmetry argument is what convinced me the two placements are not inconsistent: the search scope should be cleared by "Clear all" — that is FR-020 — whereas a browse scope living in filters would let clearing your filters navigate you out of System Host. Same rule, opposite outcome, because one narrows what you see where you are and the other moves you. Review Decision 6 records both halves so nobody later reads this as "scopes go in filters".

Noted on assetPath: "//site/" already meaning "the whole site at any depth" and the picker sending exactly that. The general rule is the one this spec already leans on throughout — an existing payload keeps meaning what it means today — and it is why the search scope is additive and optional at every layer rather than a new reading of anything that already exists.

---

@ihoffmann-dot — ADR-0018 gets its own section and an explicit call rather than silence. Short version: deferred, with reasons, and I think the evidence is a little stronger than "possibly a pre-existing gap".

1. The ADR defers its own union. It states contentlet.title "is not reliably populated", that the display title is derived and Contentlet.title() is @Nullable, and that "populating contentlet.title reliably is a known gap to be addressed in a separate issue" — the ADR fixes the routing contract that will consume the column, not a claim that it is usable today.

2. No text search consults it today, in either scope. This is the part I verified rather than assumed, and it became Premise Correction 5. The browsing service has two text paths: Elasticsearch, or a SQL predicate running contentlet_as_json::text ILIKE '%token%' when useElasticsearchFiltering is false (BrowserAPIImpl.java:2053-2060, appendFilterQuery at :2222). The builder default is false, so the SQL path is the general fallback — but Content Drive never takes it: ContentDriveHelper.java:180-184 sets useElasticsearchFiltering(true) unconditionally whenever the request carries text. So contentlet text matching is index-only in both scopes, and the exposure to index lag is identical in All Fields and would be identical in Title. Narrowing the clause does not deepen it — an item missing from the index is missing from both scopes equally.

3. The union arrives for free. When the title-persistence work lands and the text path becomes DB-title ∪ index, Title scope is the scope that benefits most directly, because the column it unions in is the title. FR-026 makes that a requirement: Title scope must not foreclose the union, and adding the DB column later must be additive rather than a rewrite.

There is also a new edge case covering the scenario you said nothing covered — an item saved or renamed moments before the search, not yet indexed — stating plainly that it is missing in both scopes, as it is today.

Premise Correction 5 was worth the trip on its own: it rules out a second, SQL-shaped text match that a Title scope could have silently failed to narrow, which would have been a real hole in FR-010.

---

Pushed as 77abd40b77. PR title updated to match the rename. Both of you please re-review.

[KevinDavilaDotCMS]

Scope change: #37532 folded in — please re-review

Pushed 1ce8cdd1bf. The spec now resolves two issues. @zJaaal your approval predates this, so it needs another look.

#37532 (High, customer ticket 39185) reports that field filters return "no results found" for values containing Lucene query-syntax characters — ABC Bank (XETRA: DBKGn.DB / NYSE: DB) and similar. It was raised against the Content Search portlet, so at first glance it looked unrelated to this spec. It is not.

The same defect is in Content Drive's search box, in the exact clause FR-010 rewrites:

Defect	Content Drive	Evidence
Gate built from unescaped input	yes	GlobalSearchAttributeStrategy:37-38 uses the raw value; only :46-48 escapes, and only its own clause
/ missing from the escape set	yes	private SPECIAL_CHARS_TO_ESCAPE at :20 — this is #37532's fifth AC verbatim
Consecutive separators emit empty clauses	yes	:40-45 splits with no empty-token filter → a term-less title:^5
Parse failure swallowed → silent zero results	yes	BrowserAPIImpl:893-895, structurally identical to the ContentletAjax:1058 the issue names

Nothing sanitizes upstream — ContentDriveHelper:183 passes filters().text() straight through. So implementing #37479 alone would have meant rewriting the buggy clause and leaving the bug in it.

The issue also asks for this directly: "Content Drive is intended to replace the Content Search portlet, so the improved field-level search behaviour should land there."

What changed in the spec

• Premise Correction 6 — the evidence above, with the point that the spec's existing edge case ("a term is never allowed to alter the structure of the query") was aspirational, not a property the search box has.
• Why Content Search: escape Lucene query-syntax characters in content type field filters #37532 lands here — a per-AC table of what this spec delivers and what it does not.
• User Story 4 (P1, listed fourth) — the customer's headline, in both scopes.
• FR-027 to FR-031, SC-009 to SC-012 — literal matching across the full reserved set via the shared LuceneQueryUtils.escape rather than a strategy-private set, no empty clauses, error state instead of a silent empty list, and field-filter parity confirmed by test.

Two things I want you to push back on if you disagree

1. FR-009 and SC-002 had to yield. Escaping the gate changes All Fields results for affected terms. That contradicts the no-regression promise you approved, so rather than smuggle it in I wrote the carve-out down and bounded it: reserved characters and consecutive separators only, every other term byte-identical, and the tests that change enumerated in the PR. If you'd rather the escaping ride in its own PR to keep FR-009 absolute, say so — it's a defensible call, it just costs a second trip through this code.

2. The Content Search portlet is NOT fixed. ContentletAjax.searchContentletsByUser() keeps its current behaviour. The issue frames the legacy construction as something to replace rather than patch, which is the direction I followed — but the customer on ticket 39185 is using the portlet today, not Content Drive. Closing #37532 on this work is only honest if the team accepts that. I flagged it in Legacy Considerations and in Assumptions rather than assume it; it's the issue owner's call, not the spec's.

Also worth noting: the fix lands in GlobalSearchAttributeStrategy itself, which the Search portlet and the Relationships dialog also use, so they stop mis-parsing reserved characters too. That's deliberate (FR-031) and it's the one place this feature intentionally changes behaviour beyond Content Drive — unlike the Title scope, which stays a sibling path outside the shared strategy. Calling it out so it doesn't read as scope creep.

One thing that was already fine: Content Drive's field filters escape correctly today via TextFieldStrategy:45-46,53-54, so that AC is verification rather than implementation (FR-030 / SC-012).