Skip to content

Fix #2570,#2572: Allow upserts to recognize $and clauses - #2571

Draft
toptobes wants to merge 7 commits into
mainfrom
KG-upserts-recognize-and
Draft

Fix #2570,#2572: Allow upserts to recognize $and clauses#2571
toptobes wants to merge 7 commits into
mainfrom
KG-upserts-recognize-and

Conversation

@toptobes

@toptobes toptobes commented Aug 25, 2026

Copy link
Copy Markdown

What this PR does:

Updates Collection update operations to allow $eq filters nested inside of $ands to be included in the reconstructed document. See #2570 for a lot more information.

  • Filters are recursively traversed for all $and nodes, and "validated" when being reconstructed, throwing a new UNSUPPORTED_OVERLAPPING_UPSERT_PATHS error when contradictory paths are given, similar to Mongo's implementation.

This PR also addresses the TODO to have document reconstruction be inside of documentUpdater rather than split into two places.

  • Note though this wasn't just done for the sake of trying to resolve the TODO
  • It was necessary to allow us to control reconstruction based on the operation:
    • for update, we reconstruct the full document from filters, but
    • for replace, we only reconstruct only the _id. This prevents FindOneAndReplace operations from unnecessarily/wrongly throwing errors when contradictory filters exist outside the _id field.

Finally, this PR fixes #2572 which wrongly allows certain non-$eq fields to be included in the reconstructed document.

P.S. I left a comment for a minor optimization which could address one of aaron's comments + would remove an extra deepCopy in replace operations. Lmk if you just want me to do it in this PR, it's like a 4 line change

it should work, I just need to finish testing

Which issue(s) this PR fixes:
Fixes #2570 and #2572

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

📈 Unit Test Coverage Delta vs Main Branch

Metric Value
Main Branch 53.40%
This PR 53.48%
Delta 🟢 +0.08%
✅ Coverage improved!

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Unit Test Coverage Report

Overall Project 53.48% -0.06% 🍏
Files changed 79.02% 🍏

File Coverage
UpdateException.java 100% 🍏
DocumentUpdater.java 97.49% 🍏
ReadAndUpdateCollectionOperation.java 96.94% 🍏
FindCollectionOperation.java 83.62% -2.88% 🍏
UpdateOperation.java 75.21% 🍏
ArrayEqualsCollectionFilter.java 55.17% -44.83%
SubDocEqualsCollectionFilter.java 55.17% -44.83%
MatchCollectionFilter.java 0% -3.51%
IsNullCollectionFilter.java 0% -50%

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

📉 Integration Test Coverage Delta vs Main Branch (dse69-it)

Metric Value
Main Branch 71.45%
This PR 71.43%
Delta 🔴 -0.02%
⚠️ Coverage decreased

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (dse69-it)

Overall Project 71.43% -0.08% 🍏
Files changed 72.13% 🍏

File Coverage
UpdateException.java 100% 🍏
ReadAndUpdateCollectionOperation.java 95.48% 🍏
DocumentUpdater.java 92.48% 🍏
IsNullCollectionFilter.java 90.91% -9.09% 🍏
FindCollectionOperation.java 85.24% -6.34% 🍏
UpdateOperation.java 69.42% 🍏
ArrayEqualsCollectionFilter.java 55.17% -44.83%
SubDocEqualsCollectionFilter.java 55.17% -44.83%
MatchCollectionFilter.java 0% -3.51%

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

📉 Integration Test Coverage Delta vs Main Branch (hcd-it)

Metric Value
Main Branch 72.76%
This PR 72.74%
Delta 🔴 -0.02%
⚠️ Coverage decreased

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (hcd-it)

Overall Project 72.74% -0.08% 🍏
Files changed 72.13% 🍏

File Coverage
UpdateException.java 100% 🍏
ReadAndUpdateCollectionOperation.java 96.94% 🍏
DocumentUpdater.java 92.48% 🍏
IsNullCollectionFilter.java 90.91% -9.09% 🍏
FindCollectionOperation.java 90.31% -6.34% 🍏
UpdateOperation.java 80.99% 🍏
ArrayEqualsCollectionFilter.java 55.17% -44.83%
SubDocEqualsCollectionFilter.java 55.17% -44.83%
MatchCollectionFilter.java 42.11% -3.51%

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables upserts to reconstruct documents from equality filters nested in $and clauses.

Changes:

  • Recursively reconstructs upsert documents and validates overlapping paths.
  • Moves reconstruction into DocumentUpdater, with _id-only reconstruction for replacements.
  • Adds error handling and unit tests for reconstruction.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
DocumentUpdaterTest.java Updates calls for the new reconstructor parameter.
FindCollectionOperationTest.java Tests reconstruction and path validation.
errors.yaml Defines the overlapping-upsert-path error.
DocumentUpdater.java Integrates operation-specific reconstruction.
ReadAndUpdateCollectionOperation.java Delegates upsert reconstruction.
FindCollectionOperation.java Traverses $and filters and validates paths.
UpdateException.java Registers the new error code.
UpdateOperation.java Changes the action accessor return type.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +460 to +461
public ReadDocument newEmptyDocument() {
return ReadDocument.from(null, null, objectMapper().createObjectNode());

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug before my PR as well (as it wasn't guaranteed for an IDCollectionFilter to be present in getNewDocument(). My PR just amplifies the issue by never having the id set in the ReadDocument at all.

The issue is a deeper one which I'm not 100% sure how to solve

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may need to save writableShreddedDocument.id() and use that in readDocumentAgain()? I'm really not sure though

@toptobes toptobes changed the title Fix #2570: Allow upserts to recognize $and clauses Fix #2570,#2572: Allow upserts to recognize $and clauses Aug 25, 2026
@toptobes
toptobes force-pushed the KG-upserts-recognize-and branch from be38117 to 373fd29 Compare August 25, 2026 22:41
@toptobes
toptobes requested a balanced review from Copilot August 25, 2026 22:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants