Skip to content

Add baseBackoffMS support - #2052

Open
vbabanin wants to merge 11 commits into
mongodb:backpressurefrom
vbabanin:baseBackoffMS-support
Open

vbabanin wants to merge 11 commits into
mongodb:backpressurefrom
vbabanin:baseBackoffMS-support

Conversation

@vbabanin

@vbabanin vbabanin commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary

This PR brings the driver's Client Backpressure implementation up to date with the latest
specification changes: the server-supplied baseBackoffMS override, the corrected
exponential backoff formula exponent, and the versioned backpressure: "2" handshake
flag. Also adds MongoDB 9.0 to the Evergreen version matrix so the new 9.0-gated
prose test has a CI variant to run against.

Tickets

JAVA DRIVERS Description
JAVA-6238 DRIVERS-3535 Client Backpressure with baseBackoffMS — backoff formula exponent, baseBackoffMS server override, handshake backpressure: "2"
JAVA-6255 DRIVERS-3578 Correct sum of backoffs for the convenient transactions API backoff prose test
JAVA-6265 DRIVERS-3584 Provide updated retryable overload error code examples for 9.0 — out of scope for this PR (documentation deliverable, tracked separately as TODO)

Upstream spec commits:

Commits

Commit Ticket Summary
a8c755e829 JAVA-6238 use retry number as backoff exponent
407b3f0430 JAVA-6255 correct withTransaction backoff test sum to 2.3s
ac31095dc9 JAVA-6238 update overload backoff prose test to 0.6s sum per new formula
9b6d1a4995 JAVA-6238 send backpressure: "2" in handshake
64d1a438b1 JAVA-6238 support server-supplied baseBackoffMS override for overload backoff
d5377190cd JAVA-6238 add baseBackoffMS override prose test (Test 5)
7a42e3ac5b JAVA-6238 add MongoDB 9.0 to Evergreen version matrix
5472e7b055 JAVA-6238 extract baseBackoffMS field name to a constant

Changes

Backoff formula exponent (JAVA-6238 / DRIVERS-3535)

The spec changed the exponential backoff to use the retry number directly as the
exponent, instead of one less than the retry number:

  • Overload retry: jitter * min(MAX_BACKOFF, BASE_BACKOFF * 2^attempt) (was 2^(attempt-1))
    → first retry 200ms, second 400ms (was 100ms / 200ms).
  • Convenient transactions API: jitter * min(BACKOFF_INITIAL * 1.5^attempt, BACKOFF_MAX)
    (was 1.5^(attempt-1)).

Both share ExponentialBackoff.calculateBackoffMs, so a single-line exponent fix
covers both. BASE_BACKOFF remains 100ms; MAX_BACKOFF remains 10000ms.

Server-supplied baseBackoffMS override (JAVA-6238 / DRIVERS-3535)

When an overload error document contains a positive top-level baseBackoffMS field,
the driver now uses it in place of the 100ms BASE_BACKOFF constant:

  • New ExponentialBackoff.calculateOverloadBackoff(int, @Nullable Long baseBackoffMs)
    overload. A null or non-positive value falls back to the 100ms default; a positive
    value replaces it. The 10000ms cap is preserved.
  • SpecRetryPolicy extracts baseBackoffMS from MongoCommandException.getResponse()
    and passes it through. Extraction is lenient: absent, non-numeric, or non-positive
    values are ignored.

Handshake backpressure: "2" (JAVA-6238 / DRIVERS-3535)

The handshake now sends backpressure: "2" — the BSON string "2", not a boolean
and not the number 2 — to version the supported backpressure specification. Previously
the driver sent backpressure: true. The value is sent unconditionally on every hello
(no server-version gating), matching the Python and Node reference implementations;
the flag is currently unused by the server and older servers ignore unexpected
handshake field values.

Test updates

  • Prose Test 1 (BackpressureProseTest.operationRetryUsesExponentialBackoff):
    expected overload backoff sum 0.3s → 0.6s.
  • New prose Test 5 (BackpressureProseTest.overloadErrorsWithBaseBackoffMsOverrideBaseBackoff):
    verifies baseBackoffMS overrides the default backoff. Gated on
    serverVersionAtLeast(9, 0); pins jitter to 1, toggles
    externalClientBaseBackoffMS via setParameter, and asserts timing bounds
    (default run ≥ 600ms; override run ≥ 300ms and < 600ms).
  • Handshake prose Test 9 (AbstractClientMetadataProseTest): asserts the
    backpressure field is the string "2".
  • withTransaction backoff test (WithTransactionProseTest.testRetryBackoffIsEnforced):
    expected sum of 13 backoffs 1800ms → 2300ms.
  • Unit tests (ExponentialBackoffTest): updated the transaction backoff expected
    array for the new exponent; added 4 tests covering the baseBackoffMS override
    (default-when-absent, positive-override, non-positive-ignored, max-cap-preserved).
  • Groovy unit spec (InternalStreamConnectionInitializerSpecification): updated
    the 4 handshake expectations to backpressure: "2".

Evergreen (JAVA-6238)

Added a 9.0 entry to the version axis in .evergreen/.evg.yml and included 9.0
in all test matrix variants that currently test against 8.0, mirroring the Python
and Node drivers. This gives the 9.0-gated prose Test 5 a pinned CI variant. The
Atlas-specific MONGODB_VERSION: "8.0" deploy target is left untouched (Atlas
manages its own server version).

@vbabanin vbabanin closed this Sep 11, 2026
@vbabanin vbabanin reopened this Sep 11, 2026
@vbabanin
vbabanin force-pushed the baseBackoffMS-support branch from a424287 to 1e202c4 Compare September 11, 2026 00:03
- Bypass the retryable-write server-capability break when all observed
  failures are retryable overload errors (load-shed, never executed)
- Track the cumulative all-overload flag on the retry policy itself,
  keeping the session flag only for startTransaction attachment

JAVA-6238
@vbabanin
vbabanin requested a review from jyemin September 14, 2026 16:38
@katcharov
katcharov marked this pull request as ready for review September 14, 2026 17:02
@katcharov
katcharov requested a review from a team as a code owner September 14, 2026 17:02

@jyemin jyemin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looking good to me. Just one nit in the prose test ordering.

* @see MongoException#RETRYABLE_ERROR_LABEL
* @see MongoException#SYSTEM_OVERLOADED_ERROR_LABEL
*/
private boolean observedNoneOrOnlyRetryableOverloadErrors;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I apologize for leaving questions without really reviewing the code. If they are stupid, please ignore.

  1. Is it true that we can't reuse DefaultCommandExecutionScoped.observedNoneOrOnlyRetryableOverloadErrors instead of introducing this new field?
  2. If the answer to the above questions is "yes", maybe we then could use this new field instead of DefaultCommandExecutionScoped.observedNoneOrOnlyRetryableOverloadErrors? The duplication (at least, that is how it looks like at first glance) is concerning.

@vbabanin
vbabanin requested a review from jyemin September 15, 2026 17:10

@jyemin jyemin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants