Repository navigation
fix(spanner): cancel queries promptly on close and synchronize error handling - #14887
Conversation
There was a problem hiding this comment.
Code Review
This pull request addresses concurrency issues and race conditions when closing result sets or transactions before a stream starts or a transaction ID is returned. It introduces safe handling of concurrent closures during stream initialization, prevents unnecessary RPC calls, and ensures that onError and onDone callbacks are invoked in the correct order without duplicates. Additionally, comprehensive unit tests are added to verify these concurrent behaviors. Feedback on the changes suggests simplifying a redundant conditional check in AbstractReadContext.java where onClosedBeforeStreamStarted is called.
…handling When a query or transaction is cancelled or closed while the network request is still being established, the underlying gRPC stream was previously left unregistered, allowing the query to continue running in the background and wasting database and network resources. Additionally, closing a query concurrently with an error could cause completion callbacks to race ahead of error handlers, corrupting transaction state and masking the underlying failure. This change fixes several concurrency edge cases in query execution: - Prompt query cancellation: Registers streams immediately so that queries cancelled or closed during startup are aborted right away, or skipped entirely if already closed, preventing redundant network RPCs. - Clean failure for early closes: When a query that initiates a transaction is closed before the request finishes, waiting callers fail fast with a clear error rather than hanging indefinitely for a transaction ID. - Coordinated error and close callbacks: Guarantees that query errors are fully dispatched before completion notifications run, ensuring transactions observe the actual database failure and can retry appropriately instead of being masked by a generic "result set closed" error. - Fast failure on repeated reads: Once a query stream encounters an error, subsequent read calls immediately surface the original failure rather than attempting further operations.
267a910 to
c8c9b3c
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request addresses concurrency issues and race conditions when closing a ResultSet or transaction concurrently with stream initialization or error dispatching in the Google Cloud Spanner Java client. Key changes include tracking stream state via setStream to prevent unnecessary RPC calls if closed concurrently, deferring onDone notifications until onError callbacks complete to avoid race conditions, and caching exceptions in GrpcResultSet to ensure subsequent next() calls consistently throw the same error. Extensive unit tests have been added to verify these concurrent behaviors and prevent regressions. I have no feedback to provide as there are no review comments.
…heck (#14896) Fix generation diff, unblocks the Librarian upgrade and API sources regen. ## What Runs `google-java-format` 1.25.2 on two handwritten test files, so their long string literals are reflowed the same way `librarian generate --all` reflows them: - `java-storage/.../it/ITGrpcDirectPathTest.java`: `@Ignore` message (added in #14006) - `java-spanner/.../AbstractReadContextTest.java`: `FAILED_PRECONDITION` message (added in #14887) The result is identical to the diff in the failing [generation check run](https://github.com/googleapis/google-cloud-java/actions/runs/41497486252) (same blob hashes: `06333bef`, `bc58e312`). These are formatting-only changes, with no behavior change. ## Why the PRs passed lint but broke `main` - **PR `lint` job:** runs `fmt-maven-plugin` 2.25. Its `Formatter` calls `formatSource()` and then only fixes imports. It never reflows long strings (`StringWrapper.wrap`). - **`librarian generate`:** runs the `google-java-format --replace` CLI, where `reflowLongStrings` defaults to `true`. It runs on every `.java` file in each library, including handwritten tests. - **`librarian_generation_check.yaml`:** runs only on push to `main`, so the mismatch shows up only after merge. Both tools use google-java-format 1.25.2, so the version isn't the cause. The difference is the long-string reflow step. This will happen again whenever a handwritten string literal is longer than 100 columns, until the two formatters agree. That follow-up is out of scope for this PR. Fixes #14862
🤖 I have created a release *beep* *boop* --- <details><summary>1.95.0</summary> ## [1.95.0](v1.94.0...v1.95.0) (2026-11-02) ### Features * **bigquery-jdbc:** expose BigQueryResultSetMetadata and getField(int) ([#14761](#14761)) ([f01245a](f01245a)) * **gax:** align resumable upload chunk size to server granularity and surface effective chunk size ([#14670](#14670)) ([6cbb096](6cbb096)) * **google/cloud/designcenter/v1:** add designcenter ([#14823](#14823)) ([01d27dc](01d27dc)) * **google/cloud/observability/v1:** add observability ([#14722](#14722)) ([624f28e](624f28e)) * **oauth2:** support certificate-bound OAuth2 tokens and mTLS rotation for Workload Identity Federation ([#14781](#14781)) ([dcb29b0](dcb29b0)) * **storage:** add bucket IP filtering samples and tests ([#14328](#14328)) ([cb2b726](cb2b726)) * **storage:** add support for DirectPath over Interconnect ([#14006](#14006)) ([ad0eb94](ad0eb94)) * **storage:** OpenTelemetry client metrics configuration and options ([#14484](#14484)) ([7c29e48](7c29e48)) * **storage:** OpenTelemetry client metrics instrument registry and views ([#14398](#14398)) ([60233b7](60233b7)) ### Bug Fixes * add user agent to session path ([#14835](#14835)) ([a653638](a653638)) * **auth:** align mTLS transport and bound token certificate startup polling and well-known discovery ([#14748](#14748)) ([0bae268](0bae268)) * **bigquery-jdbc:** expand handling to OOM errors ([#14759](#14759)) ([7d4b921](7d4b921)) * **bigquery-jdbc:** fix naive fallback of parameter count ([#14779](#14779)) ([c3e4a5b](c3e4a5b)) * **bigquery-jdbc:** format RANGE bounds consistently across REST and Arrow ([#14807](#14807)) ([2108b8e](2108b8e)) * **bigquery-jdbc:** parse scientific-notation epoch TIMESTAMPs ([#14777](#14777)) ([71a31b1](71a31b1)) * **bigquery-jdbc:** preserve pre-1582 and pre-standard-time DATE/DATETIME digits ([#14787](#14787)) ([3dc394f](3dc394f)) * **bigtable:** avoid mutating Query in-place in createPointReadCallable ([#14797](#14797)) ([1b1c6a7](1b1c6a7)) * **bigtable:** check isOk and getRejected first in RetryingVRpc.shouldRetry ([#14795](#14795)) ([d821bd3](d821bd3)) * **bigtable:** check remaining deadline on client-initiated retries in RetryingVRpc ([#14796](#14796)) ([a04e24c](a04e24c)) * **bigtable:** Fix empty RowResult semantics in TestProxy. ([#14810](#14810)) ([1d2fb9a](1d2fb9a)) * **bigtable:** include reversed in Query equals, hashCode, and toString ([#14799](#14799)) ([fe9232b](fe9232b)) * **bigtable:** pass cell value size to RowBuilder.startCell in RowBuilderShim ([#14800](#14800)) ([cfa7949](cfa7949)) * **bigtable:** reject empty split points in RowSetUtil and clarify QueryPaginator Javadoc ([#14811](#14811)) ([1bc507f](1bc507f)) * **bigtable:** remove legacy PollingConfiguration migration in ClientConfigurationManager ([#14822](#14822)) ([95ead45](95ead45)) * **bigtable:** resolve session diversion through a single shared rule ([#14738](#14738)) ([00f7bea](00f7bea)) * **bigtable:** respect query limit smaller than pageSize and avoid mutating Query in QueryPaginator ([#14821](#14821)) ([cecd09e](cecd09e)) * **firestore:** graduate pipeline search from preview to GA ([#14741](#14741)) ([a993026](a993026)) * **gax:** do not treat a SocketTimeoutException as a thread interrupt ([#14778](#14778)) ([7b91805](7b91805)) * **jdbc:** populate `ResultSetMetaData` on empty `DatabaseMetaData` resultsets ([#14809](#14809)) ([4c6a7e3](4c6a7e3)) * **pubsub:** make publish hedging robust to clock jumps, queue errors, and shutdown ([#14860](#14860)) ([9b9e2a5](9b9e2a5)) * **spanner:** attach routing hints to read-write transaction requests ([#14786](#14786)) ([54e0585](54e0585)) * **spanner:** cancel queries promptly on close and synchronize error handling ([#14887](#14887)) ([323a8f7](323a8f7)) * **spanner:** do not cache retryable errors in AbstractLazyInitializer ([#14752](#14752)) ([6a9ac76](6a9ac76)) * **spanner:** restore setCall overload for stream test compatibility ([#14885](#14885)) ([cfe728e](cfe728e)) * **spanner:** synchronize stream lifecycle and retry buffering in ResumableStreamIterator ([#14770](#14770)) ([4edefc5](4edefc5)) * **spanner:** trigger best-effort rollback on indeterminate commit failures ([#14831](#14831)) ([b9d1b95](b9d1b95)) * **storage:** preserve HTTP status for XML multipart upload errors so retries work ([#14804](#14804)) ([3c03034](3c03034)) * **storage:** send destination on gRPC copy when target BlobInfo is provided ([#14784](#14784)) ([43a67a0](43a67a0)) * **storage:** validate CRC32C on HTTP downloads that were not decompressed ([#14805](#14805)) ([61aa22f](61aa22f)) ### Documentation * **bigtable:** document that skipped large rows count towards row limit ([#14801](#14801)) ([91d60e0](91d60e0)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
When a query or transaction is cancelled or closed while the network request is still being established, the underlying gRPC stream was previously left unregistered, allowing the query to continue running in the background and wasting database and network resources. Additionally, closing a query concurrently with an error could cause completion callbacks to race ahead of error handlers, corrupting transaction state and masking the underlying failure.
This change fixes several concurrency edge cases in query execution: