Skip to content

fix(spanner): attach routing hints to read-write transaction requests - #14786

Draft
samyakpjgoogle wants to merge 2 commits into
googleapis:mainfrom
samyakpjgoogle:spanner-rw-txn-routing-hint
Draft

samyakpjgoogle wants to merge 2 commits into
googleapis:mainfrom
samyakpjgoogle:spanner-rw-txn-routing-hint

Conversation

@samyakpjgoogle

Copy link
Copy Markdown

Problem

When location-aware routing is enabled, KeyAwareChannel pins every request that belongs
to a read-write transaction to the transaction's affinity endpoint. In that case
routeFromRequest skipped ChannelFinder.findServer(...) entirely, so these reads and
queries were sent without a routing hint: no operation_uid, no key, no target range.

As a result:

  1. Key recipes are never learned for statements that only run inside read-write
    transactions.
    The server only returns key recipes for requests that carry an
    operation_uid, so the client can never build a routing hint for these statement shapes.
  2. These requests can never use the server fastpath, even when the key is local to the
    transaction owner. The server falls back to the slow path for every one of them.

In a TPC-C run on a 9-node Spanner Omni deployment (3 nodes per zone), only about 10% of
requests used the fastpath. With this change it went up to about 35%.

Fix

For requests pinned to a read-write transaction's affinity endpoint, still call
finder.findServer(reqBuilder, ...) so the routing hint (operation UID, database ID, schema
generation, key and target range) is populated. The endpoint returned by the finder is
ignored, so transaction affinity is unchanged and the request still goes to the
transaction owner.

The hint is attached even when the key's leader is not the affinity endpoint. In that case
the transaction owner can forward the request through the fastpath.

Tests

Added to KeyAwareChannelTest:

  • readWriteTransactionReadAttachesRoutingHintWhilePinnedToAffinityEndpoint
  • readWriteTransactionQueryAttachesRoutingHintWhilePinnedToAffinityEndpoint

Both pin a read-write transaction to server-a, send a read or query for a key owned by
server-b, and check two things: the request still goes to server-a, and it carries a
full routing hint (non-zero operation UID, database ID, key, group UID).

Requests that belong to a read-write transaction are pinned to the
transaction's affinity endpoint. In that case KeyAwareChannel skipped
ChannelFinder.findServer() and sent reads and queries without a routing
hint (no operation UID, key or target range). As a result the client
never learned key recipes for statements that only run inside read-write
transactions, and those requests could never use the server fastpath.

Populate the routing hint for these requests as well, while ignoring the
endpoint selected by the finder so that transaction affinity is
preserved.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request ensures that routing hints are populated for read-write transactions in KeyAwareChannel, even when they remain pinned to their affinity endpoints, with corresponding unit tests added. The review feedback points out an inconsistency in how the preferLeader state is managed and updated within the new conditional blocks for both read and query requests, and provides suggestions to align this behavior with the existing routing logic.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant