@google-cloud/bigtable: a row-key-only read path that skips per-cell decode cost

[charles-yubo]

Library Name

@google-cloud/bigtable

A screenshot that you have tested with "Try this API".

Not applicable — this is a client-library implementation request, not a discrepancy between the client and the raw API. The ReadRows RPC already returns the CellChunk data needed (row key, family, qualifier, value); the request is about how @google-cloud/bigtable decodes and materializes that response client-side, which APIs Explorer can't exercise (it's not something visible in the raw wire response, only in the generated client's object graph).
If the form truly requires an image upload and rejects text, you may need to attach a blank/placeholder image or leave a follow-up comment explaining the same thing after submitting.

What would you like to see in the library?

Summary

For read patterns that only need the row key (not any cell value), the current client still pays the full per-cell decode cost: every CellChunk's familyName/qualifier gets unwrapped into a StringValue/BytesValue message, and the result is reassembled into a full, family-then-qualifier-grouped Row object. A row-key-only consumer never touches any of that.
We'd like to propose a keysOnly read mode that skips it.

Motivation

A possible id pattern is embedding a second identifier directly in the row key (e.g. ${id}${linkedId}, or ${id}${reverse-timestamp}${linkedId}), specifically so a range/prefix scan can answer "what is this entity linked to" without reading any column at all. We have several tables like this, and the read pattern is: scan a prefix, collect just the row keys, parse the linked id back out of each key.

Doing this today through getRows()/createReadStream() means paying for a full Row object per row: family map, qualifier map, cell values, all built and then discarded unread. A server-side StripValueTransformerFilter removes the value, but the client still decodes and reassembles everything else. We also checked the existing decode: false option on GetRowsOptions: it only controls whether a cell's value comes back as a Buffer or a string (Mutation.convertFromBytes), applied after the full Row is already built, so it doesn't help here either.

What we measured

We benchmarked three approaches to this exact pattern, same data, same requests:

1. Standard getRows()/createReadStream(), with a server-side filter (family scope + cellsPerColumnLimitFilter(1) + stripValueTransformer), reading only row.id from the result.
2. The same server-side filter, called via the low-level generated client directly (bypassing the Table wrapper), still using the standard generated response decoder.
3. A hand-written response decoder over a raw grpc-js connection to the same RPC, that reads only the rowKey and commitRow fields of each CellChunk (protobuf fields 1 and 9) and calls reader.skipType() on everything else, so no per-cell wrapper objects and no grouped Row are ever built.

On one real user's data, run sequentially with no server-side filter, comparing approach 2 against approach 3:

Rows	Without keysOnly	With keysOnly	Improvement
26,492	204.5ms	146.4ms	-28%
82,865	421.1ms	198.8ms	-53%
125,606	1539.4ms	646.1ms	-58%
756,377	4342.6ms	1722.4ms	-60%

The saving grows with row count, which is what we'd expect from removing a fixed per-cell decode cost: the more rows in the scan, the more per-cell wrapper-object construction there was to eliminate.

Across those same tables, approach 3 cut wall-clock time by 53-60% versus approach 2, and turned a workload that was clearly slower than a comparable Java client (using its own standard, fully-materializing read path) into one that matched or beat it — by 1.6x to 3.8x on the tables with the widest rows.

At 1,000 simulated users (capped at 100k rows on the largest table, matching a real production limit), approach 3 averaged 32.0ms per user vs. 42.0ms for the Java comparison, and beat it on p50/p90/p99 as well.

We also tried the newer executeQuery/GoogleSQL SELECT _key FROM ... path as an alternative, official way to get server-side projection down to just the key. On the same 1,000-user benchmark it was 4-8x slower than approach 3 on every percentile, so it isn't a substitute for this request.

Proposed API

Something in the shape of:

​js table.createReadStream({ prefix: '...', keysOnly: true }) // or table.getRows({ prefix: '...', keysOnly: true }) ​

returning row keys only (as Buffer/string), without constructing family/qualifier/value structures at all. Internally this could reuse the existing gapic readRows call and swap in a lighter response transform, without touching the generated protobuf layer.

Describe alternatives you've considered

• StripValueTransformerFilter removes the value but not the family/qualifier decode or the Row reassembly cost.
• decode: false on GetRowsOptions only changes the cell value's return type (Buffer vs string), applied after the full Row is already built.
• executeQuery (see above) is much slower for this pattern in our testing, likely because it runs a different, newer query-execution path server-side rather than the long-established ReadRows path.
• A private raw-grpc-js decoder, which is what we're using internally today, works, but means reimplementing retry/credential/channel-pooling logic that google-gax already provides, and isn't something we think every consumer of this client should have to build themselves.

Additional context/notes

Happy to contribute a PR if this is something the maintainers would want, or to share more of our benchmark methodology if useful.