Refget sequence endpoint declares the full sequence length as `Content-Length` on subsequence requests

[bencap]

Summary

GET /api/v1/refget/sequence/{alias} sets Content-Length to the length of the whole sequence on every response, including requests that ask for a subsequence. The streamed body is then shorter than the declared length, and the ASGI server aborts the response with RuntimeError: Response content shorter than Content-Length after the status line is already committed. Clients receive a truncated body.

Problem

In the refget router's get_sequence, the response headers are built from the sequence metadata rather than from the requested interval:

headers = {"Content-Length": str(seqinfo["len"])}

For GET /api/v1/refget/sequence/NC_000016.10?start=2077608&end=2085800, the body is 8,192 bytes (one DEFAULT_CHUNK_SIZE chunk from sequence_generator) while the header declares 90,338,345. Uvicorn raises when the stream ends short.

The traceback surfaces inside CatchAllErrorMiddleware, but the middleware is not at fault: response_started is already True, so it re-raises rather than converting a committed 200 into a 500. The failure is in the header the route set.

The defect is masked in two places:

• GZipMiddleware strips Content-Length from compressed responses, so any client sending Accept-Encoding: gzip never sees the mismatch. Only clients requesting identity encoding (e.g. curl without --compressed) hit it.
• TestClient sends Accept-Encoding: gzip by default, so no existing test could observe the header.

A second defect shares the same root: coordinate validation is gated on start is not None and end is not None, so a request supplying only one bound is never bounds-checked. ?start=99999999 alone streams an empty body under a 200 whose Content-Length is the full sequence length — the same crash by a different route.

The seqrepo router's get_sequence does not set Content-Length and therefore cannot produce this crash, but it shares the validation gap. Its only check is start > end → 422. Coordinates past the end of the sequence, and negative coordinates, are passed to sequence_generator unchecked and return a truncated or empty body under a 200.

Steps to reproduce

1. Request a subsequence with identity encoding:

curl -H 'Accept-Encoding: identity' \
  'https://api.mavedb.org/api/v1/refget/sequence/NC_000016.10?start=2077608&end=2085800'

2. Observe the server-side RuntimeError and the client-side incomplete read.

Expected: 200 with an 8,192-byte body and Content-Length: 8192.

Observed: RuntimeError: Response content shorter than Content-Length on the server; the client reads 8,192 bytes and reports IncompleteRead(8192 bytes read, 90330153 more expected) against a declared Content-Length: 90338345.

Requesting the same URL without Accept-Encoding: identity succeeds, because gzip removes the header.

Proposed behavior

Both sequence routes resolve the requested interval to concrete half-open coordinates once, immediately after the sequence length is known, and derive validation, headers, and the streamed body from those same values.

Refget route:

• Content-Length equals the number of bytes actually streamed (end - start), not the sequence length.
• Bounds validation runs whenever either bound is supplied, not only when both are.
• Content-Range and the generator bounds use the resolved coordinates.
• Out-of-range coordinates return 416 with a Content-Range: bytes */{length} header, matching the existing two-sided behavior.

Seqrepo route:

• Bounds validation runs whenever either bound is supplied.
• Out-of-range or negative coordinates return 422, consistent with the 422 this route already returns for start > end. This route has no Range header, Content-Range, or Content-Length semantics, so 416 would be the wrong code here even though refget uses it.

Acceptance criteria

• For every request shape against the refget sequence route — no bounds, both bounds, start only, end only, and a Range header — the returned Content-Length equals the byte length of the returned body. Tests must send Accept-Encoding: identity, or GZipMiddleware will strip the header and the assertion will not exercise the fix.
• A Range: bytes=1-2 request against a 4-base sequence returns 206 with Content-Range: bytes 1-2/4 and Content-Length: 2.
• Refget requests supplying only start, or only end, return the correct subsequence with a 200.
• Refget requests supplying only start beyond the sequence length, or only end beyond the sequence length, return 416 with a Content-Range header. These currently return an empty 200.
• Seqrepo requests supplying only start, or only end, return the correct subsequence with a 200.
• Seqrepo requests with coordinates outside the sequence — start=10&end=12, start=1&end=12, start=10, end=12, start=-1&end=2 against a 4-base sequence — return 422 with Invalid coordinates in the detail. These currently return a truncated or empty 200.
• Existing refget and seqrepo router tests continue to pass unchanged, including the two-sided 416 cases and the seqrepo start > end 422.

Implementation notes

• The shared sequence_generator helper in the seqrepo lib module already normalizes None bounds internally. Passing it resolved coordinates instead is equivalent but makes the length contract explicit at the call site, and removes the possibility of the route and the generator disagreeing about what interval is being served.
• Refget's start > end check raises 501 (circular chromosomes unsupported) and runs before the alias lookup, so invalid coordinates are rejected even for unknown aliases. Keep that ordering; only the length-dependent checks need to move below the lookup.
• Once bounds are resolved, the 206 branch condition can be reduced to if range_header: — the Range header regex requires both bounds, so the extra start is not None and end is not None conjuncts are always true when a header is present.
• sequence_generator breaks out of its loop on an empty chunk. For a validated range this can only happen on corrupt or missing seqrepo data, but it would silently truncate the body and reintroduce the Content-Length mismatch. Out of scope here; worth tracking separately if this error recurs on a range that passes validation.
• The refget sequence route is the only place in the codebase that sets an explicit Content-Length on a StreamingResponse. The NDJSON export routes in the score sets router stream without one and are unaffected.
• The seqrepo sequence route has no callers in the UI beyond its generated OpenAPI type, so the 200 → 422 change for out-of-range coordinates has a small blast radius.