Skip to content

Validate TTL and invalidation duration arithmetic consistently #33

Description

@lan17

Priority

P2 — Medium — numeric-boundary correctness hardening. An implementation is ready for review but is not merged or shipped.

Current state at v0.11.0

Static/default policy rejects malformed TTLs early, while malformed dynamic provider policy fails open during invocation. The remaining gap on released v0.11.0 is that local, Redis, tracked-watermark, and invalidation paths apply multiplication and addition at different boundaries.

Current released evidence:

  • TTL resolution:
    export function resolveLayerConfigResult(options: ResolveLayerConfigOptions): LayerConfigResolution {
    const config = options.config;
    if (config === null) {
    return { status: "disabled", reason: "policy_disabled" };
    }
    const ttlSec = config.ttlSec[options.layer];
    if (ttlSec === undefined) {
    return { status: "disabled", reason: "policy_disabled" };
    }
    if (!Number.isSafeInteger(ttlSec) || ttlSec <= 0) {
    return { status: "disabled", reason: "invalid_ttl" };
    }
    const configuredRampValue = config.ramp[options.layer];
    const configuredRamp = configuredRampValue === undefined ? 100 : configuredRampValue;
    if (!Number.isFinite(configuredRamp)) {
    return { status: "disabled", reason: "invalid_ramp" };
    }
    const ramp = clampPercentage(configuredRamp);
    if (ramp <= 0) {
    return { status: "disabled", reason: "ramped_down" };
    }
    if (ramp >= 100) {
    return { status: "enabled", config: { ttlSec, ramp } };
    }
    const sample = deterministicRampSample(options.key, options.layer);
    return sample < ramp
    ? { status: "enabled", config: { ttlSec, ramp } }
    : { status: "disabled", reason: "ramped_down" };
  • Process-local conversion:
    async put<T>(key: DialCacheKey, value: T, config?: { readonly ttlSec: number }): Promise<void> {
    const ttlSec = config?.ttlSec ?? await this.resolveLocalTtlSec(key);
    if (ttlSec === null) {
    return;
    }
    // lru-cache expires when age > ttl, while DialCache historically expired
    // when its integer-millisecond clock reached the configured boundary.
    const ttlMs = ttlSec * 1000 - 1;
    this.cache?.set(key.urn, { value }, { size: 1, ttl: ttlMs });
    }
  • Redis conversion:
    async put<T>(key: DialCacheKey, value: T, config?: { readonly ttlSec: number }): Promise<boolean> {
    const ttlSec = config?.ttlSec ?? await this.resolveRemoteTtlSec(key);
    if (ttlSec === null) {
    return true;
    }
    const start = performance.now();
    let serialized: string | Buffer;
    try {
    serialized = await this.serializerFor(key).dump(value);
    } catch (error) {
    this.recordError(key, "serialization_dump");
    throw error;
    } finally {
    this.recordMetric((metrics) => metrics.observeSerialization({ ...labelsFor(key, CacheLayer.REMOTE), operation: "dump" }, elapsedSeconds(start)));
    }
    this.recordMetric((metrics) => metrics.observeSize(labelsFor(key, CacheLayer.REMOTE), payloadSize(serialized)));
    try {
    const cacheTtlMs = ttlSec * 1000;
    if (!Number.isSafeInteger(cacheTtlMs)) {
    throw new RangeError("Redis cache TTL is too large");
    }
    const request = {
    valueKey: this.redisKey(key),
    cacheTtlMs,
    value: serialized,
    } as const;
    return key.trackForInvalidation
    ? await this.client.write({
    ...request,
    watermarkKey: this.redisWatermarkKeyFromKey(key),
    })
    : await this.client.write(request);
    } catch (error) {
    this.recordError(key, "cache_write");
    throw error;
    }
  • Tracked TTL and buffer sums:
    export const WRITE_TRACKED_CACHE_SCRIPT = [
    PARSE_WATERMARK_LUA,
    CEIL_FINITE_NUMBER_LUA,
    VALIDATE_WRITE_ARGUMENTS_LUA,
    REDIS_TIME_LUA,
    String.raw`local raw_watermark = redis.call("GET", KEYS[2])
    local watermark = 0
    if raw_watermark then
    watermark = parse_watermark(raw_watermark)
    if not watermark then
    return redis.error_reply("ERR invalid DialCache watermark")
    end
    end
    if watermark >= now_ms then
    return 0
    end`,
    WRITE_FRAME_LUA,
    String.raw`local desired_ttl_ms = cache_ttl_ms + ${WATERMARK_TTL_MARGIN_MS}
    if not raw_watermark then
    redis.call("SET", KEYS[2], "0", "PX", desired_ttl_ms)
    else
    local current_ttl_ms = redis.call("PTTL", KEYS[2])
    if current_ttl_ms == -2 then
    redis.call("SET", KEYS[2], raw_watermark, "PX", desired_ttl_ms)
    elseif current_ttl_ms ~= -1 and current_ttl_ms < desired_ttl_ms then
    redis.call("PEXPIRE", KEYS[2], desired_ttl_ms)
    end
    end`,
    "return 1",
    ].join("\n\n");
    export const INVALIDATE_CACHE_SCRIPT = [
    PARSE_WATERMARK_LUA,
    CEIL_FINITE_NUMBER_LUA,
    String.raw`local future_buffer_ms = ceil_finite_number(ARGV[1])
    if not future_buffer_ms or future_buffer_ms < 0 then
    return redis.error_reply("ERR invalid DialCache future buffer")
    end`,
    REDIS_TIME_LUA,
    String.raw`local proposed_watermark = now_ms + future_buffer_ms
    local raw_watermark = redis.call("GET", KEYS[1])
    local current_watermark = 0
    if raw_watermark then
    local parsed_watermark = parse_watermark(raw_watermark)
    if parsed_watermark then
    current_watermark = parsed_watermark
    end
    end
    local watermark = math.ceil(math.max(current_watermark, proposed_watermark))
    local current_ttl_ms = -2
    if raw_watermark then
    current_ttl_ms = redis.call("PTTL", KEYS[1])
    end
    local desired_ttl_ms = math.max(
    future_buffer_ms + ${WATERMARK_TTL_MARGIN_MS},
    watermark - now_ms + ${WATERMARK_TTL_MARGIN_MS}
    )
    if current_ttl_ms > desired_ttl_ms then
    desired_ttl_ms = current_ttl_ms
    end
    local encoded_watermark = string.format("%.0f", watermark)
    if current_ttl_ms == -1 then
    redis.call("SET", KEYS[1], encoded_watermark)
    else
    redis.call("SET", KEYS[1], encoded_watermark, "PX", desired_ttl_ms)
    end`,

#88 removed the fixed watermark-retention floor and made derived duration arithmetic the active contract, but deliberately did not choose the public maxima.

Proposed resolution under review

Ready pull request #96 proposes one shared fixed 365-day ceiling:

  • Cache TTL: 1 through 31,536,000 seconds.
  • Invalidation futureBufferMs: 0 through 31,536,000,000 milliseconds.
  • Static/default TTLs outside that range reject before fallback or serialization work.
  • Dynamic TTLs outside that range preserve fail-open behavior with invalid_ttl.
  • High-level invalidation rejects before Redis mutation.
  • Local and Redis conversions share one internal validation/conversion helper.
  • Exported Lua scripts independently enforce the same maximum for custom semantic adapters.
  • Exact lower, upper, upper-plus-one, and Number.MAX_SAFE_INTEGER cases are covered.

Live PR state on 2026-07-26:

  • Base: main@e06d833ba245706a499d66056fdad15dc1210b68
  • Head: 8ff528a9aef980828f358ea7dbb32749c73f80b6
  • Ready for review, mergeable, and all reported checks pass.
  • Merge is blocked on required review.
  • The change is not part of v0.11.0; this issue remains open until the implementation is reviewed and merged.

The separate runtime-ramp finding is tracked in #102: finite ramps below 0 or above 100 should be rejected rather than clamped into an active cache policy.

Acceptance criteria

  • Use the agreed shared one-year duration range in all affected paths.
  • Validate all relevant products and sums before fallback data reaches storage or invalidation mutates Redis.
  • Keep process-local, Redis, tracked-watermark, and exported Lua protocol handling consistent.
  • Reject invalid static values before fallback/serialization work.
  • Preserve dynamic-policy fail-open behavior with the existing bounded disabled/error telemetry.
  • Preserve explicit invalidation failure reporting.
  • Cover exact boundaries and direct semantic-client/Lua use.
  • Document the compatibility change.
  • Close only after the implementation merges.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions