Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 35 additions & 10 deletions reference/database/schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,18 @@ type MyTable @table {

Optional arguments:

| Argument | Type | Default | Description |
| -------------------- | --------- | ----------------------------- | ------------------------------------------------------------------------------------------- |
| `table` | `String` | type name | Override the table name |
| `database` | `String` | `"data"` | Database to place the table in |
| `expiration` | `Int` | β€” | Seconds until a record goes stale (useful for caching tables) |
| `eviction` | `Int` | `0` | Additional seconds after `expiration` before a record is physically removed |
| `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
| `replicate` | `Boolean` | true | Enable replication of this table |
| `cacheControl` | `String` | β€” | `Cache-Control` header value emitted on anonymous GET/HEAD 200/304 responses for this table |
| `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | [Pin this table's record encoding](#randomaccessfields) |
| Argument | Type | Default | Description |
| ------------------------ | --------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `table` | `String` | type name | Override the table name |
| `database` | `String` | `"data"` | Database to place the table in |
| `expiration` | `Int` | β€” | Seconds until a record goes stale (useful for caching tables) |
| `eviction` | `Int` | `0` | Additional seconds after `expiration` before a record is physically removed |
| `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
| `replicate` | `Boolean` | true | Enable replication of this table |
| `replicaExpiration` | `Float` | `3600` | Seconds a record fetched from another node is kept as a [local replica](#replicaexpiration-and-replicarefreshfraction); `0` stops new admissions |
| `replicaRefreshFraction` | `Float` | `0.5` | Fraction of `replicaExpiration` after which a read of a replica renews its lifetime; `1` never renews |
| `cacheControl` | `String` | β€” | `Cache-Control` header value emitted on anonymous GET/HEAD 200/304 responses for this table |
| `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | [Pin this table's record encoding](#randomaccessfields) |

**`expiration`, `eviction`, and `scanInterval`**

Expand Down Expand Up @@ -128,6 +130,29 @@ If the server starts at 12:05, the first eviction runs at 12:15 β€” not 12:20. T

Eviction removes non-indexed record data, but it does _not_ remove a record from its secondary indexes. If an evicted record matches a search query, Harper fetches the full record from the source on demand to satisfy the query. This means indexes remain fully functional even when most of the data has been evicted.

#### `replicaExpiration` and `replicaRefreshFraction`

<VersionBadge version="v5.4.0" />

<EngineBadge engines="RocksDB" />

Local replicas are RocksDB-only. On the deprecated LMDB engine, remote reads remain proxy reads: no replica is stored and no replica sweep runs, regardless of these settings. The options still validate on LMDB.

On RocksDB, schema rescans apply changes to already-loaded tables on other workers. The new lifetime applies to subsequent admissions and renewals; existing deadlines are not rewritten. Setting `replicaExpiration: 0` stops new admissions and the sweep; existing replicas remain until overwritten or evicted.

On a table whose records live on specific nodes (see [sharding](../replication/sharding.md#by-record-content-setresidency)), a node that does not hold a record fetches it from a node that does. The fetched record is kept locally as a _replica_: a complete copy that reads and searches on that node use without contacting another node again. A replica does not move the record and is never replicated onward. A write by an owner reaches the node as an invalidation, which turns the replica back into a stub, so the next read fetches the new version. Tables sharded by primary key only (`setResidencyById`) keep no replicas.

- **`replicaExpiration`** β€” Seconds a replica is kept. A sweep runs every quarter of this time, with a minimum interval of one second, and reduces replicas past their lifetime back to their indexed stubs. A replica can still be served until the sweep reaches it; the next read after demotion fetches the record again. `0` disables new replica admissions and the sweep. A replica never outlives the record's own `expiration`, if it has one.
- **`replicaRefreshFraction`** β€” A read of a replica after this fraction of `replicaExpiration` has passed renews the lifetime, so records that stay in use stay local. Must be greater than `0` and at most `1`; `1` gives every replica a fixed lifetime.

```graphql
# Keep fetched records for a day; a read after six hours renews them
type Order @table(replicaExpiration: 86400, replicaRefreshFraction: 0.25) {
id: ID @primaryKey
region: String @indexed
}
```

#### `randomAccessFields`

<VersionBadge version="v5.1.0" />
Expand Down
12 changes: 9 additions & 3 deletions reference/replication/sharding.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,13 @@ Once shards are configured, use `setResidency` or `setResidencyById` (described

Define a custom residency function that is called with the full record. Return an array of node hostnames or a shard number.

With this approach, record metadata (including residency information) and indexed properties are replicated to all nodes, but the full record is only stored on the specified nodes.
With this approach, record metadata (including residency information) and indexed properties are replicated to all nodes, but the full record is only stored on the specified nodes, apart from the short-lived replicas described next.

<VersionBadge type="changed" version="v5.4.0" />

<EngineBadge engines="RocksDB" />

On RocksDB, a node outside a record's residency that reads the record fetches it from a resident node and keeps the result as a local replica for `replicaExpiration` seconds (default one hour), renewed by reads after `replicaRefreshFraction` of that time. The record stays where the residency function placed it: a replica is never written back, replicated onward or counted as a resident, and a resident's write turns it back into a stub so the next read fetches the new version. See the [`@table` arguments](../database/schema.md#replicaexpiration-and-replicarefreshfraction). On the deprecated LMDB engine, remote reads are always proxy reads; no local replica is stored and no replica sweep runs.

Return node hostnames:

Expand All @@ -163,7 +169,7 @@ MyTable.setResidency((record) => {

### By Primary Key Only (`setResidencyById`)

Define a residency function based solely on the primary key. Records (including metadata) are only replicated to the specified nodes β€” metadata does not need to be replicated everywhere, which allows data to be retrieved without needing access to record data or metadata on the requesting node. This does not hold with [dedicated replication threads](./overview.md#dedicated-replication-threads) (`replication.threads` above `0`), which do not run the residency function.
Define a residency function based solely on the primary key. A read from another node is answered by a resident and keeps no local replica. Records (including metadata) are only replicated to the specified nodes β€” metadata does not need to be replicated everywhere, which allows data to be retrieved without needing access to record data or metadata on the requesting node. This does not hold with [dedicated replication threads](./overview.md#dedicated-replication-threads) (`replication.threads` above `0`), which do not run the residency function.

Return a shard number:

Expand All @@ -183,7 +189,7 @@ MyTable.setResidencyById((id) => {

## Disabling Cross-Node Access

By default, sharding allows data stored on specific nodes to be accessed from any node β€” requests are forwarded transparently. To disable this and only return data if it is stored on the local node, set `replicateFrom` to `false`.
By default, sharding allows data stored on specific nodes to be accessed from any node β€” requests are forwarded transparently. To disable this and only return data stored on the local node (including a replica it currently holds), set `replicateFrom` to `false`.

Via the operations API:

Expand Down
4 changes: 4 additions & 0 deletions release-notes/v5-lincoln/5.4.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ All patch release notes for 5.4.x are available on the [releases page](https://g

`replication.threads` (default `0`) starts worker threads that run replication only. They own the replication port and every subscription, so a peer catching up no longer competes with application requests. These threads load no application code, which limits applications that use `setResidencyById` or `setComputedAttribute` on indexed attributes. See [Dedicated Replication Threads](/reference/v5/replication/overview#dedicated-replication-threads).

### Local Replicas of Records Fetched From Another Node

On RocksDB, a table sharded with `setResidency` lets a node that reads a record it does not hold keep the fetched record as a local replica instead of moving it. The replica is served locally until an owner writes the record or its lifetime passes, set by the new `@table` arguments [`replicaExpiration` and `replicaRefreshFraction`](/reference/v5/database/schema#replicaexpiration-and-replicarefreshfraction) (default one hour, renewed by reads after half of it). Local replicas are RocksDB-only; LMDB continues to proxy remote reads without storing replicas or running a replica sweep.

## Subscriptions

### Resuming a Subscription From a Checked Position
Expand Down
Loading