From d0d81b2fac5ef6a510c502a2a2eb18c2c24e49c4 Mon Sep 17 00:00:00 2001 From: Iliyan Velichkov Date: Wed, 19 Aug 2026 13:05:44 +0300 Subject: [PATCH] proposal: mapping on arrival - a business key becomes a relation, and a gate ignores what is not understood An `inbound` entry ingests the arriving JSON as the entity, which is only expressible when the sender's payload already IS the entity, field for field. A real arrival contract is an envelope, so an application can declare its entity, its deduplication, its process trigger and its arrival - and still need custom code purely because a relation arrives as a name instead of a reference. Proposes two optional keys, valid on every arrival because they describe the payload rather than the transport: `accept` gates on the envelope keys it names, and `map` fills a property from an envelope key or - given `{ lookup, by, from }` - from the record a business key identifies. The rules carry their reasons. `by` must identify at most one record, because a lookup that could match several would have to pick one and picking silently is worse than failing. A lookup matching none rejects the arrival, because a record stored with an unresolved reference cannot be traced back to the party that asked for it. A message the gate does not match is acknowledged and ignored rather than failed, because redelivery cannot change the outcome. Mirrors 0017's argument in the other direction: the format admitted a declared envelope for messages leaving on the grounds that real contracts are not records, and an arriving message is the same contract read the other way. Proven by the reference implementation in eclipse-dirigible/dirigible#6769. --- proposals/0021-arrival-mapping.md | 185 ++++++++++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 proposals/0021-arrival-mapping.md diff --git a/proposals/0021-arrival-mapping.md b/proposals/0021-arrival-mapping.md new file mode 100644 index 0000000..8efb36f --- /dev/null +++ b/proposals/0021-arrival-mapping.md @@ -0,0 +1,185 @@ +# Mapping on arrival — a business key becomes a relation, and a gate ignores what is not understood + +- **Status:** draft +- **Issue:** [IntentFile/intent-specification#30](https://github.com/IntentFile/intent-specification/issues/30) +- **Implementation:** [eclipse-dirigible/dirigible#6769](https://github.com/eclipse-dirigible/dirigible/issues/6769) + ([PR #6839](https://github.com/eclipse-dirigible/dirigible/pull/6839)) +- **Companion:** [`0017-declared-payload.md`](0017-declared-payload.md) — the same problem in the + other direction, whose value vocabulary this one mirrors; and + [`0018-outbound-departures.md`](0018-outbound-departures.md), whose own issue named this half. + +## The problem + +An `inbound` entry ingests the arriving JSON **as the entity**. That is only expressible when the +sender's payload already *is* the entity, field for field — and a real arrival contract is not. It is +an envelope, frozen between two products long before either one modelled anything: + +```json +{ "messageId": "9f9d1c9e-...", "type": "user.assignment.requested", "version": 1, + "tenantId": "acme", "appId": "library", "email": "new.user@example.com", "role": "User" } +``` + +Ingesting that needs three things the construct cannot say: + +1. **Field mapping** — the envelope's keys are not the entity's names. +2. **A business key resolved to a relation** — `tenantId: "acme"` and `role: "User"` identify the + `Tenant` and the `AssignmentRole` records, and what the entity stores are references to them. This + is the single most common requirement of any arrival, and on its own it is enough to force a + hand-written consumer. +3. **A type and version gate** — accept `user.assignment.requested` at version 1, and *ignore* + anything else with a warning rather than failing it, so a sender rolling out version 2 does not + fill the receiver's error queue with messages it will keep sending. + +The consequence is the one worth avoiding: an application can declare its entity, its deduplication, +its process trigger **and** its arrival, and still need custom code purely because a relation arrives +as a name instead of a reference. The construct looks like it covers the case, and the hand-off is +discovered when the first payload lands. + +The asymmetry is also hard to defend on its own terms. The format admitted the declared envelope for +messages **leaving** ([`0017`](0017-declared-payload.md)) on the argument that real contracts are not +records; an arriving message is the same contract read in the other direction, and until now it had to +be a record. + +## The proposed shape + +Two optional keys on an `inbound` entry, valid on **every** arrival, because they describe the payload +rather than the transport it came over. + +```yaml +inbound: + - name: userAssignments + source: { queue: "codbex.user-assignment-requests" } + accept: { type: user.assignment.requested, version: 1 } # anything else: warn and ignore + create: TenantUserAssignment + map: + messageId: messageId # entity field <- envelope key + email: email + seats: seatCount + tenant: { lookup: Tenant, by: tenantId, from: tenantId } # business key -> relation + role: { lookup: AssignmentRole, by: name, from: role } +``` + +`map:` omitted keeps the current behaviour exactly — the payload is the record — so no existing file +changes meaning and the simple case stays one line. + +## Expected behaviour + +**The gate.** `accept:` names envelope keys and the value each must carry. A message matching all of +them is ingested. A message that does not match is **acknowledged and ignored**, with a warning +recording what arrived and what was expected. It is deliberately *not* failed: failing an arrival asks +the sender to deliver it again, and a message this receiver structurally does not understand will be +identical every time. Over a request/response arrival the answer says the same thing — the message was +received and not acted on — rather than reporting a fault the sender did not commit. + +**The projection.** `map:` fills the named property of `create:` from the named envelope key. A key the +map does not mention is not part of the record; an envelope key that is absent leaves its property +unset, which the entity's own required-value rules then judge like any other missing value. The +authored order is irrelevant to the result. + +**The lookup.** `{ lookup: , by: , from: }` reads the envelope key, finds +the `lookup` entity whose `by` field equals it, and stores a reference to that record in the mapped +relation. Exactly one match fills the relation. **No match rejects the whole arrival** — with a +diagnostic naming the value that could not be resolved — because a record stored with an unresolved +reference is a record nobody can trace back to the party that asked for it, and it is indistinguishable +from one that never named a party at all. + +**The write path is unchanged.** Whatever is declared, the record is saved through the entity's +ordinary write path, so validations, translations and the create event behave exactly as for any other +write. Mapping changes what the record is built from, never how it is stored. + +## Edge rules + +- **`by:` must identify at most one record** — a field declared unique, or the entity's key. A lookup + that could match several rows would have to pick one, and picking silently is a worse outcome than + failing; so a non-unique `by:` is refused when the file is read, not when two rows first collide. +- **`by:` must be a field a business key can travel as** — text or an integer. A temporal key would + make the comparison depend on how the sender formatted a date, which is not a key. +- **A lookup fills a to-one relation only.** On a plain field it is meaningless: there is nothing to + resolve to. +- **`lookup:` must name the relation's own target.** Two ways of saying which entity is read could only + drift apart. +- **A mapped key must be a field or a to-one relation of `create:`.** A key naming neither is an + authoring error, reported with the nearest declared name (per + [`0014`](0014-unknown-keys-are-errors.md)). +- **A lookup value's keys are closed** — `lookup`, `by`, `from`, all three required. An unrecognised + one is an error. +- **The entity's key MUST NOT be mapped.** It is assigned when the record is created. An arrival that + carries its own identifier gives it a field of its own, declared unique — which is also what makes a + redelivery of the same message refuse itself. +- **A valueless key is an error, not an empty one.** `accept: { type: }` and `map: { email: }` are + reported. Read as "no value", the first would gate on nothing — accepting every message — and the + second would fill nothing, in both cases losing an authored promise silently. +- **`accept:` values are scalars.** A nested object or list is not a gate. +- **A plain envelope key on a to-one relation** copies the target's key as it arrives, which remains + valid: a sender that already speaks in references needs no lookup. +- **Same-model targets in this revision.** `lookup:` reads an entity declared in the same model. + Resolving a business key against a model that only declares the entity by reference is a larger + question and is left out deliberately rather than half-specified. + +## Prior art / workarounds + +A hand-written subscriber or endpoint alongside the generated application: parse the envelope, query +each register by its business key, build the entity, save it through the generated write path. It is +mechanical, it is written again in every application that consumes the same contract, and — because the +contract and the resolution live in that code — the arrival is invisible to anyone reading the model. + +Some formats offer a general transformation language for this. That is the trade this proposal +deliberately does not make: three value forms and one lookup shape express the contracts people +actually have, while a transformation language moves the arrival out of the model and into a second +program written in a worse language. An arrival that needs more than this is an algorithm, and the +honest answer is a hand-written handler over the same write path. + +## Specification text + +**Anchor:** Declarative glue > `inbound — arrivals from outside` (immediately after that section's +normative block) + +An arrival may declare how its payload is **read**. Without these keys the payload is the record; with +them it is an envelope, which is what an integration contract normally is. + +```yaml +inbound: + - name: userAssignments + source: { queue: "user-assignment-requests" } + accept: { type: user.assignment.requested, version: 1 } + create: TenantUserAssignment + map: + messageId: messageId + seats: seatCount + tenant: { lookup: Tenant, by: tenantId, from: tenantId } +``` + +`accept` gates the arrival on the envelope keys it names. `map` fills a property of `create` from an +envelope key, or — given `{ lookup, by, from }` — from the record of another entity that a **business +key** identifies: the envelope carries a name, the entity stores a reference. + +> **Normative.** +> `accept` and `map` are independent, both optional, and both valid on every arrival — they describe +> the payload, not the transport. An arrival declaring neither behaves exactly as one specified before +> them. +> +> A message matching every key of `accept` is ingested. A message that does not match MUST be +> acknowledged and ignored, with a diagnostic; it MUST NOT be failed, since redelivery cannot change +> the outcome. `accept` values are scalars. +> +> Each key of `map` MUST name a field or a to-one relation of `create`; the entity's key MUST NOT be +> mapped, being assigned on creation. A value is an envelope key, or a lookup declaring exactly +> `lookup`, `by` and `from`. A lookup MUST fill a to-one relation, `lookup` MUST name that relation's +> target, and `by` MUST name a field of the target that identifies at most one record — a field +> declared unique, or the target's key — and one that a business key can travel as, namely text or an +> integer. A generator MUST refuse a `by` that does not, rather than resolve a lookup that could match +> several records. +> +> A lookup matching exactly one record fills the relation. A lookup matching none MUST reject the +> arrival, reporting the unresolved value; it MUST NOT store the record with the relation unset. A key +> declared with no value at all — in either block — MUST be reported as an authoring error rather than +> read as an empty one. +> +> A mapped record MUST be saved through the entity's ordinary write path, exactly as an unmapped one +> is: mapping changes what the record is built from, never how it is stored. + +## DSL index + +| Construct | What it gives you | +| --- | --- | +| [`inbound.accept` / `inbound.map`](#inbound--arrivals-from-outside) | read an arrival as an envelope: gate on its type and version, map its keys onto the record, resolve a business key to a relation |