[proposal] processes: declarative step resilience - retry, error routing, and step data

[delchev]

The problem

A provisioning flow — create a tenant schema, register a client in the identity provider, call the
application's provisioning API, record the outcome — is structurally declarative: every step is a
serviceTask whose work is a delegate, and the routing is a plain chain. It still cannot be
written as processes: today, because what makes it a provisioning flow is not the steps but their
failure semantics:

• each remote step must be retried a few times, spaced out — a transient 503 from the identity
  provider must not fail a tenant's onboarding;
• when the retries are exhausted the record must land in a failed status carrying the message,
  rather than leaving a stuck instance for an operator to discover;
• a value generated in one step (a database password) must reach a later step, and must not survive
  in the instance data afterwards.

The step vocabulary is userTask / serviceTask / decision / script / wait / end, with
setField / setRelationField / notify / delegate. Time is expressible on a user task
(timeout / expire), and a terminal status can cancel an instance (abortOn) — but there is
nothing about a step that fails: no retry declaration, no error routing, no declared step data.
What happens when a delegate throws is left entirely to the runtime's defaults, invisible in the
file. A model that reads as a complete flow therefore silently isn't one, and the whole process has
to be hand-written to get three attributes back.

This sits inside the scope boundary: the delegate's body is an algorithm and stays code. What is
missing is the structure around it, which is model.

The proposed shape

processes:
  - name: TenantProvisioning
    trigger: { onCreate: TenantApplication }
    vars:
      - { name: dbPassword, clearAfter: provisionApp }     # produced by a step, gone after this one
    steps:
      - name: createSchema
        kind: serviceTask
        args:
          delegate: SchemaProvisioner
          produces: [dbPassword]
          retry:   { count: 3, every: PT30S }
          onError: recordFailure

      - name: provisionApp
        kind: serviceTask
        args:
          delegate: AppProvisioner
          uses:    [dbPassword]
          retry:   { count: 5, every: PT1M }
          onError: recordFailure
          next: done

      - name: recordFailure
        kind: serviceTask
        args: { setField: failureMessage, value: "{error}", next: markFailed }

      - name: markFailed
        kind: serviceTask
        args: { setRelationField: ProvisioningStatus, value: Failed, next: end }

      - { name: done, kind: end }

Expected behaviour

Normative, stated platform-neutrally.

• retry: { count: <n>, every: <ISO-8601 duration> } on a step that can fail (a serviceTask
  carrying a delegate, or a step whose work is an outbound call): the step is re-attempted up to
  count further times, spaced by every, before it counts as failed. count is at least 1;
  every uses the same duration vocabulary as timeout.after. A step without retry behaves
  exactly as today, so every existing file is unchanged.
• onError: <step> routes a failed step — after its retries are exhausted, or immediately when
  it declares none — to a declared step instead of leaving the instance stalled. It must name a
  declared step or the literal end, parse-validated like next / then. Without onError, the
  failure surfaces as the runtime's own incident: today's behaviour, deliberately unchanged.
• {error} interpolates the failure message, and is resolvable only on a step reachable from an
  onError route. A generator rejects it elsewhere rather than emitting an empty string.
• vars: declares the instance data steps exchange. produces: / uses: name declared vars
  only — an unknown name is a parse error, so a typo cannot silently pass null down the chain.
  clearAfter: <step> removes the value once that step completes; a conforming generator MUST
  NOT leave a cleared var readable in the instance's durable data or history, which is the whole
  point for a generated credential.
• None of this changes what a delegate does. It declares only what surrounds it: how often it is
  tried, where a failure goes, and which values cross between steps.

Prior art / workarounds

Two hand-written process definitions, each with a per-service-task retry cycle (R3/PT30S) and an
error subprocess that writes the failed status and the message, plus a final task that clears the
generated password from the instance variables. The task bodies are genuinely custom; the
retry/error/variable scaffolding around them is mechanical and identical in every
provisioning-shaped flow we have written. Reported from a real service whose entire domain layer is
intent-first — these two files are the largest remaining block of hand-written model.

[delchev]

Implementation tracked in the Dirigible reference implementation as eclipse-dirigible/dirigible#6762 — it follows this proposal rather than leading it.

[iliyan-velichkov]

This proposal is now implemented in the reference implementation: eclipse-dirigible/dirigible#6783 (issue eclipse-dirigible/dirigible#6762), with the platform docs in dirigible-io/dirigible-io.github.io#193. The shape landed as proposed; implementation pinned a few semantics the spec text should capture when this converges:

• retry: { count, every } — count is the number of further attempts after the first (total attempts = count + 1), an integer >= 1; every is an ISO-8601 duration, the same vocabulary as timeout.after. Intermediate failed attempts roll back cleanly (engine-level job retry), so a partially-written attempt never leaks.
• onError: <step | end> — validated like next/then. The exhausted (or, with no retry, the first) failure is converted into a caught error routed through a boundary on the task; the final attempt's writes commit with the error route (standard error-boundary semantics — the conversion is what makes {error} capturable at all, since a plain failure rolls back). v1 restricts both keywords to delegate: service tasks — that is where the runtime conversion hook lives; widening is non-breaking.
• {error} — the FINAL attempt's failure message. Implemented as a whole-value token on a setField value; a partial-value use and any use on a step not reachable from an onError route are parse errors (nothing else ever populates it — the alternative is the silent empty-string fallback the proposal warns about).
• vars: + produces:/uses: — declaration-level contracts: an undeclared name in either list is a parse error; var names must be plain identifiers. clearAfter must name a task step (a gateway/end has no completion to attach to) and fires on the step's normal completion only — an error route out of the producing step leaves the value until the instance ends. That last edge is a deliberate v1 boundary the spec may want to state (or extend with an instance-end sweep).
• A file using none of the keywords generates byte-identically (test-pinned).

Coverage in the reference implementation spans parse validation, emission, a live-engine conversion test (which is what empirically pinned the count-vs-total-attempts arithmetic), and an end-to-end scenario where a flaky call recovers on exactly its last declared attempt and an exhausted one records the final attempt's message.