Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 86 additions & 3 deletions sdx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ A Kubernetes deployment for running SDX (Secure Data Exchange) Edge Servers as h

### Helm Chart (`chart/sdx-edge`)

**Chart Version:** 0.3.4
**Chart Version:** 0.3.6
**App Version:** 3.9.1

Deploys a Kong Gateway data plane node configured for secure data exchange operations. The chart includes:
Expand Down Expand Up @@ -54,7 +54,7 @@ helm upgrade --install ${EDGE_ID} \

```sh
helm package sdx-edge
helm push sdx-edge-0.3.5.tgz oci://ghcr.io/bcgov/aps-devops
helm push sdx-edge-0.3.6.tgz oci://ghcr.io/bcgov/aps-devops
```

#### Configuration
Expand All @@ -73,7 +73,12 @@ The following table lists the configurable parameters in `values.yaml`:
| `sdx_aggregator_url` | SDX aggregator service endpoint | `gwaggregator-api-gov-bc-ca-lab.dev.api.gov.bc.ca` |
| `mtls_required` | Enable/disable mutual TLS requirement | `true` |
| `https_proxy` | HTTP proxy URL for restricted network environments | `""` (empty, disabled) |
| `bootstrap.tls.token` | Bootstrap token for initial certificate request | `""` (must be provided) |
| `bootstrap.tls.token` | One-time CA token for certificate bootstrap. Clear on promote with `--set-string bootstrap.tls.token=""` so the bootstrap Job is dropped, not mutated. | `""` (must be provided) |
| `bootstrap.stageSecret` | Write `{release}-client-next` and skip Kong restart | `false` |
| `bootstrap.deferRestart` | Skip Kong restart after writing live bootstrap secrets | `false` |
| `renewal.deferRestart` | Skip Kong restart after certificate renewal | `false` |
| `rotation.promote` | One-shot: copy `{release}-client-next` to live secrets and restart Kong. Reset to `false` after the Job succeeds. | `false` |
| `rotation.nonce` | Suffix for the promote Job name (bump on each promote) | `"1"` |
| `tls.client.cn` | Common Name for client certificate | `example.com` |
| `tls.server.ip` | IP address to add as SAN to edge server certificate | `""` (optional) |
| `tls.public_ca` | PEM-encoded public CA certificates for trust chain | (includes Sectigo, USERTrust, SDX, APS, Amazon, Let's Encrypt root CAs) |
Expand All @@ -97,6 +102,84 @@ The following values must be set during installation:
- `bootstrap.tls.token` - Required for certificate bootstrapping
- `route.host` - Required for proper external routing

#### Runtime-group key rotation

Private-key generation and Kong restart are separate Helm steps so the new
public key can be published before the data plane starts signing with it.

`--reuse-values` persists flags in the release, so stage and promote must
clear one-shot values instead of only toggling `stageSecret`.

1. Create a one-time CA token, then bootstrap with staging. Use
`--wait --wait-for-jobs` so the bootstrap Job finishes before the next
step (it is not a Helm hook):

```sh
helm upgrade ${EDGE_ID} oci://ghcr.io/bcgov/aps-devops/sdx-edge \
--reuse-values \
--wait --wait-for-jobs \
--set bootstrap.tls.token=${TOKEN} \
--set bootstrap.stageSecret=true
```

This writes `sdx-edge-${EDGE_ID}-client-next` and does **not** restart Kong.
The Job signs the CSR itself (`step ca sign`); only `tls.crt` and `tls.key`
are stored.

2. Extract the staged certificate and publish it with `sdx-keys.r1`
`operation=rotate` (keeps the old key for overlap). Confirm JWKS lists both
kids.

```sh
kubectl get secret sdx-edge-${EDGE_ID}-client-next \
-o jsonpath='{.data.tls\.crt}' | base64 -d
```

Pass that PEM as `certificatePem` on `sdx-keys.r1` (see
`docs/sdx-keys-rotation.md` in api-services-portal).

3. Promote the staged secret and rolling-restart Kong. Clear the consumed
bootstrap token with an **empty string** so Helm **drops** the
`*-boot-<token hash>` Job and its Secret. Do not use `--set …=null`: with
`--reuse-values`, Helm 3 coalescing strips the null and the previous token
is rendered again. Leaving the token set keeps that Job in the release
while `stageSecret` flips the pod template from staging writes to live
writes. Kubernetes Job specs are immutable, so the upgrade is rejected
while the completed Job still exists; if TTL already removed it, Helm
recreates the Job with the spent one-time token.

```sh
helm upgrade ${EDGE_ID} oci://ghcr.io/bcgov/aps-devops/sdx-edge \
--reuse-values \
--wait \
--set-string bootstrap.tls.token="" \
--set bootstrap.stageSecret=false \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we clear bootstrap.tls.token as part of this promotion upgrade, or otherwise remove and recreate the bootstrap Job? --reuse-values keeps the staged token, so the same *-boot-<token hash> Job remains rendered while stageSecret changes its pod template from staging to live-secret writes. Kubernetes Job pod templates are immutable, so a stage-to-promote upgrade will be rejected while that completed Job still exists. If TTL has already removed it, Helm can instead recreate the bootstrap Job with the consumed one-time token. A consecutive stage/promote test would help pin this down.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this. I traced the --reuse-values merge in Helm 3.20, and null unfortunately does not clear a value stored by the previous release: the null entry is removed during coalescing, then the old token is supplied again when Helm renders. That means this command still renders the same bootstrap Job with stageSecret=false and runs into the immutable pod template. Could we use an explicit empty string instead (--set-string bootstrap.tls.token="") and cover a real stage-to-promote reuse-values sequence? An empty string remains an explicit override and rendered correctly in my check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phowells Thanks — promote now uses --set-string bootstrap.tls.token="" so --reuse-values drops the bootstrap Job instead of mutating it (null does not clear on Helm 3).

Ran consecutive stage → promote on Gold dev (pzgw / b8840c-dev, local chart). Stage wrote sdx-edge-pzgw-client-next with Kong still at gen 100. Promote cleared the token with no immutable-spec error, copied next → live, and rolled Kong to gen 101. Then --set rotation.promote=false.

Live provisioner has no operation=rotate, so JWKS stayed a single :0 kid. Helm path is what this PR covers.

--set rotation.promote=true \
--set rotation.nonce=$(date +%s)
```

Helm waits for the post-upgrade promote Job before this command returns.

4. Reset `rotation.promote` so later `--reuse-values` upgrades do not render
the hook again (that would copy `{release}-client-next` over the live
secrets and restart Kong):

```sh
helm upgrade ${EDGE_ID} oci://ghcr.io/bcgov/aps-devops/sdx-edge \
--reuse-values \
--set rotation.promote=false
```

5. Confirm `trust-sign` emits the kid that matches the mounted private key,
wait through the verifier grace period, then delete the old kid with
`operation=delete`.

Certificate **renewal** (`step ca renew`) keeps the same private key, so the
cron job still restarts Kong by default. Set `renewal.deferRestart=true` only
when restart is handled separately.

See `docs/sdx-keys-rotation.md` in api-services-portal for the full contract.

**Example Override:**

```yaml
Expand Down
2 changes: 1 addition & 1 deletion sdx/chart/sdx-edge/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ apiVersion: v2
name: sdx-edge
description: A Helm chart for deploying an SDX Edge Server to a Kubernetes cluster
type: application
version: 0.3.5
version: 0.3.6
appVersion: "3.9-21984e88"
9 changes: 9 additions & 0 deletions sdx/chart/sdx-edge/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{{- if and .Values.rotation .Values.rotation.promote }}
rotation.promote is true. After the promote Job succeeds, reset it so later
--reuse-values upgrades do not copy {{ include "sdx-edge.fullname" . }}-client-next
over the live TLS secrets and restart Kong:

helm upgrade {{ .Release.Name }} oci://ghcr.io/bcgov/aps-devops/sdx-edge \
--reuse-values \
--set rotation.promote=false
{{- end }}
15 changes: 14 additions & 1 deletion sdx/chart/sdx-edge/templates/job-cert-bootstrap.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# Cert Bootstrap Job
# - This job is used to register a new TLS certificate for Kong
# - It runs a container that executes the certificate sign script
# - Rendered only while bootstrap.tls.token is set. Clear the token on promote
# with --set-string bootstrap.tls.token="" (not null; see README) so Helm
# deletes this Job instead of changing its pod template (stageSecret). Job
# specs are immutable; a consumed one-time token must not recreate the Job
# after TTL either.
{{ if .Values.bootstrap.tls.token }}
apiVersion: batch/v1
kind: Job
Expand Down Expand Up @@ -63,6 +68,13 @@ spec:
curl -sLO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl

{{- if .Values.bootstrap.stageSecret | default false }}
kubectl create secret \
--save-config --dry-run=client -o yaml \
tls ${EDGE_NAME}-client-next \
--cert=./tls.crt \
--key=./tls.key | kubectl apply -f -
{{- else }}
kubectl delete secret ${EDGE_NAME}-client || true
kubectl delete secret ${EDGE_NAME}-server || true

Expand All @@ -77,6 +89,7 @@ spec:
tls ${EDGE_NAME}-server \
--cert=./tls.crt \
--key=./tls.key | kubectl apply -f -
{{- end }}

{{- if .Values.shared.ca_secret }}

Expand All @@ -85,7 +98,7 @@ spec:
generic sdx-edge-ca \
--from-file=ca.crt=./roots.pem | kubectl apply -f -
{{- end }}
{{- if .Values.kong.enabled }}
{{- if and .Values.kong.enabled (not (.Values.bootstrap.stageSecret | default false)) (not (.Values.bootstrap.deferRestart | default false)) }}
kubectl rollout restart deployment $EDGE_NAME
{{- end }}

Expand Down
96 changes: 96 additions & 0 deletions sdx/chart/sdx-edge/templates/job-cert-promote.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
{{ if and .Values.rotation .Values.rotation.promote }}
# Promote a staged runtime-group TLS secret ({release}-client-next) to the live
# client and server secrets, then rolling-restart Kong so trust-sign mounts the
# new private key. Run only after sdx-keys.r1 operation=rotate has published
# the matching public key.
#
# One-shot: Helm stores rotation.promote in release values. Reset it to false
# after this hook succeeds; otherwise the next --reuse-values upgrade renders
# the hook again, overwrites live secrets from client-next, and restarts Kong.
apiVersion: batch/v1
kind: Job
metadata:
name: "{{ include "sdx-edge.fullname" . }}-promote-{{ .Values.rotation.nonce | default "1" }}"
labels:
app.kubernetes.io/name: sdx-edge
app.kubernetes.io/component: job-cert-promote
app.kubernetes.io/instance: "{{ include "sdx-edge.fullname" . }}"
data-plane: "{{ include "sdx-edge.fullname" . }}"
annotations:
helm.sh/hook: post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
parallelism: 1
completions: 1
backoffLimit: 0
ttlSecondsAfterFinished: 604800
template:
metadata:
labels:
DataClass: Low
app.kubernetes.io/name: sdx-edge
app.kubernetes.io/component: job-cert-promote
app.kubernetes.io/instance: "{{ include "sdx-edge.fullname" . }}"
data-plane: "{{ include "sdx-edge.fullname" . }}"
spec:
restartPolicy: Never
serviceAccountName: sdx-job-cert-bootstrap
schedulerName: default-scheduler
enableServiceLinks: true
terminationGracePeriodSeconds: 30
containers:
- name: cert-promote
image: 'docker.io/smallstep/step-cli:0.23.0'
imagePullPolicy: IfNotPresent
command:
- /bin/bash
- +x
- -e
- -c
- |-
cd /tmp
export PATH=$PATH:/tmp

curl -sLO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl

kubectl get secret ${EDGE_NAME}-client-next -o jsonpath='{.data.tls\.crt}' | base64 -d > tls.crt
kubectl get secret ${EDGE_NAME}-client-next -o jsonpath='{.data.tls\.key}' | base64 -d > tls.key

kubectl create secret \
--save-config --dry-run=client -o yaml \
tls ${EDGE_NAME}-client \
--cert=./tls.crt \
--key=./tls.key | kubectl apply -f -

kubectl create secret \
--save-config --dry-run=client -o yaml \
tls ${EDGE_NAME}-server \
--cert=./tls.crt \
--key=./tls.key | kubectl apply -f -

{{- if .Values.kong.enabled }}
kubectl rollout restart deployment $EDGE_NAME
{{- end }}
env:
- name: EDGE_NAME
value: "{{ include "sdx-edge.fullname" . }}"
{{- if .Values.https_proxy }}
- name: HTTPS_PROXY
value: "{{ .Values.https_proxy }}"
- name: NO_PROXY
value: ".cluster.local,.svc,10.91.0.0/16,10.93.0.0/16,172.30.0.0/16,127.0.0.1,localhost,.gov.bc.ca"
{{- end }}
volumeMounts:
- name: working-dir
mountPath: /tmp
- name: kube-cache
mountPath: .kube
automountServiceAccountToken: true
volumes:
- name: working-dir
emptyDir: {}
- name: kube-cache
emptyDir: {}
dnsPolicy: ClusterFirst
{{ end }}
2 changes: 1 addition & 1 deletion sdx/chart/sdx-edge/templates/job-cert-renewal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ spec:
--cert=./tls.crt \
--key=./tls.key | kubectl apply -f -

{{- if .Values.kong.enabled }}
{{- if and .Values.kong.enabled (not (.Values.renewal.deferRestart | default false)) }}
kubectl rollout restart deployment $EDGE_NAME
{{- end }}
{{- if .Values.shared.fluentbit.enabled }}
Expand Down
23 changes: 23 additions & 0 deletions sdx/chart/sdx-edge/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,35 @@ kong:

bootstrap:
tls:
# One-time CA token. Clear on promote with an empty string
# (`--set-string bootstrap.tls.token=""`), not null: Helm 3 --reuse-values
# coalescing drops null and restores the previous token (see README).
token: ""
cn: "example.com"

# Added as a SAN to the edge certificate
ip: ""

# Write {release}-client-next instead of live client/server secrets and skip
# Kong restart. Use after generating a new key/CSR so the public key can be
# published (operation=rotate) before the private key is mounted.
stageSecret: false
# Skip Kong restart after writing live bootstrap secrets.
deferRestart: false

renewal:
# Certificate renewal keeps the same private key. Restart afterwards so Kong
# picks up the new certificate; set true only when restart is handled separately.
deferRestart: false

rotation:
# One-shot: copy {release}-client-next onto live client and server TLS
# secrets, then rolling-restart Kong. Helm persists this across
# --reuse-values, so set false in a follow-up upgrade after the Job succeeds
# (see README). Bump nonce so each intentional promote gets a new Job name.
promote: false
nonce: "1"

environment: prod

sdx_control_url_by_env:
Expand Down