fix: preserve CAR recovery controls and rotate all DB instances - #494
Merged
Merged
Conversation
What changed: - Reuse the ordered data-volume, control-volume, recovery, and environment setup after initial image availability succeeds. - Cover cached startup, delayed retries, paused and semaphore states, control request recovery, and unavailable system volumes. Why: - Fixed-volume stale-mount cleanup must run before provisioning /r1en_system, including after a recovered image pull, so the container does not lose its control volume.
What changed: - Decide whether a certificate regeneration operation is pending before processing plugin instances that share allocation metadata. - Verify all node and root-client bundles rotate for two plugins, repeated operation IDs preserve inputs, and new operation IDs rotate both again. Why: - The first plugin marked the shared operation complete and caused subsequent accepted plugins to keep their previous certificates.
toderian
approved these changes
Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up fixes for the two findings in the review of #492. This is a new branch
from current
develop, which already contains #490 and #488.request recovery -> environment setup after initial image-pull recovery. Reuse
the same initialization block as cached-image startup; do not provision the
control volume while waiting for the image.
any plugin updates shared allocation metadata, so every accepted database
plugin rotates and replayed operation IDs remain no-ops.
Production scope is two files, seven net added lines. No API/schema, image,
workflow, dapp, volume format, or loop-device allocation changes.
Verification
fixed volumes, cached startup, retry ordering, pause/restart deferral, semaphore
readiness, restored control request files, and unavailable system volumes.
all three nodes, root-client rotation, replay/new-operation behavior, preserved
node environment, admission and update lifecycle tests.
git diff --checkpasses. Final scoped criticreview found no remaining blockers.
Commands:
python -m unittest discover -s extensions/business/container_apps/tests -p 'test_*.py'and the equivalent underextensions/business/deeploy/tests.Environment: existing Python 3.13.13 venv, cryptography 42.0.7; CAR diskapi tests
used the expected nested core path linked to the local core checkout at
ded22ead. No dependencies or test fixtures changed to hide failures.Limits and Rollout
These are local regression/integration tests: real temporary control files and
generated certificates, mocked Docker/mount and external service boundaries.
No live local/hybrid cluster deployment or privileged mount test was performed.
The patched edge-node runtime must be deployed for the fixes to take effect.
This does not repair an already-running container's broken mount automatically.
No data migration is required. Reverting these commits restores the previous
behavior and its two defects.