direct: report resources as they are deployed - #6361
Draft
denik wants to merge 4 commits into
Draft
Conversation
The direct engine printed all per-resource lines after the whole deployment finished, so a deploy that failed part way through reported nothing about the resources it had already applied. Resources are applied in parallel, so the lines are no longer in a deterministic order. Add a SortLines/SortLinesOn acceptance config option that sorts runs of matching output lines before comparison, and enable it for these lines. Co-authored-by: Isaac
The blocks in the selftest sorted the same way whether or not the boundary was respected. Pick values so a merged sort would move lines across it, and add the matching unit test case. Co-authored-by: Isaac
Collaborator
Integration test reportCommit: e7f63e5
8 interesting tests: 4 SKIP, 3 flaky, 1 RECOVERED
Top 17 slowest tests (at least 2 minutes):
|
Co-authored-by: Isaac
The migration ran inside deployCore, before the postdeploy script. Order is now deploy, then the postdeploy script, then the migration. The migration is still gated on the deploy succeeding and still not on the script: the resources are deployed either way, so the state describes the same deployment. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The per-resource output added in #5720 (CLI v1.13.0)
is printed after the whole deployment has finished, so a deploy that failed part way
through reported nothing about the resources it had already applied. On the direct engine
each resource is now reported as soon as it is applied.
Changes
Resources are applied in parallel, so these lines are no longer in a deterministic order.
Added
SortLines(name -> regexp) andSortLinesOn(name -> bool) to the acceptanceconfig: each run of consecutive output lines matching a pattern is sorted before
comparison, so a test pins which resources were applied rather than the order. Enabled
once in
acceptance/test.toml; an innertest.tomlcan replace a pattern by reusing itsname or switch it off.
resource_deps/create_errordeploys with-qnow: itsjobs.independenthas nodependency on the failing job, so where its line lands relative to the errors is not
stable, and that test is about which errors a failing deploy reports.
Tests
New
selftest/sortlinescovers sorting, per-name override and disabling. 18 goldensupdated: 12 failure and kill paths now report what succeeded, 6 are lines moving to their
true chronological position.