Repository navigation
Conversation
Rework the context the site-type packages copy: - delete only sites matching SITE_PREFIX after each feature (never every listed site), purge the prefix's remote backups for @Remote features, and undo per-scenario hosts entries, scratch dirs and backup leftovers - anchor the old regex steps: the unanchored `I run '(.*)'|"(.*)"` matched any step with a quoted string, and the `:index` placeholders inside regexes never matched - fix the inverted container label count check and look containers up by label instead of v1 compose names - add generic steps for exit codes, output, JSON, the sites table, files, hosts entries, docker objects, HTTP (resolved to 127.0.0.1, the public IP or verified DNS), TLS certificates, polling waits, config guards and rclone backups, with {NAME} variables in every argument
…vers Purge a site's remote backups only after listing the rclone path and finding that exact directory, and fail if backups remain. After each feature, once no suite site is left, remove suite-named proxy certs, ACME state, site dirs and volumes that known bugs leave behind. Add a guard step for host-wide LE commands and keep /etc/hosts lines intact when the file lacks a trailing newline.
- Hold the fake backup lock with flock(1) so the scenario also works once PR EasyEngine#480 switches to flock, and assert a failed restore through the next backup instead of the lock file. - Use a reserved .invalid name for the failing LE alias instead of a real rt.gw host outside the test subdomain. - Guard ssl-renew --all against other LE sites on the host. - Poll after reloading custom nginx config instead of one request. - Check SEC-1 leftovers by label and directory. - Add known-bug scenarios for update --php on html (SC-13) and update --ssl=custom (SC-15), and reference SC-* IDs and the open PRs that fix each known bug.
Correct the tag counts, document the remote listing, leftover cleanup and LE guards, map every known-bug scenario to its issues.md ID and the open PR that fixes it, and list the PRs that change what passing scenarios assert.
CI runs the @le scenarios against Pebble, whose intermediate CN differs per start. Defaults to Let's Encrypt.
Table output needs SHELL_PIPE=0 without a TTY, and JSON is pretty-printed. The clone error scenario needs a site on the host (SC-17). The self-signed ssl-info DNS scenario (SC-16) and the invalid --proxy-cache value (SC-18) expose product bugs, so they're tagged @known-bug; a no-SSL site now covers the non-DNS branch.
Backup ids have one-second resolution (BKP-5). On a fast local remote two backups landed in the same directory, so the retention scenario passed without testing anything and the by-id restore scenarios could compare a backup with itself.
On PHP 7.4 is_file() answers from the stat cache for the last path checked, so a file ee deleted in between still exists: "Deleting a self-signed site removes its certificate" and both redirect removal scenarios failed on the 7.4 runner, and the MIG-15 known-bug scenario passed vacuously there.
…gine#502) AUTH-1 is fixed on develop, so the underscore alias scenario is a regular regression check. "Delete an alias domain" now waits for the remaining alias to answer 200 before checking the deleted one: the recreated nginx container drops all of the site's domains from the proxy until docker-gen catches up, and on develop the command returns inside that gap.
…ct collision An html site whose name maps to the volumes and compose project of an existing site (bh-site-command-160x.test vs bh-site-command-160.x.test) is refused and leaves the existing site untouched. Needs the fix/site-name-collisions branch of site-command.
Regular now (fail on develop until fix/proxy-cache-rollback-and-le-alias merges): - SSL-2: a failing LE alias is rejected, the DB, certificate and renewals are unchanged. - SSL-5/SSL-9: deleting an LE site leaves no ACME state. - SC-3: `--ssl=off` works on a self-signed site; it also removes the certificate and the redirect's HTTPS block (WPT-12). New: deleting a site without SSL removes a leftover certificate, `--ssl=off` on an LE site removes the certificate and ACME state, deleting an alias removes its proxy cache location, and a failed proxy cache alias sync (PRX-6) leaves a config nginx can test and restart with.
…d restore scope Known-bug scenarios for BKP-7 (invalid no-of-backups), BKP-8 (hosts sharing rclone-path, via two backup-host-id config copies), BKP-9 (WordPress webroot files), BKP-10 (symlinked uploads), BKP-12 (restore without backups) and BKP-14 (dotfiles left by a restore). The remote-backup steps now also find backups under <rclone-path>/<site>/<host-id>/.
…ecks Adds @known-bug scenarios for CLONE-3, CLONE-7 and CLONE-8 (fixed on fix/clone-sync-safety): a loopback address as the same host, ssh options in a remote location, a remote `.`, a site name with shell syntax, the sync help, --ssl=custom without --ssl-crt, and admin passwords with shell syntax or characters the destination shell would expand.
update_ssl() saves false, which the DB stores as 0; assert it is no longer self, like the Disable SSL scenario.
… regular ones WP-1, SC-3, SC-15, CACHE-1, CLONE-3/7/8, SEC-1 and SEC-3 pass on develop. The wildcard-update refusal is checked with --ssl=le, since self-signed certificates are always wildcard (5c34be8).
…e certificate On PHP 7.4 the proxy still served its default certificate right after the HTTPS check returned 200 (PRX-7).
…line output compareContents() compared list elements by position, so [1,3,5] wasn't found in [1,2,3,4,5], and a missing key or property raised a notice. Both something-like steps only used the first line of the expected text.
Runs the package's features/ inside a fresh core develop checkout, with this checkout installed as the package through a Composer path repository, so the PR's own code is under test. @le is excluded (no public DNS on runners); @Remote backups use a runner-local rclone alias remote. @known-bug scenarios run in a separate non-gating job.
EE's hardcoded Let's Encrypt endpoint is redirected to a local Pebble on the runner (hosts entry, throwaway CA, resolved routing for the test domain). No product code changes.
Run PHP 7.4 and 8.5, strip control characters from the junit files before building failures.txt, and cap the Behat step at 75 minutes so a hung run still uploads its logs. The @le lanes against Pebble move into behat.yml (lanes named le*), replacing behat-le.yml. Diagnostics: the host addresses and routes, and the docker-compose version after the first run (MIG-20).
EE's first run downloads docker-compose 2.27.0 without --fail (MIG-10); on a GitHub error it moves the HTML error page over the binary, as on an admin-tools-command runner. Fetch 2.27.0 again with --fail and retries in that case.
Runs every lane on PHP 7.4, 8.0, 8.1, 8.2, 8.3, 8.4 and 8.5 instead of only the floor and ceiling; the jobs run in parallel.
…the behat workflow
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The TLS hostname assertion can accept certificates whose DNS subjectAltNames exclude the requested hostname.
1 open issue
What changed in this PR
Expands site-command’s Behat coverage and replaces the old CI workflow so tests exercise the pull request’s package inside EasyEngine core.
Changes:
- Adds command, lifecycle, SSL, backup, and clone/sync scenarios.
- Extends shared test helpers and documents coverage.
- Adds PHP matrix lanes using local Pebble and rclone services.
| File | Description |
|---|---|
| features/update-ssl-php.feature | Tests SSL updates and PHP validation. |
| features/update-alias.feature | Tests alias changes and validation. |
| features/ssl.feature | Covers certificate creation and inspection. |
| features/site.feature | Expands routing, help, and argument checks. |
| features/share.feature | Tests sharing guards. |
| features/restart-reload.feature | Tests service restarts and reloads. |
| features/redirect.feature | Expands www redirect coverage. |
| features/proxy-cache.feature | Tests cache configuration and cleanup. |
| features/list.feature | Covers listing formats and filters. |
| features/le.feature | Tests ACME issuance and certificate lifecycle. |
| features/labels.feature | Checks Docker resource labels. |
| features/info.feature | Tests site information and auto-detection. |
| features/enable-disable.feature | Covers site state transitions and refresh. |
| features/delete.feature | Tests site and resource removal. |
| features/create.feature | Expands HTML site creation coverage. |
| features/COVERAGE.md | Maps scenarios and coverage gaps. |
| features/clone-sync.feature | Tests clone/sync validation and WordPress transfers. |
| features/clean.feature | Tests cache cleaning. |
| features/bootstrap/support.php | Improves partial-content comparisons. |
| features/bootstrap/FeatureContext.php | Adds shared assertions, fixtures, and cleanup. |
| features/backup-restore.feature | Covers backup and restore operations. |
| .github/workflows/test_and_build.yml | Removes the previous workflow. |
| .github/workflows/behat.yml | Adds package-under-test matrix execution. |
| .github/behat/pebble-setup.sh | Configures local ACME infrastructure and isolation. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the three old feature files into a Behat suite for site-command (
features/: 211 scenarios, outline rows counted) and replacestest_and_build.ymlwith aBehatworkflow that runs it on every pull request, on pushes todevelopandmaster, and on demand.It covers create, delete, list and info, routing, enable and disable, restart and reload, clean and the proxy cache, SSL (self-signed, custom, inherited, Let's Encrypt issuance, renewal and aliases), alias, SSL and PHP updates, www redirects, share, backup and restore (local and remote), and clone and sync.
features/COVERAGE.mdmaps each command, flag and behaviour to its scenarios.developis cloned, everyeasyengine/*package is set todev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's. The old job tested the Packagistdev-developpackage instead.Behat Tests - PHP <version>, so the required checks keep matching. The main lane also builds the phar without dev autoloading, as the old job did.@destructivescenarios run there too, since the runner is thrown away.@leruns against a local Pebble ACME server (.github/behat/pebble-setup.sh). Every other lane points the Let's Encrypt host at a blackhole and checks that the real service can't be reached, and on CI the suite refuses to start unless one of the two is in place, so no run can reach the real Let's Encrypt.@remotebackup scenarios use a local rclone alias remote.Merge after EasyEngine/easyengine#1950, which stops core's test job from running this suite unfiltered.
To run it locally (as root, on a test machine, since the suite creates and deletes
*.testsites): copyfeatures/over a core checkout'sfeatures/and runsudo -E vendor/bin/behat --tags='~@le&&~@remote'from the core root; add~@destructiveto skip the scenarios that change global services or config.Green run on my fork (PHP 7.4 and 8.0 to 8.5, every lane): https://github.com/mrrobot47/site-command/actions/runs/39377584767