Skip to content

ci: failed E2E jobs keep the k3s log and node description - #400

Merged
HardMax71 merged 1 commit into
mainfrom
ci/save-k3s-logs-on-failure
Oct 18, 2026
Merged

HardMax71 merged 1 commit into
mainfrom
ci/save-k3s-logs-on-failure

Conversation

@HardMax71

@HardMax71 HardMax71 commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Summary

Since #390 moved k3s into a container, 3 of 49 E2E jobs timed out in "Finalize k3s" because the node never reported Ready within 120 s. The latest were two jobs in one main run, 39175060996. In the other jobs the step takes 0.7 to 11 s. None of the failed jobs kept anything from k3s, so there is no way to tell why the node stayed NotReady, and main goes red until someone reruns it.

Changes

Both E2E jobs' failure steps now also save docker logs k3s as k3s.log and kubectl describe nodes as k8s-nodes.log in the uploaded logs. kubectl reads the kubeconfig that "Finalize k3s" copies before its wait, so the node description works even when that wait is what failed. cicd.md lists the new files.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or tooling changes

Test Plan

  • Backend unit tests pass (cd backend && uv run pytest tests/unit)
  • Frontend unit tests pass (cd frontend && npm run test)
  • Backend E2E tests pass (docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')
  • Frontend E2E tests pass (cd frontend && npm run test:e2e)
  • API spec and client regenerated if routes or schemas changed (./deploy.sh openapi, then ./deploy.sh types), never edited by hand
  • Manual testing performed

Validation: actionlint passes, and both commands ran against a local k3s container (the node's conditions and 10720 log lines). The steps only run on failure, so this PR's green run won't exercise them.

Checklist

  • My code follows the project's style guidelines
  • I have run pre-commit run --all-files and all checks pass
  • I have added tests that prove my fix/feature works
  • I have updated documentation if needed
  • My changes don't introduce new warnings

Summary by cubic

Failed E2E jobs now keep the k3s container log and node description, so timeouts in "Finalize k3s" can be diagnosed instead of rerunning blind.

Both E2E jobs' failure steps now also run docker logs k3s and kubectl describe nodes, uploading them as k3s.log and k8s-nodes.log with the existing logs. kubectl reads the kubeconfig that "Finalize k3s" copies before its wait, so the node description works even when that wait is what failed. docs/operations/cicd.md lists the new files.

Written for commit bd12121. Summary will update on new commits.

View guided diff Turn on auto-fix

Three of 49 E2E jobs since k3s moved into a container timed out in Finalize k3s with the node never Ready, and none of them kept anything from k3s. Both E2E jobs now save docker logs k3s and kubectl describe nodes with their other failure logs.
@HardMax71
HardMax71 enabled auto-merge (squash) October 18, 2026 23:24
@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a0be8a95-a453-4a9e-8cf9-eb8ae13d0045
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added documentation Improvements or additions to documentation area: ci GitHub workflows, test infrastructure, repo tooling labels Oct 18, 2026
@sonarqubecloud

Copy link
Copy Markdown

@codecov-commenter

codecov-commenter commented Oct 18, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Flag Coverage Δ
backend-e2e 82.81% <ø> (-0.10%) ⬇️
backend-unit 73.22% <ø> (ø)
frontend-unit 92.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

Components Coverage Δ
Backend 92.02% <ø> (-0.10%) ⬇️
Frontend 92.60% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HardMax71
HardMax71 merged commit 1e4abdf into main Oct 18, 2026
21 checks passed
@HardMax71
HardMax71 deleted the ci/save-k3s-logs-on-failure branch October 18, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci GitHub workflows, test infrastructure, repo tooling documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants