ci: move system tests from Github Actions to Testing Farm by pbrezina · Pull Request #9304 · SSSD/sssd

pbrezina · 2026-09-21T11:16:36Z

The system tests can quickly deplete available Github runners (20)
and we need them for process purposes (Accepted label). So we keep
only quick checks in GHA now.

Testing Farm is supported infrastructure for Fedora Project, we should
be able to see infra issues resolved more quickly as we can actually
ping people when needed.

pbrezina · 2026-09-22T07:52:49Z

This is waiting for new feature in packit:

And SSSD/sssd-ci-containers#219

pbrezina · 2026-09-22T11:05:56Z

Note: Do not forget to switch sssd-ci-containers repo in the test plan after SSSD/sssd-ci-containers#219 is merged.

pbrezina · 2026-09-22T12:10:41Z

/packit build
/packit check

pbrezina · 2026-09-24T07:25:18Z

/packit build
/packit check

pbrezina · 2026-09-29T17:36:22Z

/packit check

packit-as-a-service · 2026-09-29T17:36:26Z

There appears to be a syntax error in the command provided. Please refer to the Packit documentation or use the /packit help command.

pbrezina · 2026-09-29T19:03:26Z

/packit test

pbrezina · 2026-09-30T07:01:44Z

/packit test

pbrezina · 2026-09-30T10:11:26Z

I get this failure on centos-10... it is a segfault in autofs, I can't reproduce this locally so I'm trying to catch it in reserved TF run.

FAILED tests/test_autofs.py::test_autofs__new_map_entries_added_to_provider_are_visible_after_reload (ldap) - pytest_mh.conn.ssh.SSHProcessTimeoutError: 
Command #19113 timed out after 300 seconds:
  Command:
    set -ex
    pushd "/var/export/export2"
    mount | grep "nfs.test:/dev/shm/exports/export2 on /var/export/export2"
    popd
    umount "/var/export/export2"
  CWD:
  Env:
  Output:
  Error output:
    + pushd /var/export/export2

Other two errors are in provisioning phase. Both looks like a network error.

On TF side, reported, they will implement a retry

[2026-09-30 07:04:47] [+] [worker_0] [guest-setup-order] Running guest setup for fedora-copr-build artifact
[2026-09-30 07:04:47] [+] [worker_0] [fedora-copr] Initialized with 11049366:fedora-rawhide-x86_64: package 'sssd' build '11049366' target 'fedora-rawhide-x86_64' (https://copr.fedorainfracloud.org/coprs/packit/SSSD-sssd-9304/build/11049366/)
[2026-09-30 07:04:47] [E] [Fedora-latest:x86_64:/tmt/plans/system-tests] guest setup failed: 'utf-8' codec can't decode byte 0xe2 in position 4642988: invalid continuation byte

Other one gets further but fails when downloading containers. I implemented a retry on our side.

unable to copy from source docker://quay.io/sssd/ci-keycloak:fedora-45: writing blob: storing blob to file "/var/tmp/container_images_storage484318116/2": happened during read: unexpected EOF (while reconnecting: Get "https://cdn01.quay.io/quayio-production-s3/sha256/5e/5e45f5cd39f75b56f8265cfad98cc04c5ca0c13ec20bc28b6b87dc757e650c0d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260930%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260930T070847Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=f65e83168abafa3d93496c56b822de2a33b720be40388ce4189d8f19cb740a45&region=us-east-1&namespace=sssd&repo_name=ci-keycloak&akamai_signature=exp=1790753027~hmac=acd19ced820e2b607fc7b6017192e006e942755280af2463b297ac9bda1be5b3": EOF)
        Error: executing /usr/libexec/docker/cli-plugins/docker-compose up --no-recreate --detach: exit status 1

pbrezina · 2026-09-30T11:06:11Z

/packit retest-failed

pbrezina · 2026-09-30T14:51:24Z

/packit retest-failed

pbrezina · 2026-10-13T12:05:21Z

/packit test

pbrezina · 2026-10-13T17:20:57Z

/packit test

pbrezina · 2026-10-14T08:41:08Z

/packit test

pbrezina · 2026-10-14T19:47:36Z

/packit test -e SSSD_TEST_FRAMEWORK=pbrezina/sssd-test-framework@ipa-copy-backup

packit-as-a-service · 2026-10-14T19:47:41Z

There appears to be a syntax error in the command provided. Please refer to the Packit documentation or use the /packit help command.

pbrezina · 2026-10-14T19:48:06Z

/packit test --env SSSD_TEST_FRAMEWORK=pbrezina/sssd-test-framework@ipa-copy-backup

This enables automatic collection of coredumps, failing test if the coredumps occurs in any SSSD binary. Other coredumps will be ignored if test is passed. All coredumps are collected if test fails.

pbrezina · 2026-10-15T10:34:03Z

/packit test --env SELECT_TESTS=test_sssctl__invalid_option_in_domain

pbrezina · 2026-10-15T10:58:04Z

/packit test --env SELECT_TESTS=test_sssctl__invalid_option_in_domain

pbrezina · 2026-10-15T20:11:54Z

/packit test

The system tests can quickly deplete available Github runners (20) and we need them for process purposes (Accepted label). So we keep only quick checks in GHA now. Testing Farm is supported infrastructure for Fedora Project, we should be able to see infra issues resolved more quickly as we can actually ping people when needed.

pbrezina · 2026-10-16T16:01:26Z

Well, I don't want to jinx it but hopefully our PR CI will be now in better shape.

This patch is finalized and all dependencies and test fixes are merged. This is ready for review.

I still saw failure in dyndns tests and one passkey test. Dan is looking into it, but I will add it to the known issues.

justin-stephenson

Thank you for your effort working on this.

alexey-tikhonov · 2026-10-19T17:37:03Z

+* `plans/` -- FMF plans, discovered and run by tmt.
+
+The plans' `prepare` step fetches
+[sssd-ci-containers](https://github.com/pbrezina/sssd-ci-containers)`/tmt/setup.sh`,


Link is to a fork - pbrezina.

alexey-tikhonov · 2026-10-19T17:39:29Z

Looks like Check polarion metadata step is dropped -- is this intentional?

pbrezina force-pushed the tft branch 3 times, most recently from ef7f7e8 to 400131b Compare September 21, 2026 12:08

This was referenced Sep 21, 2026

ci: add tmt scripts that can be reused in tmt plans SSSD/sssd-ci-containers#219

Merged

Add skip_install option to tests job packit/packit#2775

Closed

pbrezina force-pushed the tft branch from 400131b to 38c1ebb Compare September 21, 2026 14:17

pbrezina force-pushed the tft branch from 38c1ebb to 8c7620a Compare September 22, 2026 11:04

pbrezina force-pushed the tft branch 2 times, most recently from afee380 to 482ba1f Compare September 22, 2026 11:25

alexey-tikhonov requested a review from justin-stephenson September 24, 2026 12:56

alexey-tikhonov assigned justin-stephenson Sep 24, 2026

pbrezina force-pushed the tft branch 2 times, most recently from b8273c4 to fa6197b Compare September 25, 2026 09:59

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

pbrezina force-pushed the tft branch 2 times, most recently from 66e45be to 752e0b5 Compare September 29, 2026 09:27

pbrezina force-pushed the tft branch 4 times, most recently from 5e6741c to b223323 Compare October 13, 2026 11:34

pbrezina force-pushed the tft branch from b223323 to fa24c18 Compare October 13, 2026 12:52

tests: automatically collect coredumps

3e25b78

This enables automatic collection of coredumps, failing test if the coredumps occurs in any SSSD binary. Other coredumps will be ignored if test is passed. All coredumps are collected if test fails.

pbrezina force-pushed the tft branch from f317e12 to b9bf3fe Compare October 15, 2026 10:32

pbrezina force-pushed the tft branch 2 times, most recently from 0ffe7e5 to eb0fe01 Compare October 15, 2026 17:16

pbrezina marked this pull request as ready for review October 15, 2026 18:20

pbrezina force-pushed the tft branch from eb0fe01 to 513a419 Compare October 16, 2026 15:59

justin-stephenson approved these changes Oct 16, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

Comment thread tmt/readme.md

pbrezina added backport-to-sssd-2-9 backport-to-sssd-2-10 backport-to-sssd-2-9-4 Corresponds to C8S backport-to-sssd-2-12 backport-to-sssd-2-13 labels Oct 19, 2026

alexey-tikhonov reviewed Oct 19, 2026

View reviewed changes

Conversation

pbrezina commented Sep 21, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pbrezina commented Sep 29, 2026

Uh oh!

packit-as-a-service Bot commented Sep 29, 2026

Uh oh!

pbrezina commented Sep 29, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Oct 13, 2026

Uh oh!

pbrezina commented Oct 13, 2026

Uh oh!

pbrezina commented Oct 14, 2026

Uh oh!

pbrezina commented Oct 14, 2026

Uh oh!

packit-as-a-service Bot commented Oct 14, 2026

Uh oh!

pbrezina commented Oct 14, 2026

Uh oh!

pbrezina commented Oct 15, 2026

Uh oh!

pbrezina commented Oct 15, 2026

Uh oh!

pbrezina commented Oct 15, 2026

Uh oh!

pbrezina commented Oct 16, 2026

Uh oh!

justin-stephenson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexey-tikhonov Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

alexey-tikhonov commented Oct 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants