Skip to content

ci: establish a hermetic unit-test suite - #848

Open
linhongyu510 wants to merge 1 commit into
zilliztech:mainfrom
linhongyu510:ci/hermetic-unit-tests
Open

ci: establish a hermetic unit-test suite#848
linhongyu510 wants to merge 1 commit into
zilliztech:mainfrom
linhongyu510:ci/hermetic-unit-tests

Conversation

@linhongyu510

@linhongyu510 linhongyu510 commented Aug 13, 2026

Copy link
Copy Markdown

Summary

This is a focused first phase of #819:

  • establish an allowlisted tests/unit suite that does not require provider SDK extras, credentials, databases, downloads, or network sockets
  • split the dataset tests so download and large-dataset iteration cases live under tests/e2e
  • add unit-test and e2e-test Make targets while keeping unittest as a compatibility alias
  • run the hermetic unit suite in pull-request CI
  • document the temporary boundary between classified unit/E2E tests and remaining legacy provider tests

Scope

This PR intentionally does not claim to complete the provider-by-provider reorganization proposed in #819. Tests still directly under tests/ remain legacy and unclassified; they can be split in follow-up changes.

Validation

Validated in fresh isolated environments with only the project and .[test] installed:

  • Python 3.11: 255 passed with sockets disabled
  • Python 3.12: 255 passed with sockets disabled
  • Black: 225 source files unchanged
  • Ruff: all checks passed
  • git diff --check: passed

Progresses #819.

@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: linhongyu510
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@linhongyu510

Copy link
Copy Markdown
Author

/assign @XuanYang-cn

@linhongyu510
linhongyu510 force-pushed the ci/hermetic-unit-tests branch from a54b4b2 to 993ed4d Compare August 28, 2026 13:11
@linhongyu510

Copy link
Copy Markdown
Author

Rebased this PR onto the latest main and resolved the dataset-test split conflict. The branch is now conflict-free (mergeable).

The conflict came from new dataset tests added upstream after this PR was opened. I preserved all 15 upstream dataset tests and kept the intended hermetic boundary:

  • 12 local-only tests now live in tests/unit/test_dataset.py
  • 3 download / large-dataset iteration tests remain in tests/e2e/test_dataset.py

I also updated two existing FTS unit-test stubs with the newly required config.case_config.k field introduced upstream. No production logic was changed.

Local verification on Python 3.11:

255 passed in 3.08s

Black and import-order checks also pass for the touched test files. Could a maintainer take another look when convenient? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants