Repository navigation
Stop installing test-only packages with the executorch package - #24206
Open
pytorchbot wants to merge 1 commit into
Open
pytorchbot wants to merge 1 commit into
pytorchbot wants to merge 1 commit into
Conversation
### Summary `pip install executorch` installs four packages that are only there for ExecuTorch's own tests: `expecttest`, `hypothesis`, `kgb` and `parameterized`. `hypothesis` also brings `sortedcontainers`. So every user installs five packages they never need. This moves the four from the package's runtime dependencies (`_base_dependencies()` in `setup.py`) to `requirements-dev.txt`. The developer setup and every CI job that runs tests already install that file, so tests still get them. Nothing a user imports needs them. Three shipped files do, but only tests or a code generator use them: `backends/cadence/aot/typing_stubs.py`, `examples/models/llama/source_transformation/test_attention_sink.py`, and `exir/dialects/edge/spec/gen.py`. Effect, resolving the declared dependencies for Python 3.12 (without `torch`): 55 packages before, 50 after on macOS arm64 and Linux x86_64. The same five go away on Linux aarch64 and Windows. Kept on purpose: `coremltools` and `scikit-learn` (the Core ML backend needs both), and `hydra-core`, `omegaconf`, `pandas`, `tabulate` and `pytorch-tokenizers`, which shipped modules import. The first commit fixes the wheel's clean-install check. It hid undeclared packages so that `importlib.util.find_spec` raised instead of returning `None`. `torch._dynamo` probes for `hypothesis` with `find_spec` when it loads, so the check failed once `hypothesis` was no longer declared, although a real install works. ### Test plan On macOS arm64, in a fresh environment with only the trimmed dependencies, `torch` 2.14.0 and a nightly `executorch` installed with `--no-deps`: - Tried to import all ~3,000 modules in the package. Only the three files above failed for lack of a removed package. The rest imported, or stopped on unrelated optional packages such as vendor SDKs. - Exported a small model with XNNPACK and with Core ML, ran both, and matched eager PyTorch. - The clean-install check fails before the first commit and passes after it, and still fails when a checked module imports an undeclared package. Tests that use the moved packages pass with `requirements-dev.txt` installed. (cherry picked from commit ff09af0)
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24206
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New FailuresAs of commit f2bf53c with merge base 4db778e ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pip install executorchinstalls four packages that are only there for ExecuTorch's own tests:expecttest,hypothesis,kgbandparameterized.hypothesisalso bringssortedcontainers. So every user installs five packages they never need.This moves the four from the package's runtime dependencies (
_base_dependencies()insetup.py) torequirements-dev.txt. The developer setup and every CI job that runs tests already install that file, so tests still get them.Nothing a user imports needs them. Three shipped files do, but only tests or a code generator use them:
backends/cadence/aot/typing_stubs.py,examples/models/llama/source_transformation/test_attention_sink.py, andexir/dialects/edge/spec/gen.py.Effect, resolving the declared dependencies for Python 3.12 (without
torch): 55 packages before, 50 after on macOS arm64 and Linux x86_64. The same five go away on Linux aarch64 and Windows.Kept on purpose:
coremltoolsandscikit-learn(the Core ML backend needs both), andhydra-core,omegaconf,pandas,tabulateandpytorch-tokenizers, which shipped modules import.The first commit fixes the wheel's clean-install check. It hid undeclared packages so that
importlib.util.find_specraised instead of returningNone.torch._dynamoprobes forhypothesiswithfind_specwhen it loads, so the check failed oncehypothesiswas no longer declared, although a real install works.Test plan
On macOS arm64, in a fresh environment with only the trimmed dependencies,
torch2.14.0 and a nightlyexecutorchinstalled with--no-deps:Tests that use the moved packages pass with
requirements-dev.txtinstalled.