-
Notifications
You must be signed in to change notification settings - Fork 4
Prevent mixing cached and uploaded content in repositories #50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
aKlimau
wants to merge
1
commit into
pulp:main
Choose a base branch
from
aKlimau:mixed-content
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+176
−7
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| A repository can no longer be used for both pull-through caching and uploads at the same time. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| """Tests preventing a repository from mixing pull-through cached and uploaded content. | ||
|
|
||
| A repository must be used either for pull-through caching or for uploads, never both. | ||
| """ | ||
|
|
||
| import pytest | ||
|
|
||
| from pulpcore.client.pulp_rust.exceptions import ApiException | ||
|
|
||
| from pulp_rust.tests.functional.utils import CRATES_IO_URL | ||
|
|
||
|
|
||
| def test_single_distribution_remote_and_uploads_rejected( | ||
| rust_repo_factory, | ||
| rust_remote_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """A single distribution cannot have both a remote and allow_uploads.""" | ||
| repo = rust_repo_factory() | ||
| remote = rust_remote_factory(url=CRATES_IO_URL) | ||
|
|
||
| with pytest.raises(ApiException) as exc: | ||
| rust_distribution_factory( | ||
| repository=repo.pulp_href, remote=remote.pulp_href, allow_uploads=True | ||
| ) | ||
| assert exc.value.status == 400 | ||
|
|
||
|
|
||
| def test_upload_distribution_on_pull_through_repo_rejected( | ||
| rust_repo_factory, | ||
| rust_remote_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """A repo already used for pull-through caching cannot get an upload distribution.""" | ||
| repo = rust_repo_factory() | ||
| remote = rust_remote_factory(url=CRATES_IO_URL) | ||
|
|
||
| # First distribution caches into the repo via a remote. | ||
| rust_distribution_factory(repository=repo.pulp_href, remote=remote.pulp_href) | ||
|
|
||
| # A second distribution accepting uploads into the same repo must be rejected. | ||
| with pytest.raises(ApiException) as exc: | ||
| rust_distribution_factory(repository=repo.pulp_href, allow_uploads=True) | ||
| assert exc.value.status == 400 | ||
|
|
||
|
|
||
| def test_pull_through_distribution_on_upload_repo_rejected( | ||
| rust_repo_factory, | ||
| rust_remote_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """A repo already used for uploads cannot get a pull-through distribution.""" | ||
| repo = rust_repo_factory() | ||
| remote = rust_remote_factory(url=CRATES_IO_URL) | ||
|
|
||
| # First distribution accepts uploads into the repo. | ||
| rust_distribution_factory(repository=repo.pulp_href, allow_uploads=True) | ||
|
|
||
| # A second distribution caching into the same repo must be rejected. | ||
| with pytest.raises(ApiException) as exc: | ||
| rust_distribution_factory(repository=repo.pulp_href, remote=remote.pulp_href) | ||
| assert exc.value.status == 400 | ||
|
|
||
|
|
||
| def test_upload_distribution_on_repo_with_remote_rejected( | ||
| rust_repo_factory, | ||
| rust_remote_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """A repo with its own remote (for syncing) cannot get an upload distribution.""" | ||
| remote = rust_remote_factory(url=CRATES_IO_URL) | ||
| repo = rust_repo_factory(remote=remote.pulp_href) | ||
|
|
||
| with pytest.raises(ApiException) as exc: | ||
| rust_distribution_factory(repository=repo.pulp_href, allow_uploads=True) | ||
| assert exc.value.status == 400 | ||
|
|
||
|
|
||
| def test_separate_repos_for_cache_and_uploads_succeed( | ||
| rust_repo_factory, | ||
| rust_remote_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """Using separate repositories for caching and uploads is allowed.""" | ||
| remote = rust_remote_factory(url=CRATES_IO_URL) | ||
|
|
||
| cache_repo = rust_repo_factory() | ||
| cache_distro = rust_distribution_factory( | ||
| repository=cache_repo.pulp_href, remote=remote.pulp_href | ||
| ) | ||
| assert cache_distro is not None | ||
|
|
||
| upload_repo = rust_repo_factory() | ||
| upload_distro = rust_distribution_factory(repository=upload_repo.pulp_href, allow_uploads=True) | ||
| assert upload_distro is not None | ||
|
|
||
|
|
||
| def test_multiple_upload_distributions_on_one_repo_succeed( | ||
| rust_repo_factory, | ||
| rust_distribution_factory, | ||
| ): | ||
| """Multiple upload distributions may share one repo (no caching involved).""" | ||
| repo = rust_repo_factory() | ||
|
|
||
| first = rust_distribution_factory(repository=repo.pulp_href, allow_uploads=True) | ||
| second = rust_distribution_factory(repository=repo.pulp_href, allow_uploads=True) | ||
| assert first is not None | ||
| assert second is not None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems fragile, at the very least the Distribution could probably be pointing to a repository version - "frozen" so to speak.
Which probably ought to also disable uploads?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But why would anyone try to upload something to a repository version? We are trying to prevent mixing of cached and uploaded content and uploading to a repo version doesn't lead to it.
I'm open to other solutions, but I don't see an issue with this one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They're not uploading to the repository version, they're uploading to the distribution, which has a repo version set. And it shouldn't work. I think right now (not under this PR, I haven't checked, but master branch) it would upload to the repository and make new versions even though the distribution & distributed content doesn't change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understood the code correctly, uploading via cargo publish won't work if a distribution has repo_version set, since it relies on distro.repository having a value: https://github.com/pulp/pulp_rust/blob/main/pulp_rust/app/views.py#L347
And base distribution class doesn't allow for a distro to have both repo and repo_version set.
https://github.com/pulp/pulpcore/blob/main/pulpcore/app/serializers/publication.py#L315
Was that your concern or am I missing smt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose that covers it, although it's not a very good error message for this specific case.