chore: Merge changes from 0.10.x to main - #3227
Draft
dannycjones wants to merge 9 commits into
Draft
dannycjones wants to merge 9 commits into
dannycjones wants to merge 9 commits into
Conversation
* chore: Bump version to 0.11.0, update changelog * Update changelog
fix: Fix CI failure due to integration test's Hive install (apache#3174) Install Hive directly (cherry picked from commit 4687d26) Signed-off-by: Jannik Steinmann <jannik.steinmann@datadoghq.com> Co-authored-by: Jannik Steinmann <jannik.steinmann@datadoghq.com>
* chore: exclude repo-only files from the source archive (apache#3189) The release tarball is built with git archive and ships .asf.yaml, .devcontainer, .github, .gitignore, .gitattributes, and .idea, none of which are needed to build, test, or verify from source. Ryan raised this in the 0.2.0 vote and Kurtis called out .idea in the 0.9.1 vote. Files the verification scripts rely on (.licenserc.yaml, lint configs, dev/) stay in. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit d6c2eb4) * fix(ci): keep .github/actions in the source archive (apache#3193) `.github export-ignore` (apache#3189) hides `.github/actions` from repository archives. GitHub resolves the `uses: ./...` self-repository references by fetching such an archive, so every workflow that uses a local composite action fails at action resolution: Can't find 'action.yml', 'action.yaml' or 'Dockerfile' for action 'apache/iceberg-rust/.github/actions/setup-builder@<sha>' List the .github entries individually instead, so the repo-only files apache#3189 targeted stay out of the release tarball while `.github/actions` remains resolvable. A `-export-ignore` negation on the subdirectory does not work: git does not descend into an export-ignored directory. (cherry picked from commit 6f062aa) --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Xander <zander181@googlemail.com>
…rty-macro (apache#3199) fix: break dev-dependency cycle that blocks publishing iceberg-property-macro (apache#3184) * fix: break dev-dependency cycle that blocks publishing iceberg-property-macro `iceberg` depends on `iceberg-property-macro`, and the macro crate had a versioned dev-dependency on `iceberg` via `workspace = true`. When cargo packages the macro crate for publishing it must resolve `iceberg ^0.11.0` from crates.io, which does not exist until the next step, so `cargo publish --workspace` fails with "failed to select a version". This is the same failure that broke the 0.5.0 publish and it reproduces on the v0.11.0-rc.1 tag. Make the dev-dependency path-only so cargo strips it when packaging. Mark the macro crate as internal to `iceberg` in its description and README, and add a `cargo publish --workspace --dry-run` step to CI so an unpublishable crate fails on the PR instead of after the release vote. * ci: split publish dry-run between PR CI, RC tags, and RC creation The full `cargo publish --dry-run` compiles every packaged crate and took 13 minutes per PR. PR CI now runs it with `--no-verify`, which still catches dependency cycles, unresolvable versions, and missing metadata in seconds. The compiled dry-run moves to where it matters for a release: - `create_rc.sh` runs it before creating any artifact or the RC tag, so a crate that cannot be published fails RC creation. Opt out with `--check_publish 0`. - `publish.yml` runs it on RC tag pushes, which previously did nothing, so the exact tag is also checked on the MSRV toolchain the real publish uses. * docs: drop RC tag workflow verification note from release guide The RC creation script now blocks on the publish dry-run, so the workflow run on the tag is a secondary check that does not need a manual step. * docs: note requirements for create_rc.sh --check_publish * Apply suggestion from @dannycjones --------- (cherry picked from commit 0aeccb8) Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Docker Hub no longer serves the MinIO server and client repositories, preventing integration test containers from starting. Use the corresponding Quay images with the existing release tags. Generated-by: Codex (cherry picked from commit 9e1670d) Co-authored-by: Sreesh Maheshwar <maheshwarsreesh@gmail.com> Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
) (apache#3218) * ci: check new crates are reserved on crates.io before merge Trusted publishing can't create a crate, and the release workflow publishes the whole workspace in one `cargo publish --workspace` run, which cargo aborts if any crate already exists at the release version. So the existing "publish the new crate by hand after the workflow fails" instructions no longer work. Add a lint step that fails if a publishable crate is missing from crates.io, and rewrite the release guide appendix around reserving the crate with a placeholder version when it is added. * docs: reserve the crate name with a standalone placeholder crate Overriding the version inside the workspace fails because iceberg depends on the new crate at the workspace version. Verified with cargo publish --dry-run. * ci: match the existing publishable-crate filter; document partial publish recovery * docs: clarify that reserving a crate needs a token with the publish-new scope Owning the existing crates does not imply the token in ~/.cargo/credentials.toml has publish-new, and crates.io reports a missing scope as a generic authentication failure. Say so, and point at where to create the token. * docs: fix stale first-publish note, empty placeholder, and owner invitations - The release steps still said the publish workflow is expected to fail on new crates and linked to a heading that no longer exists. - `cargo new --lib` generates a non-empty lib.rs; use mkdir and touch. - `cargo owner --add <user>` sends an invitation that must be accepted. * ci: fail the crates.io check if the crate list is empty A failed `cargo metadata` inside `for pkg in $(...)` never trips `bash -e`, so the loop ran zero times and the step passed without checking anything. Capture the list first and fail if it is empty. --------- (cherry picked from commit cb912c9) Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* chore: Update changelog for 0.11.0 RC2 * chore: Add apache#3202 MinIO Quay backport to changelog
Brings the 0.11.0 release metadata back to `main`. Only `CHANGELOG.md`, `Cargo.toml`, and `Cargo.lock` change; everything else on `0.11.x` is either already on `main` or was deliberately superseded there. `CHANGELOG.md` and `Cargo.toml` are three-way merged, not copied from the release branch. `main` has moved its dependencies on since the 0.11.x cut (arrow 58.4 -> 59.2, DataFusion 54 -> 55, pyo3 0.28 -> 0.29, and datafusion-ffi dropped in apache#3149), and a merge keeps those while taking the release branch's version bumps. The result is `workspace.package.version` 0.10.1 -> 0.11.0, the nine workspace path dependencies 0.10.0 -> 0.11.0, and the `## [v0.11.0]` changelog section. No third-party dependency version changes. `Cargo.lock` is regenerated from `main`'s lock with `cargo update --workspace`, not merged. Only the 17 workspace-member versions change. All other paths are resolved to `main`, including the two that conflict: `dev/hms/Dockerfile` (`main` is on the Hive 4.2.1 image) and `.gitattributes` (apache#3193). `dev/release/` and `deny.toml` also differ, because `main` is ahead (apache#3194) or removed things on purpose (apache#3200, apache#3149). Fixes land on `main` first and are backported, so nothing needs to travel in this direction. `DEPENDENCIES.rust.tsv` is not merged. Those files are generated during release prep on the release branch and describe the released dependency graph, which is older than `main`'s, so carrying them across would replace stale data with data that is wrong for `main`. Per apache#2706 generating them is a release-manager task and is not enforced by CI, so `main`'s copies are refreshed at the next release, on the next release branch. apache#3074 excluded them likewise.
Contributor
Author
|
Leaving in draft until 0.11.0 is released. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to #3034.
What changes are included in this PR?
Simply merging changes from
0.11.xintomain, such that it would be ready for a future release branch being cut from it. No other changes are introduced unless absolutely necessary.The exact steps for making these changes still needs documenting in the release guide.
Are these changes tested?
N/A
AI Disclosure
LLM used to draft merge commit changes.