Skip to content

Sync to upstream 1.2.0 and add PG19 Beta 4 packages - #2

Open
PThorpe92 wants to merge 16 commits into
masterfrom
pthorpe92/pg19-beta4-pg_roaringbitmap
Open

PThorpe92 wants to merge 16 commits into
masterfrom
pthorpe92/pg19-beta4-pg_roaringbitmap

Conversation

@PThorpe92

Copy link
Copy Markdown

Changes

  • 95fcdbb Pinned per-major builds + exact-package test. Dockerfile.package replaces Dockerfile.pscale-debian12 (floating postgres:18.1-bookworm, postgresql-all, arm64 under QEMU, hard-coded VERSION). It uses digest-pinned bookworm/trixie bases and exact PGDG pins (19~beta4-1.pgdg{12,13}+1, 18.6-1.pgdg{12,13}+2, 17.11-1.pgdg12+2), and enables component 19 plus pinned libpq only on PG 19 rows. One major per build, asserted. scripts/test-package checks names, .sha256 files, metadata and exact manifests (taken from debian/control.in and the real v1.2.0-1.pscale1 debs), then installs into a clean image: dpkg -i must add nothing else. It then runs initdb and CREATE EXTENSION, followed by probes: the postgres-private probe list, rb_or/rb_and, rb_build_agg/rb_or_agg, roaringbitmap64 above 2^32, a binary COPY round-trip, and corrupt blobs rejected by *_recv. Last, it runs sql/*.sql against the installed .so with postgresql-client's pg_regress.
  • 7f9d0a8 Release inventory. packaging/release-assets.txt plus a fail-closed scripts/verify-release-assets and its fixture self-test, which covers 20 broken cases.
  • 2907881 CI. packages.yml replaces release.yml. It has 10 package rows on native depot runners and one "Release inventory" job that gates the v<V> release. It runs on PRs, master, tags, merge_group and workflow_dispatch. build-test.yml (pgxn-tools) gains PG 19.
  • cb4bee4 Merge upstream v1.2.0. The fork was synced just before the release and lacked c9e0d60: roaringbitmap_recv/roaringbitmap64_recv now parse the message directly and reject empty input. It also lacked the 1.2.0 version bump in META.json and CHANGELOG. Source, Makefile, META.json and CHANGELOG now match v1.2.0. The CI matrix stays at 17–19.
  • ad45a6a Probe update for the new empty roaring bitmap binary data error, now also probed for the 64-bit type.

The only source changes come from upstream v1.2.0. The Debian 12 names are unchanged: postgresql-<M>-roaringbitmap[-dbgsym]_<V>_<arch>.deb + .sha256. Debian 13 adds pg<M>-trixie-<x86_64|arm64>-…. 40 files in all: PG 17 on Debian 12; PG 18 and 19 on Debian 12 and 13. The version is still ${tag#v}-1, so the next tag v1.2.0-1.pscale2 ships …_1.2.0-1.pscale2-1_<arch>.deb. postgres-private's --pattern still matches exactly one asset per major/arch.

ChenHuajun and others added 13 commits June 27, 2026 00:00
use StringInfo parse input of roaringbitmap_recv and Bump the version to 1.2.0
add -Wno-missing-variable-declarations to avoid build error
add postgresql 19 to build-test.yml
Dockerfile.package builds the .debs for exactly one PostgreSQL major
(17, 18 or 19) on a digest-pinned Debian 12 or 13 base, from exact PGDG
pins. Component 19 and the libpq pins are enabled only on PG 19 rows, and
postgresql-all/server-dev-all are never installed. pg_buildext is
restricted to the one major via PG_SUPPORTED_VERSIONS and asserted, as
is /usr/lib/postgresql. Debian 12 assets keep the canonical names;
Debian 13 assets get the release-only pg<M>-trixie-<arch>- prefix. Each
.deb gets a `sha256sum` sidecar, as the v1.2.0-1.pscale1 release does.

scripts/test-package tests the exact assets in a clean image with only
the pinned PostgreSQL runtime: asset set and sidecars, metadata and
manifests (derived from debian/control.in and the v1.2.0-1.pscale1
assets), dpkg -i adding nothing else, then initdb, CREATE EXTENSION and
behavior probes (the postgres-private probes, rb_or/rb_and, aggregates,
roaringbitmap64 above 2^32, a binary COPY round-trip, corrupt blobs
rejected by *_recv). Finally it runs sql/*.sql against the installed
.so with postgresql-client's pg_regress. --probes-only runs the
server-side part against a local installation.
packaging/release-assets.txt lists the 10 consumed (major, Debian,
arch) rows: PG 17 on Debian 12, PG 18 and 19 on Debian 12 and 13, on
amd64 and arm64. Each row ships the main and -dbgsym .deb plus their
.sha256 sidecars, 40 files in all, as v1.2.0-1.pscale1 does for its rows.

scripts/verify-release-assets rejects missing, duplicate, unexpected or
malformed assets, orphaned or wrong sidecars, mixed versions and versions
without the Debian -1 revision. scripts/test-verify-release-assets runs it
against the good set and 20 broken sets.
packages.yml replaces release.yml and Dockerfile.pscale-debian12, which
built every major in one floating postgres:18.1-bookworm image (arm64
under QEMU) with a hard-coded VERSION default. It runs on pull requests,
master, v* tags, merge_group and workflow_dispatch:

- 10 package rows on native depot runners: docker build --target test,
  then scripts/test-package on the exact assets, then upload.
- "Release inventory": the verifier self-test plus the verifier on all
  40 downloaded files. It is the single aggregate check.
- release (tags only): verify again and upload the .debs and .sha256
  sidecars. The package version is still `${tag#v}-1`, so
  v1.2.0-1.pscale2 ships postgresql-<M>-roaringbitmap_1.2.0-1.pscale2-1_<arch>.deb.

build-test.yml (pgxn-tools source build + installcheck) gains PG 19.
The fork was synced to upstream before the 1.2.0 release and lacked:
- c9e0d60 use StringInfo parse input of roaringbitmap_recv (and the
  64-bit recv): reads the binary input directly from the message and
  rejects empty input
- eee57df/896e1f4 version 1.2.0 in META.json and CHANGELOG
- bdd8e95 -Wno-missing-variable-declarations (already in the fork)
- d6a2885 PG 19 in build-test.yml (we keep the 17-19 matrix)

Source, Makefile, META.json and CHANGELOG.md now match v1.2.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Upstream 1.2.0 (c9e0d60) rejects empty binary input in roaringbitmap_recv
and roaringbitmap64_recv with 'empty roaring bitmap binary data'. Expect
that, and probe the 64-bit type's empty input too.

@hiradp hiradp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Packages workflow hasn't run on this PR yet. Run 37508482475 has been queued since 18:04 UTC and Package version never got a runner, so I suspect the depot-* labels aren't enabled for this repo. I haven't checked the Depot side.

Since this deletes release.yml, I'd like to see one green Packages run on the head before it merges. Right now nothing in Dockerfile.package, scripts/test-package or the verifier has run in CI.

Not blocking:

  • Release inventory has no if:, so it's skipped rather than failed when a package row fails. That's harmless while master has no required checks. If it becomes the required check, it needs if: always() and an explicit check of needs.*.result.
  • The PGDG pins will fail every run once PGDG drops those versions, 19~beta4 first. Renovate annotations on the matrix would make that a PR instead of a surprise.
  • canonical-on-trixie in scripts/test-verify-release-assets only deletes an asset, so it repeats the missing case. It needs the unprefixed file added to test what its label says.
  • stray-changes and unexpected-pg16 take the same unexpected asset name branch as stray-buildinfo and could go.

…drop repeats

canonical-on-trixie only deleted the prefixed Debian 13 asset, which
repeated the missing case. Ship that build under its canonical name in a
per-job directory instead and assert it is rejected as a duplicate of
the Debian 12 tuple.

stray-changes and unexpected-pg16 hit the same unexpected-asset-name
branch as stray-buildinfo; drop them.
Run 37508482475 never got a runner for its depot-* jobs, apparently
because those labels aren't available to this public repository. Use
ubuntu-latest, as release.yml and build-test.yml did, and the native
GitHub-hosted ubuntu-24.04-arm for arm64 rows instead of release.yml's
QEMU emulation.
Release inventory now always runs and its first step fails unless every
needed job succeeded, so it fails rather than skips when a package row
fails and can serve as the required check. release still runs only after
inventory succeeds.

Add renovate comments to the pinned PGDG versions in the deb-datasource
form postgres-private uses (suite/arch/component per row).
@PThorpe92
PThorpe92 force-pushed the pthorpe92/pg19-beta4-pg_roaringbitmap branch from 24fe7e4 to d53c9e4 Compare October 6, 2026 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants