Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions .github/workflows/smoke-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ on:

permissions: {}

env:
SDP_VERSION: 1.2.3

jobs:
generate-matrix:
name: Generate Version List
Expand Down Expand Up @@ -62,7 +65,7 @@ jobs:
uses: ./build-product-image
with:
product-version: ${{ matrix.versions }}
sdp-version: 1.2.3
sdp-version: ${{ env.SDP_VERSION }}
boil-config-file: smoke/container-image/boil.toml
boil-version: latest
registry-namespace: smoke
Expand Down Expand Up @@ -96,14 +99,32 @@ jobs:
with:
persist-credentials: false

- name: Install boil
uses: ./setup-tools
with:
boil-version: latest

# NOTE (@Techassi): I'm not super happy about how this looks/works, but as already mentioned,
# this should only be a stepping stone towards a more robust solution.
- name: Extract floating image index manifest tag
id: extract-floating-tag
env:
IMAGE_VERSION: ${{ matrix.versions }}
shell: bash
run: |
FLOATING_IMAGE_INDEX_MANIFEST_TAG=$(boil tools floating-tag "${SDP_VERSION}-pr-321")
FLOATING_IMAGE_INDEX_MANIFEST_TAG="${IMAGE_VERSION}-stackable${FLOATING_IMAGE_INDEX_MANIFEST_TAG}"
echo "FLOATING_IMAGE_INDEX_MANIFEST_TAG=[\"$FLOATING_IMAGE_INDEX_MANIFEST_TAG\"]" | tee -a "$GITHUB_OUTPUT"

- name: Publish and Sign Image Index Manifest to oci.stackable.tech
uses: ./publish-image-index-manifest
with:
image-registry-uri: oci.stackable.tech
image-registry-username: robot$smoke+github-action-build
image-registry-password: ${{ secrets.HARBOR_ROBOT_SMOKE_GITHUB_ACTION_BUILD_SECRET }}
image-repository: smoke/container-image
image-index-manifest-tag: ${{ matrix.versions }}-stackable1.2.3-pr-321
canonical-image-index-manifest-tag: ${{ matrix.versions }}-stackable${{ env.SDP_VERSION }}-pr-321
other-image-index-manifest-tags: ${{ steps.extract-floating-tag.outputs.FLOATING_IMAGE_INDEX_MANIFEST_TAG }}

publish-helm-chart:
name: Package/Publish ${{ matrix.versions }} Helm Chart
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,17 @@ particular step in a workflow.

## Definitions

| Name | Example |
| ---------------------------------- | ---------------------------------------------------------------------- |
| Image Registry | `oci.stackable.tech` |
| Image Repository | `stackable/kafka` |
| Image Index Manifest Tag | `3.4.1-stackable0.0.0-dev` |
| Image Manifest Tag | `3.4.1-stackable0.0.0-dev-amd64` |
| Image Repository URI | `oci.stackable.tech/sdp/kafka` |
| Image Index URI (if multi-arch) | `oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev` |
| Image Manifest URI (if multi-arch) | `oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev-amd64` |
| Image Repo Digest | `oci.stackable.tech/sdp/kafka@sha256:917f800259ef4915f976...` |
| Digest | `sha256:917f800259ef4915f976e93987b752fd64debf347568610d7f685d2022...` |
| Name | Example |
| ---------------------------------------- | ---------------------------------------------------------------------- |
| Image Registry | `oci.stackable.tech` |
| Image Repository | `stackable/kafka` |
| Image Index Manifest Tag | `3.4.1-stackable0.0.0-dev` |
| Image Manifest Tag | `3.4.1-stackable0.0.0-dev-amd64` |
| Image Repository URI | `oci.stackable.tech/sdp/kafka` |
| Image Index Manifest URI (if multi-arch) | `oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev` |
| Image Manifest URI (if multi-arch) | `oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev-amd64` |
| Image Repo Digest | `oci.stackable.tech/sdp/kafka@sha256:917f800259ef4915f976...` |
| Digest | `sha256:917f800259ef4915f976e93987b752fd64debf347568610d7f685d2022...` |

## Available Actions

Expand Down
21 changes: 11 additions & 10 deletions publish-image-index-manifest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,17 @@ This action creates an image index manifest, publishes it, and signs it. It does

### Inputs

| Input | Required (Default) | Description |
| -------------------------- | ------------------------- | ----------------------------------------------------------------------------------- |
| `image-registry-uri` | Yes | The image registry URI, eg `oci.stackable.tech` |
| `image-registry-username` | Yes | The username used to access the image registry |
| `image-registry-password` | Yes | The password used to access the image registry |
| `image-repository` | Yes | The path to the image, eg `sdp/kafka` |
| `image-index-manifest-tag` | Yes | Human-readable tag without architecture information, eg `3.4.1-stackable0.0.0-dev` |
| `image-architectures` | No (`["amd64", "arm64"]`) | The list of architectures the to-bo-published image was built for |
| `cosign-retries` | No (3) | The number of times cosign operations should be retried |
| `cosign-retry-timeout` | No (30s) | Duration to wait before a new cosign operation is retried, format: `NUMBER[SUFFIX]` |
| Input | Required (Default) | Description |
| ------------------------------------ | ------------------------- | ----------------------------------------------------------------------------------- |
| `image-registry-uri` | Yes | The image registry URI, eg `oci.stackable.tech` |
| `image-registry-username` | Yes | The username used to access the image registry |
| `image-registry-password` | Yes | The password used to access the image registry |
| `image-repository` | Yes | The path to the image, eg `sdp/kafka` |
| `canonical-image-index-manifest-tag` | Yes | The canonical image index manifest tag which doesn't contain any architecture specific suffixes, like: `3.4.1-stackable0.0.0-dev` |
| `other-image-index-manifest-tags` | No (`[]`) | A JSON string containing a list of other/additional tags |
| `image-architectures` | No (`["amd64", "arm64"]`) | The list of architectures the to-bo-published image was built for |
| `cosign-retries` | No (3) | The number of times cosign operations should be retried |
| `cosign-retry-timeout` | No (30s) | Duration to wait before a new cosign operation is retried, format: `NUMBER[SUFFIX]` |

### Outputs

Expand Down
67 changes: 46 additions & 21 deletions publish-image-index-manifest/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,16 @@ inputs:
Last segment of the path, for example `stackable/kafka` or
`k8s/sig-storage/csi-provisioner`
required: true
image-index-manifest-tag:
canonical-image-index-manifest-tag:
description: |
Human-readable tag (usually the version) without architecture information,
for example: `3.4.1-stackable0.0.0-dev`
The canonical image index manifest tag which doesn't contain any architecture specific
suffixes, like: `3.4.1-stackable0.0.0-dev`
required: true
other-image-index-manifest-tags:
description: |
A JSON string containing a list of other/additional tags which should be added to the image
index manifest alongside the always present canonical tag.
default: "[]"
image-architectures:
description: |
A JSON array of architectures to add to the image index manifest, like
Expand All @@ -38,15 +44,19 @@ inputs:
See `sleep --help` for the full details.
default: "30s"
outputs:
image-index-uri:
description: The Image Index URI.
value: ${{ steps.create-index.outputs.IMAGE_INDEX_URI }}
canonical-image-index-manifest-uri:
description: |
The canonical image index manifest URI, like oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev
value: ${{ steps.create-index.outputs.CANONICAL_IMAGE_INDEX_MANIFEST_URI }}
image-index-manifest-digest:
description: The digest (sha256:...) of the pushed image index manifest.
value: ${{ steps.create-index.outputs.IMAGE_INDEX_MANIFEST_DIGEST }}
runs:
using: composite
steps:
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0

- name: Set up Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2

Expand All @@ -61,7 +71,8 @@ runs:
id: create-index
shell: bash
env:
IMAGE_INDEX_MANIFEST_TAG: ${{ inputs.image-index-manifest-tag }}
CANONICAL_IMAGE_INDEX_MANIFEST_TAG: ${{ inputs.canonical-image-index-manifest-tag }}
OTHER_IMAGE_INDEX_MANIFEST_TAGS: ${{ inputs.other-image-index-manifest-tags }}
IMAGE_ARCHITECTURES: ${{ inputs.image-architectures }}
IMAGE_REPOSITORY: ${{ inputs.image-repository }}
REGISTRY_URI: ${{ inputs.image-registry-uri }}
Expand All @@ -70,36 +81,49 @@ runs:

# Construct the image index uri, which for example contains:
# oci.stackable.tech/sdp/kafka:3.4.1-stackable0.0.0-dev
IMAGE_INDEX_URI="$REGISTRY_URI/$IMAGE_REPOSITORY:$IMAGE_INDEX_MANIFEST_TAG"
echo "IMAGE_INDEX_URI=$IMAGE_INDEX_URI" | tee -a "$GITHUB_OUTPUT"
IMAGE_REGISTRY_URI="${REGISTRY_URI}/${IMAGE_REPOSITORY}"
CANONICAL_IMAGE_INDEX_MANIFEST_URI="${IMAGE_REGISTRY_URI}:$CANONICAL_IMAGE_INDEX_MANIFEST_TAG"
echo "CANONICAL_IMAGE_INDEX_MANIFEST_URI=$CANONICAL_IMAGE_INDEX_MANIFEST_URI" | tee -a "$GITHUB_OUTPUT"

AMEND_OPTIONS=$(
MANIFEST_SOURCES=$(
jq \
--raw-output \
--null-input \
--arg image_index_uri "$IMAGE_INDEX_URI" \
--arg canonical_image_index_uri "$CANONICAL_IMAGE_INDEX_MANIFEST_URI" \
--arg arch_list "$IMAGE_ARCHITECTURES" \
'
$arch_list
| fromjson
| [
.[] as $arch | "--amend \($image_index_uri)-\($arch)"
.[] as $arch | "\($canonical_image_index_uri)-\($arch)"
]
| join(" ")'
)

# `docker manifest push` directly returns the digest of the manifest list
# As it is an experimental feature, this might change in the future.
# We use --amend because the manifest list would be updated since we use
# the same tag: 0.0.0-dev
OTHER_TAGS=$(
echo "$OTHER_IMAGE_INDEX_MANIFEST_TAGS" | jq \
--raw-output \
--arg image_registry_uri "$IMAGE_REGISTRY_URI" \
'
[
.[] as $tag | "--tag \($image_registry_uri):\($tag)"
]
| join(" ")'
)

# Create an image index manifest based on the manifest sources (currently two images for
# amd64 and arm64). The canonical tag is always present. Other/additional tags are added
# as needed. This command also directly pushes the index manifest to the remote registry.
#
# Further reading: https://docs.docker.com/reference/cli/docker/manifest/push/
docker manifest create "$IMAGE_INDEX_URI" ${AMEND_OPTIONS[@]}
docker manifest push "$IMAGE_INDEX_URI"
# Further reading: https://docs.docker.com/reference/cli/docker/buildx/imagetools/create/
docker buildx imagetools create \
--tag "$CANONICAL_IMAGE_INDEX_MANIFEST_URI" \
${OTHER_TAGS[@]} \
${MANIFEST_SOURCES[@]}

# Get the image index manifest digest and expose it as an output, so
# callers can reference the pushed index by digest.
DIGEST=$("$GITHUB_ACTION_PATH/../.scripts/actions/get_manifest_digest.sh" "$IMAGE_INDEX_URI")
DIGEST=$("$GITHUB_ACTION_PATH/../.scripts/actions/get_manifest_digest.sh" "$CANONICAL_IMAGE_INDEX_MANIFEST_URI")
echo "IMAGE_INDEX_MANIFEST_DIGEST=$DIGEST" | tee -a "$GITHUB_OUTPUT"

- name: Sign Image Index Manifest
Expand All @@ -108,7 +132,6 @@ runs:
RETRY_TIMEOUT: ${{ inputs.cosign-retry-timeout }}
RETRY_COUNT: ${{ inputs.cosign-retries }}
RETRY_ARGS: --verbose
IMAGE_INDEX_URI: ${{ steps.create-index.outputs.IMAGE_INDEX_URI }}
DIGEST: ${{ steps.create-index.outputs.IMAGE_INDEX_MANIFEST_DIGEST }}
IMAGE_REPOSITORY: ${{ inputs.image-repository }}
REGISTRY_URI: ${{ inputs.image-registry-uri }}
Expand All @@ -117,6 +140,8 @@ runs:

# Construct the image repo digest, which for example contains:
# oci.stackable.tech/sdp/kafka@sha256:91...
# TODO (@Techassi): The image repository URI could be constructed once instead of over and
# over again in different steps
IMAGE_REPO_DIGEST="$REGISTRY_URI/$IMAGE_REPOSITORY@$DIGEST"

# This generates a signature and publishes it to the registry, next to
Expand Down