Skip to content

Commit

Permalink
[CI:DOCS] Multi-arch image workflow: Make steps generic
Browse files Browse the repository at this point in the history 
This duplicates the change from
containers/buildah#3385

Since this workflow is duplicated across three repositories, maintaining
changes becomes onerous if the item contents vary between
implementations in any way. Improve this situation by encoding the
repository-specific details into env. vars. then referencing those vars
throughout. This way, a meaningful diff can be worked with to compare
the contents across repositories.

Also included are abstractions for the specific command used to obtain
the project version, and needed details for filtering the output. Both
of these vary across the Buildah, Skopeo, and Podman repos.

NOTE: This change requires the names of two github action secrets
to be updated: SKOPEO_QUAY_USERNAME -> REPONAME_QUAY_USERNAME
(and *PASSWORD).

Signed-off-by: Chris Evich <[email protected]>
  • Loading branch information
@cevich
cevich committed Jul 16, 2021
1 parent 643920b commit e460b9a
Showing 1 changed file with 56 additions and 49 deletions.
105 changes: 56 additions & 49 deletions .github/workflows/multi-arch-build.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---

# Please see contrib/skopeoimage/README.md for details on the intentions
# Please see contrib/<reponame>image/README.md for details on the intentions
# of this workflow.
#
# BIG FAT WARNING: This workflow is duplicated across containers/skopeo,
Expand All @@ -11,7 +11,7 @@
name: build multi-arch images

on:
# Upstream skopeo tends to be very active, with many merges per day.
# Upstream tends to be very active, with many merges per day.
# Only run this daily via cron schedule, or manually, not by branch push.
schedule:
- cron: '0 8 * * *'
Expand All @@ -20,19 +20,23 @@ on:

jobs:
multi:
name: multi-arch Skopeo build
name: multi-arch image build
env:
SKOPEO_QUAY_REGISTRY: quay.io/skopeo
REPONAME: skopeo # No easy way to parse this out of $GITHUB_REPOSITORY
# Server/namespace value used to format FQIN
REPONAME_QUAY_REGISTRY: quay.io/skopeo
CONTAINERS_QUAY_REGISTRY: quay.io/containers
# list of architectures for build
PLATFORMS: linux/amd64,linux/s390x,linux/ppc64le,linux/arm64
# Command to execute in container to obtain project version number
VERSION_CMD: "--version" # skopeo is the entrypoint

# build several images (upstream, testing, stable) in parallel
strategy:
# By default, failure of one matrix item cancels all others
fail-fast: false
matrix:
# Builds are located under contrib/skopeoimage/<source> directory
# Builds are located under contrib/<reponame>image/<source> directory
source:
- upstream
- testing
Expand All @@ -57,53 +61,57 @@ jobs:
driver-opts: network=host
install: true

- name: Build and locally push Skopeo
- name: Build and locally push image
uses: docker/build-push-action@v2
with:
context: contrib/skopeoimage/${{ matrix.source }}
file: ./contrib/skopeoimage/${{ matrix.source }}/Dockerfile
context: contrib/${{ env.REPONAME }}image/${{ matrix.source }}
file: ./contrib/${{ env.REPONAME }}image/${{ matrix.source }}/Dockerfile
platforms: ${{ env.PLATFORMS }}
push: true
tags: localhost:5000/skopeo/${{ matrix.source }}
tags: localhost:5000/${{ env.REPONAME }}/${{ matrix.source }}

# Simple verification that stable images work, and
# also grab version number use in forming the FQIN.
- name: amd64 container sniff test
if: matrix.source == 'stable'
id: sniff_test
run: |
VERSION_OUTPUT="$(docker run localhost:5000/skopeo/${{ matrix.source }} --version)"
podman pull --tls-verify=false \
localhost:5000/$REPONAME/${{ matrix.source }}
VERSION_OUTPUT=$(podman run \
localhost:5000/$REPONAME/${{ matrix.source }} \
$VERSION_CMD)
echo "$VERSION_OUTPUT"
VERSION=$(grep -Em1 '^skopeo version' <<<"$VERSION_OUTPUT" | awk '{print $3}')
VERSION=$(awk -r -e "/^${REPONAME} version /"'{print $3}' <<<"$VERSION_OUTPUT")
test -n "$VERSION"
echo "::set-output name=version::${VERSION}"
echo "::set-output name=version::$VERSION"
- name: Generate skopeo reg. image FQIN(s)
id: skopeo_reg
- name: Generate image FQIN(s) to push
id: gen_fqin
run: |
if [[ "${{ matrix.source }}" == 'stable' ]]; then
# The `skopeo version` in image just built
# The command version in image just built
VERSION='v${{ steps.sniff_test.outputs.version }}'
# workaround vim syntax-highlight bug: '
# Image tags previously pushed to quay
ALLTAGS=$(skopeo list-tags \
docker://$SKOPEO_QUAY_REGISTRY/stable | \
docker://$REPONAME_QUAY_REGISTRY/stable | \
jq -r '.Tags[]')
# New version? Push quay.io/skopeo/stable:vX.X.X and :latest
# New version? Push quay.io/$REPONAME/stable:vX.X.X and :latest
if ! fgrep -qx "$VERSION" <<<"$ALLTAGS"; then
# Assume version-tag is also the most up to date (i.e. "latest")
FQIN="$SKOPEO_QUAY_REGISTRY/stable:$VERSION,$SKOPEO_QUAY_REGISTRY/stable:latest"
FQIN="$REPONAME_QUAY_REGISTRY/stable:$VERSION,$REPONAME_QUAY_REGISTRY/stable:latest"
else # Not a new version-tagged image
# Assume other contents changed, so this is the "new" latest.
FQIN="$SKOPEO_QUAY_REGISTRY/stable:latest"
FQIN="$REPONAME_QUAY_REGISTRY/stable:latest"
fi
elif [[ "${{ matrix.source }}" == 'testing' ]]; then
# Assume some contents changed, always push latest testing.
FQIN="$SKOPEO_QUAY_REGISTRY/testing:latest"
FQIN="$REPONAME_QUAY_REGISTRY/testing:latest"
elif [[ "${{ matrix.source }}" == 'upstream' ]]; then
# Assume some contents changed, always push latest upstream.
FQIN="$SKOPEO_QUAY_REGISTRY/upstream:latest"
FQIN="$REPONAME_QUAY_REGISTRY/upstream:latest"
else
echo "::error::Unknown matrix item '${{ matrix.source }}'"
exit 1
Expand All @@ -122,14 +130,14 @@ jobs:
VERSION='v${{ steps.sniff_test.outputs.version }}'
# workaround vim syntax-highlight bug: '
ALLTAGS=$(skopeo list-tags \
docker://$CONTAINERS_QUAY_REGISTRY/skopeo | \
docker://$CONTAINERS_QUAY_REGISTRY/$REPONAME | \
jq -r '.Tags[]')
# New version? Push quay.io/containers/skopeo:vX.X.X and latest
# New version? Push quay.io/containers/$REPONAME:vX.X.X and latest
if ! fgrep -qx "$VERSION" <<<"$ALLTAGS"; then
FQIN="$CONTAINERS_QUAY_REGISTRY/skopeo:$VERSION,$CONTAINERS_QUAY_REGISTRY/skopeo:latest"
FQIN="$CONTAINERS_QUAY_REGISTRY/$REPONAME:$VERSION,$CONTAINERS_QUAY_REGISTRY/$REPONAME:latest"
else # Not a new version-tagged image, only update latest.
FQIN="$CONTAINERS_QUAY_REGISTRY/skopeo:latest"
FQIN="$CONTAINERS_QUAY_REGISTRY/$REPONAME:latest"
fi
echo "::warning::Pushing $FQIN"
echo "::set-output name=fqin::${FQIN}"
Expand All @@ -149,55 +157,54 @@ jobs:
DELIMITER
EOF
# Separate steps to login and push for skopeo and containers quay
# repositories are required, because 2 sets of credentials are used and `docker
# login` as well as `skopeo login` do not support having 2 different
# credential sets for 1 registry.
# At the same time reuse of non-shell steps is not supported by Github Actions
# via anchors or composite actions
# Separate steps to login and push for $REPONAME_QUAY_REGISTRY and
# $CONTAINERS_QUAY_REGISTRY are required, because 2 sets of credentials
# are used and namespaced within the registry. At the same time, reuse
# of non-shell steps is not supported by Github Actions nor are YAML
# anchors/aliases, nor composite actions.

# Push to 'skopeo' Quay repo for stable, testing. and upstream
- name: Login to 'skopeo' Quay registry
# Push to $REPONAME_QUAY_REGISTRY for stable, testing. and upstream
- name: Login to ${{ env.REPONAME_QUAY_REGISTRY }}
uses: docker/login-action@v1
if: steps.skopeo_reg.outputs.push == 'true'
if: steps.gen_fqin.outputs.push == 'true'
with:
registry: ${{ env.SKOPEO_QUAY_REGISTRY }}
registry: ${{ env.REPONAME_QUAY_REGISTRY }}
# N/B: Secrets are not passed to workflows that are triggered
# by a pull request from a fork
username: ${{ secrets.SKOPEO_QUAY_USERNAME }}
password: ${{ secrets.SKOPEO_QUAY_PASSWORD }}
username: ${{ secrets.REPONAME_QUAY_USERNAME }}
password: ${{ secrets.REPONAME_QUAY_PASSWORD }}

- name: Push images to 'skopeo' Quay
- name: Push images to ${{ steps.gen_fqin.outputs.fqin }}
uses: docker/build-push-action@v2
if: steps.skopeo_reg.outputs.push == 'true'
if: steps.gen_fqin.outputs.push == 'true'
with:
cache-from: type=registry,ref=localhost:5000/skopeo/${{ matrix.source }}
cache-from: type=registry,ref=localhost:5000/${{ env.REPONAME }}/${{ matrix.source }}
cache-to: type=inline
context: contrib/skopeoimage/${{ matrix.source }}
file: ./contrib/skopeoimage/${{ matrix.source }}/Dockerfile
context: contrib/${{ env.REPONAME }}image/${{ matrix.source }}
file: ./contrib/${{ env.REPONAME }}image/${{ matrix.source }}/Dockerfile
platforms: ${{ env.PLATFORMS }}
push: true
tags: ${{ steps.skopeo_reg.outputs.fqin }}
tags: ${{ steps.gen_fqin.outputs.fqin }}
labels: |
${{ env.LABELS }}
# Push to 'containers' Quay repo only stable skopeo
- name: Login to 'containers' Quay registry
# Push to $CONTAINERS_QUAY_REGISTRY only stable
- name: Login to ${{ env.CONTAINERS_QUAY_REGISTRY }}
if: steps.containers_reg.outputs.push == 'true'
uses: docker/login-action@v1
with:
registry: ${{ env.CONTAINERS_QUAY_REGISTRY}}
username: ${{ secrets.CONTAINERS_QUAY_USERNAME }}
password: ${{ secrets.CONTAINERS_QUAY_PASSWORD }}

- name: Push images to 'containers' Quay
- name: Push images to ${{ steps.containers_reg.outputs.fqin }}
if: steps.containers_reg.outputs.push == 'true'
uses: docker/build-push-action@v2
with:
cache-from: type=registry,ref=localhost:5000/skopeo/${{ matrix.source }}
cache-from: type=registry,ref=localhost:5000/${{ env.REPONAME }}/${{ matrix.source }}
cache-to: type=inline
context: contrib/skopeoimage/${{ matrix.source }}
file: ./contrib/skopeoimage/${{ matrix.source }}/Dockerfile
context: contrib/${{ env.REPONAME }}image/${{ matrix.source }}
file: ./contrib/${{ env.REPONAME }}image/${{ matrix.source }}/Dockerfile
platforms: ${{ env.PLATFORMS }}
push: true
tags: ${{ steps.containers_reg.outputs.fqin }}
Expand Down

0 comments on commit e460b9a

Please sign in to comment.