ci: Test devcontainer image with repos

.github/workflows/ci.yaml

@@ -14,6 +14,7 @@ name: Validate DevContainer
 description: This workflow is checking that updates do not break stuff. If on main branch, publish to "latest" tag.
 on:
   pull_request:
+    -types: [opened, synchronize, reopened, labeled]
   push:
     branches:
       - main
@@ -71,6 +72,13 @@ jobs:
             # Test
             ./scripts/test.sh
 
+            # Test consumer repository, arm64 is skipped because not all bazel dependencies are available for arm64
+            # By default this is skipped in pull requests, unless the label "test-consumer" is added
+            if [ "${{ matrix.os }}" = "amd64" ] && ( [ "${{ github.event_name }}" != "pull_request" ] || echo "${{ toJSON(github.event.pull_request.labels) }}" | grep -q "test-consumer" ); then
+              ./scripts/test_consumer.sh "https://github.com/eclipse-score/inc_someip_gateway.git" "14a733a1f3d5f6fa76ac13365d57c8df30b62a86"
+              ./scripts/test_consumer.sh "https://github.com/eclipse-score/score.git" "7cbb86d54dced80e33a6405bc92f8abe62200ad4"
+            fi
+
             # Optionally: Publish
             # We do not use the push feature of devcontainers/ci here, since that would push the wrong container.
             # Instead, we use the publish script which pushes the correct container (residing in src/s-core-devcontainer).

README.md

@@ -18,7 +18,7 @@ SPDX-License-Identifier: Apache-2.0
 This repository contains the common [development container](https://containers.dev) for [Eclipse S-CORE](https://github.com/eclipse-score).
 It contains all tools required to develop (modify, build, ...) Eclipse S-CORE.
 All tool version are well-defined, and all tools are pre-configured to work as expected for Eclipse S-CORE development.
-The container is [pre-built](https://containers.dev/guide/prebuild) in GitHub Actions as part of this repository, tested, published, and ready for use.
+The container is [pre-built](https://containers.dev/guide/prebuild) in GitHub Actions as part of this repository, [tested, published](./docs/ci.md), and ready for use.
 
 Using the pre-built container in an Eclipse S-CORE repository is described in the [Usage](#usage) section.
 
@@ -312,6 +312,10 @@ The Visual Studio Code instance related to the targeted S-CORE module will now a
 If not, press <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> and run from there "Dev Containers: Rebuilt Container Without Cache".
 Do so, and you have a running instance of `S-CORE DevContainer` related to the targeted S-CORE module.
 
+### Consumer tests
+
+To enable consumer tests in the pull request, read the [ci.md](./docs/ci.md) document.
+
 ### Version Pinning
 
 The `S-CORE DevContainer` pins feature and tool versions.

docs/README.md

@@ -98,14 +98,3 @@ It strengthens the supply chain by pinning versions and hashes, sourcing feature
 Pre-built images have a higer availability than the set of all tools which are installed (one download from a location controlled by S-CORE vs. many downloads from "everywhere").
 Pre-built images can be easily archived anywhere, e.g. for reproducibility of builds in real production use-cases.
 It also enforces a clear separation of concerns: general tooling is delivered through reusable features, S-CORE–specific logic lives in a dedicated feature, and image composition plus publishing are centralized.
-
-## Release Automation
-
-Releases are cut automatically once per week from `main`, but only if commits were added since the latest `v<major>.<minor>.<patch>` tag.
-The scheduled workflow creates the git tag and GitHub release, and the existing tag-triggered release workflow then builds, tests and publishes the matching container image.
-
-The next semantic version is derived from commit messages since the previous release:
-
-* breaking changes (`!` in the conventional commit header or `BREAKING CHANGE:` in the body) increment the major version
-* `feat` commits increment the minor version
-* every other commit increments the patch version so maintenance-only weeks still publish a new immutable image

docs/ci.md

@@ -0,0 +1,55 @@
+<!--
+*******************************************************************************
+Copyright (c) 2026 Contributors to the Eclipse Foundation
+
+See the NOTICE file(s) distributed with this work for additional
+information regarding copyright ownership.
+
+This program and the accompanying materials are made available under the
+terms of the Apache License Version 2.0 which is available at
+https://www.apache.org/licenses/LICENSE-2.0
+
+SPDX-FileCopyrightText: 2026 Contributors to the Eclipse Foundation
+SPDX-License-Identifier: Apache-2.0
+*******************************************************************************
+-->
+
+# Continuous Integration
+
+The CI workflow (`.github/workflows/ci.yaml`) validates the devcontainer on every pull request, push to `main`, and merge group event.
+
+Builds run on two architectures in parallel:
+
+- **AMD64**
+- **ARM64**
+
+The S-CORE devcontainer is build using this repo's `.devcontainer` to ensure consistent behavior at developer and CI.
+
+## Merge Job (main only)
+
+Each architecture is built and tested individually.
+After both architecture builds succeed, a merge job creates a multi-arch manifest and pushes it to `ghcr.io`.
+It would have been ideal if the multi-arch image could have been build in one go, but that did not work.
+
+## Consumer Tests
+
+At consumer tests, repos using the devcontainer are build and tested using the newly created devcontainer image.
+This is done to detect breaking changes before releasing a new devcontainer image and to reduce manual testing efforts.
+
+By default these are not run in pull requests, but in the merge queue and pushes to main to save time.
+If these shall be run in the pull request add the **`test-consumer`** label to a pull request.
+
+> [!NOTE]
+> Github may not trigger a new workflow run after the label is set.
+> You need to push another commit for that.
+
+## Release Automation
+
+Releases are cut automatically once per week from `main`, but only if commits were added since the latest `v<major>.<minor>.<patch>` tag.
+The scheduled workflow creates the git tag and GitHub release, and the existing tag-triggered release workflow then builds, tests and publishes the matching container image.
+
+The next semantic version is derived from commit messages since the previous release:
+
+* breaking changes (`!` in the conventional commit header or `BREAKING CHANGE:` in the body) increment the major version
+* `feat` commits increment the minor version
+* every other commit increments the patch version so maintenance-only weeks still publish a new immutable image

scripts/test_consumer.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+
+# *******************************************************************************
+# Copyright (c) 2026 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# SPDX-License-Identifier: Apache-2.0
+# *******************************************************************************
+
+set -euxo pipefail
+
+# Usage: test_consumer.sh <repo-url> [revision] [devcontainer-image]
+# Tests that a consumer repository can be built and tested using the devcontainer.
+# It is checked that these commands work without errors:
+#   - bazel build //...
+#   - bazel test //...
+# Parameters:
+#   repo-url             : Git URL of the consumer repository
+#   revision             : Git branch/tag/commit (default: main)
+
+REPO_URL="${1:?Repository URL is required}"
+REVISION="${2:-main}"
+
+IMAGE="s-core-devcontainer"
+
+export DOCKER_BUILDKIT=1
+
+SCRIPT_PATH=$(readlink -f "$0")
+SCRIPT_DIR=$(dirname -- "${SCRIPT_PATH}")
+PROJECT_DIR=$(dirname -- "${SCRIPT_DIR}")
+ID_LABEL="test-container=${IMAGE}"
+
+. "${SCRIPT_DIR}/functions.sh"
+set_dockerfile_name
+
+devcontainer up \
+  --id-label "${ID_LABEL}" \
+  --workspace-folder "${PROJECT_DIR}/src/${IMAGE}/" \
+  --remove-existing-container
+
+# Extract repo name from URL
+REPO_NAME=$(basename "${REPO_URL}" .git)
+REPO_WORKSPACE="/tmp/${REPO_NAME}"
+
+echo "(*) Cloning repository..."
+# --revision not supported by older git versions, so we clone first and then checkout the revision
+# devcontainer exec --id-label "${ID_LABEL}" git clone --depth 1 --revision "${REVISION}" "${REPO_URL}" "${REPO_WORKSPACE}"
+devcontainer exec --id-label "${ID_LABEL}" git clone "${REPO_URL}" "${REPO_WORKSPACE}"
+devcontainer exec --id-label "${ID_LABEL}" git -C "${REPO_WORKSPACE}" checkout "${REVISION}"
+
+# Run build and test with Bazel using docker exec
+echo "(*) Running Bazel build in devcontainer..."
+devcontainer exec --id-label="${ID_LABEL}" /bin/sh -c "set -e && cd \"${REPO_WORKSPACE}\" && bazel build //..."
+
+echo "(*) Running Bazel test in devcontainer..."
+devcontainer exec --id-label="${ID_LABEL}" /bin/sh -c "set -e && cd \"${REPO_WORKSPACE}\" && bazel test //..."
+
+echo "(*) Bazel build and test completed successfully for ${REPO_NAME}"