Skip to content

feat: publish images to ghcr.io/midnightntwrk with signing and SBOM#645

Merged
gilescope merged 19 commits into
mainfrom
SRE-1769/publish-to-midnightntwrk
Feb 12, 2026
Merged

feat: publish images to ghcr.io/midnightntwrk with signing and SBOM#645
gilescope merged 19 commits into
mainfrom
SRE-1769/publish-to-midnightntwrk

Conversation

@KrisF-Midnight

@KrisF-Midnight KrisF-Midnight commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Overview

Publish midnight-node and midnight-node-toolkit images to ghcr.io/midnightntwrk (the org-matching GHCR namespace) alongside the existing ghcr.io/midnight-ntwrk and Docker Hub registries. All images are cosign-signed and SBOM-attested.

What this does

Build (Earthfile):

  • Push arch-specific images to both ghcr.io/midnight-ntwrk and ghcr.io/midnightntwrk simultaneously during Earthly builds (no cross-registry copy needed)

Production workflows (main.yml, release-image.yml):

  • Create multi-arch manifests in midnightntwrk from its own arch-specific tags via docker buildx imagetools create
  • Create arch-specific release tags (needed for per-platform SBOM attestation)
  • Cosign keyless signing of multi-arch manifests
  • SBOM generation + vulnerability scan + attestation via reusable sbom-scan-image.yml
  • Release workflow gates finalize-release on midnightntwrk SBOM results
  • Docker Hub now pulls from ghcr.io/midnightntwrk (instead of ghcr.io/midnight-ntwrk), making midnightntwrk the canonical upstream

Signing infrastructure (sign-image.sh):

  • Sign manifest list by digest (not tag) to handle OCI index indirection
  • Add propagation delay between signing and verification

SBOM infrastructure (sbom-scan-image.yml, sbom-scan.sh):

  • Add --platform parameter support for cross-platform SBOM generation on manifest-list images
  • Attest per-platform SBOMs against the multi-arch manifest list digest (Chainguard-style), enabling cosign verify-attestation on multi-arch tags without --platform (which cosign doesn't support — verify-attestation should support --platform argument sigstore/cosign#3552)
  • Upgrade cosign-installer to v4.0.0 (cosign v3.0.2) so attestations are stored as OCI 1.1 referrers alongside signatures
  • New multi-arch-image optional input on sbom-scan-image.yml reusable workflow

Verification (verify-image.sh, docs):

  • Add midnightntwrk GHCR prefixes to signed image allowlist
  • Add operator-facing verification guide

🗹 TODO before merging

  • Ready
  • Delete .github/workflows/test-ghcr-midnightntwrk-push.yml test workflow

📌 Submission Checklist

  • Changes are backward-compatible (or flagged if breaking)
  • Pull request description explains why the change is needed
  • Self-reviewed the diff
  • I have included a change file, or skipped for this reason: CI/infrastructure only
  • If the changes introduce a new feature, I have bumped the node minor version
  • Update documentation (if relevant)
  • Updated AGENTS.md if build commands, architecture, or workflows changed
  • No new todos introduced

🧪 Testing Evidence

Test workflow (test-ghcr-midnightntwrk-push.yml) validated end-to-end on this branch:

  • Arch-specific image push to both GHCR registries via Earthly

  • Multi-arch manifest creation in midnightntwrk from its own arch tags

  • Cosign keyless signing with GitHub Actions OIDC

  • Signature verification with certificate identity + OIDC issuer

  • SBOM generation (Syft), vulnerability scanning (Grype), attestation (Cosign)

  • Multi-arch SBOM attestation: per-platform SBOMs attested to manifest list digest

  • Multi-arch SBOM verification: cosign verify-attestation --type spdxjson on multi-arch tags passes

  • All Rekor transparency log entries confirmed

  • Additional tests are provided (if possible)

🔱 Fork Strategy

  • Node Runtime Update
  • Node Client Update
  • Other:
  • N/A

Links

- JIRA: SRE-1769

Krisztian Feher added 9 commits February 10, 2026 14:27
Use docker buildx imagetools create for server-side manifest copy
instead of pull/tag/push. Add cosign signing and signature verification
steps for both midnight-node and midnight-node-toolkit.
cosign verify IMAGE:TAG resolves the tag to the manifest list index
digest and looks for a signature there. Previously we only signed
per-platform digests, causing verification by tag to fail with
"no signatures found".

Now sign_with_retry also signs the manifest list index itself for
multi-arch manifests, so both tag-based and digest-based verification
work.
- Sign manifest list by digest instead of tag (cosign v3 deprecates
  tag references and this avoids potential race conditions)
- Add 10s delay between signing and verification to allow signature
  propagation in the registry
- Improve verify step output with per-image labels
Add arch-specific tag creation and SBOM scan jobs to validate SBOM
generation, vulnerability scanning, and attestation against the
midnightntwrk registry before integrating into production workflows.
imagetools-created arch tags are OCI indexes, not single-platform
manifests, so syft on amd64 runners can't resolve arm64-only indexes.
…ion workflows

Add image copy, cosign signing, and SBOM scanning/attestation for
ghcr.io/midnightntwrk in both main and release workflows. Arch-specific
tags are created alongside multi-arch manifests to support per-platform
SBOM attestation.

- main.yml: copy+sign commit-tagged and latest-main images, 4 SBOM jobs
- release-image.yml: copy+sign release images, 2 SBOM jobs gating release
- verify-image.sh: add midnightntwrk prefixes
- docs: update signed images table, add operator verification guide
@github-actions

github-actions Bot commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

kics-logo

KICS version: v2.1.16

Category Results
CRITICAL CRITICAL 0
HIGH HIGH 0
MEDIUM MEDIUM 96
LOW LOW 12
INFO INFO 83
TRACE TRACE 0
TOTAL TOTAL 191
Metric Values
Files scanned placeholder 31
Files parsed placeholder 31
Files failed to scan placeholder 0
Total executed queries placeholder 73
Queries failed to execute placeholder 0
Execution time placeholder 9

@KrisF-Midnight KrisF-Midnight force-pushed the SRE-1769/publish-to-midnightntwrk branch from bb109ea to 4018db8 Compare February 10, 2026 14:27
Comment thread .github/workflows/test-ghcr-midnightntwrk-push.yml Fixed
Comment thread .github/workflows/test-ghcr-midnightntwrk-push.yml Fixed
Comment thread .github/workflows/test-ghcr-midnightntwrk-push.yml Fixed
Krisztian Feher and others added 9 commits February 10, 2026 15:24
Adds attest_sbom_to_multiarch() which attests each per-platform SBOM
against the manifest list digest, enabling cosign verify-attestation
on multi-arch tags. Test workflow includes a verification job.
…tations

cosign-installer v3.7.0 installed cosign v2.4.1 which stores attestations
via tag-based discovery (.att tag). cosign v3+ stores them as OCI referrers
(--new-bundle-format=true default). Since signatures are already stored as
OCI referrers, cosign verify-attestation only checked referrers and never
fell back to tag-based, causing multi-arch attestation verification to fail.
cosign verify-attestation dumps full SBOM JSON payloads to stdout,
which can be megabytes and cause the step to hang printing output.
…lows

Pass multi-arch-image to midnightntwrk SBOM jobs in main.yml and
release-image.yml so per-platform SBOMs are attested against the
manifest list digest, enabling cosign verify-attestation on multi-arch tags.
Earthly now pushes arch-specific commit-tagged images directly to
ghcr.io/midnightntwrk in addition to ghcr.io/midnight-ntwrk. Workflow
steps that create multi-arch manifests, latest-main tags, release tags,
and Docker Hub images for midnightntwrk now source from midnightntwrk
instead of midnight-ntwrk, removing the cross-registry dependency.

All midnight-ntwrk operations remain completely unchanged.
…ch SBOM attestation

Replace string-based command building with bash arrays to avoid eval,
and add a verification step after multi-arch SBOM attestation.
@gilescope gilescope enabled auto-merge February 11, 2026 14:21
@gilescope gilescope requested a review from chrisferry February 11, 2026 14:24
@gilescope gilescope disabled auto-merge February 11, 2026 14:24

@chrisferry chrisferry left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 lgtm

@gilescope gilescope added this pull request to the merge queue Feb 12, 2026
Merged via the queue into main with commit e3e74b3 Feb 12, 2026
39 checks passed
@gilescope gilescope deleted the SRE-1769/publish-to-midnightntwrk branch February 12, 2026 19:34
m2ux added a commit that referenced this pull request Apr 23, 2026
Signed-off-by: Mike Clay <mike.clay@shielded.io>
m2ux added a commit that referenced this pull request Apr 23, 2026
Signed-off-by: Mike Clay <mike.clay@shielded.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants