> ## Documentation Index > Fetch the complete documentation index at: https://docs.veristar.app/llms.txt > Use this file to discover all available pages before exploring further. # Supply chain > How images and the tracker are built, gated, signed, verified and kept up to date. How the platform's images and the tracker are built, gated, signed, verified and kept up to date (plan phase 18). Status on 2026-11-13 is stated at the end of every section, because two parts of the design cannot run yet. ## 1. What is protected | Artifact | Built by | Gated by | Signed or attested | Verified by | | - | - | - | - | - | | Platform images (gateway, collector, caddy, console, controlplane, replayer, metricswriter, jobs, platformctl, prober, postgres-ha) | `.github/workflows/release-images.yml` on a tag `v*` | syft SBOM, Trivy (fixable critical or high fails) | cosign keyless signature, build provenance and SBOM attestations (`actions/attest`) | `infra/ansible/playbooks/verify-release.yml`, imported first by `site.yml`, `deploy.yml`, `rotate-all.yml` and `restore.yml`, before any node pulls | | `sdk.js` (tracker) | `packages/tracker/build.mjs`, committed under `releases//` | immutability, byte-identical rebuild, size (`verify-releases`) | build provenance attestation | tenants: SRI hash in the snippet, `gh attestation verify` ([verify-tracker.md](https://docs.veristar.app/security/verify-tracker.md)) | | npm package of the tracker | `packages/tracker/scripts/pack.mjs` in `release-tracker.yml` on a tag `tracker-v*` | packs to the same bytes twice, installs and type-checks in a consumer | npm provenance (trusted publishing) and a build provenance attestation | `npm audit signatures`, the npm package page | | Upstream images (Umami, NATS, etcd, HAProxy, the observability images) | upstream | pinned by tag and index digest in `infra/ansible/vars/pins.yml` and the observability role defaults; the deploy refuses an upstream reference without a digest | upstream | digest pin | Tool versions (plan section 5): cosign 3.1.4, syft 1.57.1, Trivy 0.76.2, gitleaks 8.30.1, govulncheck 1.9.0. Workflows install them with `scripts/supply-chain/install-tools.sh`, which checks each download against the release's own checksum file. Every GitHub Action is pinned by commit SHA (resolved with `git ls-remote`, tag in a comment). ## 2. Image release (`release-images.yml`) For each image, in `.github/actions/release-image/action.yml`: 1. Build with BuildKit and push to `ghcr.io///:`. The postgres-ha tag is `-`, because the core role compares the version. 2. `scripts/supply-chain/scan-image.sh @`: an SPDX SBOM with syft, the full Trivy report as JSON, then the gate: Trivy with `--severity CRITICAL,HIGH --ignore-unfixed --exit-code 1`. Unfixed findings are recorded but do not fail the release, because only an upstream fix or a mitigation can address them (see the runbook). 3. Only an image that passed the gate is signed: `cosign sign --yes @`. Keyless: a short-lived Fulcio certificate issued to this workflow's GitHub OIDC identity, recorded in the Rekor transparency log. No signing key exists anywhere. 4. `actions/attest` v4.2.2 writes a build provenance attestation and an SBOM attestation for the digest and pushes both to the registry. An image that failed the gate stays in the registry unsigned. The deploy refuses unsigned digests, so such an image cannot be deployed. The last job writes `release-images.json` (schema 1: release tag, and per image the repository, tag and digest; `scripts/supply-chain/release-manifest.py` refuses a set of names other than the expected eleven), verifies every digest again with the deploy's identity, attests the file, and attaches it to the GitHub release. ## 3. Deploy-time verification (every playbook that pulls images) `site.yml` (first converge and node replacement), `deploy.yml` (rolling deploy), `rotate-all.yml` and `restore.yml` all start with `playbooks/verify-release.yml`, whose first play runs `infra/ansible/playbooks/tasks/verify-images.yml` on the controller (finding O-8; before 2026-11-13 only `deploy.yml` did, so the documented first converge pulled by tag). The local converge is the one exception, refused for any other environment: its images come unsigned from its throwaway registry, and the play says so instead of verifying. ``` ansible-playbook -i inventories// playbooks/deploy.yml \ -e sa_image_tag= -e sa_release_manifest= ``` The run stops before any node is touched when: * no manifest is given, or it names another release, another registry, or not every image in `platform_images`; * an upstream image in `vars/pins.yml` is not pinned by digest; * `cosign verify` fails for any digest with identity `https://github.com/justmert/veristar/.github/workflows/release-images.yml@refs/tags/` and issuer `https://token.actions.githubusercontent.com` (`vars/signing.yml`). This covers an unsigned digest and a signature from any other workflow, branch, tag or repository; * the release tag in the registry now points to a digest other than the manifest's. The next play sets `platform_images` on every node to the verified `repo:tag@digest` references. Host facts outrank play variables, so the roles (including the imported bootstrap play) pull exactly the verified digests. The roles themselves are unchanged. **Local proof (test-only key).** Keyless signing needs GitHub's OIDC token, which exists only inside GitHub Actions. `tests/supplychain` (`make supplychain-it`) proves the refusal logic with a real registry and real cosign signatures made with an **ephemeral key pair created for one test run** (`sa_signature_mode: test-ephemeral-key`). The task file refuses that mode for any inventory outside `tests/`, and a test case proves the refusal. This is a test of the verification logic, not the production signing design. Result on 2026-11-13: 10 of 10 cases pass (signed release passes and pins digests; unsigned digest, foreign key, other build under the release tag, moved tag, other release, other registry, incomplete manifest, missing manifest, test mode outside tests: all refused). **Status.** Built and proven locally with the test-only key. Keyless signing and `cosign verify` of a real release have not run: GitHub Actions on the account ends every workflow in `startup_failure`. Artifact attestations for a private repository also need GitHub Enterprise Cloud, so attestations need the repository to be public (owner's explicit confirmation, plan phase 18). The probe node's own images (the `prober` and the custom Caddy, from the observability role) are verified the same way: the manifest must name the `prober` (it is in `platform_images`), the pin play sets `platform_images` on the probe too, and the observability role names its `prober` and `caddy` through `platform_images`. The role also refuses a probe node whose `prober` or `caddy` is not pinned by digest outside the local converge, so a run that skipped the check stops. The probe converge has not completed on the local converge yet (`make ops-local`). The probe bundle of `docs/deploy/probe.md` uses the same check: `playbooks/probe-bundle.yml` imports `verify-release.yml` and refuses a prober not pinned by digest. ## 4. Tracker release (`release-tracker.yml`) A tag `tracker-v` must equal `packages/tracker/package.json` and the newest entry of `releases/manifest.json`. The `prove` job runs `verify-releases` (committed files unchanged, consistent manifest, byte-identical rebuild of `sdk.js` on the runner, size limit) and the tracker tests, including `test/pack.test.ts`: the npm package packs to the same tarball twice, installs into an empty consumer, works with no `__SDK_VERSION__` define (the published ESM carries the version), type-checks, and carries `sdk.js` byte for byte. The `publish` job attests `sdk.js` and the tarball, then runs `npm publish` with trusted publishing: npm exchanges the job's OIDC token for a short-lived publish credential and attaches provenance itself. It then creates the GitHub release with `sdk.js`, the manifest and the tarball. **Status.** Built; the package build and its tests pass locally (5 of 5 in `pack.test.ts`, run with a test-only scope `@sa-local-test`). Blocked: input I9 (the npm scope with this repository and workflow registered as trusted publisher; set as repository variable `NPM_PACKAGE`), GitHub Actions on the account, and a public repository (npm provenance and trusted publishing require it). The control plane still refuses `TRACKER_NPM_PACKAGE` until the package is published. ## 5. Detection (`security.yml`, hourly) * `scripts/supply-chain/advisories.py`: every published repository advisory of `umami-software/umami` not listed in `docs/security/advisories-reviewed.json`, any newer Umami release, GitHub advisory database matches for Next.js, React and React DOM at the exact versions in the pinned Umami's `pnpm-lock.yaml`, for nats-server, Caddy, etcd and the collector modules at our pins, and a newer PostgreSQL minor of the pinned major. It never reads "fixed in" wording, because Umami has published advisories without a fixed version. A source that cannot be read fails the job; it is never treated as clean. * Trivy on the digests a deploy uses: the platform images of the newest `release-images.json` and every upstream digest pin. * `govulncheck` on both Go modules, `pnpm audit` on the workspace. * Every finding opens one issue labelled `security-finding` (an open issue for the same id is not repeated) and the run posts one alert to `ALERT_WEBHOOK_URL` (input I8). Without I8 the run ends red after filing the issues, so GitHub's own failure notice still reaches the maintainers. Dependabot (`.github/dependabot.yml`) opens update pull requests for Go, npm, Dockerfiles, compose files, GitHub Actions, OpenTofu providers, the Python and Rust examples and the Ansible controller requirements. Umami is excluded: its digest moves only through `make umami-upgrade`. The digests in `infra/ansible/vars/pins.yml` are not a format Dependabot reads; the hourly Trivy run covers them. ## 6. Pull request gates (`ci.yml`) `gitleaks` (whole history, reviewed fingerprints in `.gitleaksignore`), `licences` (`scripts/supply-chain/licences.py`, see section 7), `isolation` (`make isolation`) and `supplychain` (`make supplychain-it`), next to the existing build, lint, test and tracker checks. [branch-protection.md](https://docs.veristar.app/security/branch-protection.md) lists the required checks. ## 7. Licences **Policy.** Everything this project ships, and every production dependency of it, carries a licence on the OSI approved list. There are no exceptions for shipped code. Dependencies that are never shipped (development tools, test fixtures, example apps and operator tooling) are listed in a separate section of the report. Any of them under a licence that is not OSI approved must be on an explicit allowlist, reviewed for one exact version and licence, with a written reason. Nothing is skipped silently: a package that cannot be placed in either section, or whose licence cannot be found, fails the gate. `scripts/supply-chain/licence-gate.sh` (the `licences` job in `ci.yml`, and local runs) feeds `scripts/supply-chain/licences.py`, which reads `docs/security/licence-policy.json`: | Section | What it contains | How it is derived | Fails on | | - | - | - | - | | shipped | Go modules linked into a platform binary; npm packages of the console bundle, `sdk.js` and the npm package of the tracker | `go list -deps ./cmd/...` in the root module and `go list -deps .` in the collector distribution that ocb generates (`dist/scf-otelcol`, from `collector/builder-config.yaml`); the `dependencies` and `optionalDependencies` closure of the shipped importers `apps/console` and `packages/tracker` in `pnpm-lock.yaml` | any licence not on the OSI list | | not shipped | everything else in `go.mod`, the collector module, `pnpm-lock.yaml`, the example lockfiles (`package-lock.json`, `uv.lock`, `Cargo.lock`, `go.mod`) and `infra/ansible/requirements.txt`, each with the importers that pull it in | the rest of the same graphs | a non-OSI licence without a matching allowlist entry, and any allowlist entry that no longer matches (package gone, upgraded, relicensed or now shipped) | Three checks keep the shipped section honest instead of trusting `package.json`: * The console bundle is built with its own Vite configuration and every npm package whose code lands in it is listed (`apps/console/scripts/bundle-packages.mjs`). Each must be in the shipped section, so a package imported at runtime but declared as a devDependency fails the gate. CSS is compiled by the Tailwind plugin and is not part of that module graph, so the policy names what `src/index.css` imports: `tailwindcss` (a devDependency) counts as shipped, and of `shadcn` only the file `shadcn/tailwind.css` is used, so `shadcn` is shipped but the dependencies of its CLI are not. * The tracker builds (`sdk.js` in `build.mjs`, the npm package in `scripts/pack.mjs`) fail when esbuild reports any input outside `src/` and `releases/`, and the packed `package.json` declares no dependencies. The tracker therefore ships no third-party code, including no Wallets Kit code. * Go: only modules that are linked into a shipped binary count as shipped. Test-only modules such as testcontainers are listed as not shipped. Packages whose licence trivy did not find on disk are looked up in the npm, PyPI or crates.io registry. Our own modules (the root module, the collector distribution and the example roots) are reported separately as own code under the repository's Apache-2.0 licence. **Not covered by this gate.** The Caddy binary (built by xcaddy inside `deploy/images/caddy.Dockerfile` with two plugins) and the operating system packages of the base images (distroless, Alpine in the Caddy image, Debian in `postgres-ha`) are not part of these graphs. The image SBOMs from `scan-image.sh` list them; checking their licences is a separate step that does not exist yet. **Why the Wallets Kit packages are allowlisted.** `@creit.tech/stellar-wallets-kit` 2.7.1 and 2.8.0 are installed only as devDependencies of `packages/tracker` (`swk-2.7.1`, `swk-2.8.0`, so the phase 14 adapter tests run the real kit) and as a dependency of the example app `examples/wallets-kit-spa`, a sample for tenants that is also a test target and is never deployed. The lockfile graph shows no other path to them. Checked on 2026-11-13: esbuild's input list for `sdk.js` and for the npm entry points contains only `src/` and `releases/` files; the packed npm package (`@sa-local-test/tracker` 1.1.0) has no dependency fields, no bare imports and no kit code; the console bundle contains none of the kit's packages and no `walletconnect`, `reown`, `creit.tech`, `xbull` or `trezor` string; the SBOMs of all 12 platform images list none of them. The kit pulls in 26 lockfile entries the gate reports as non-OSI (25 distinct versions): 19 Reown and WalletConnect versions and `@trezor/crypto-utils` under "SEE LICENSE IN LICENSE.md" (not an OSI licence), and five with no licence in their registry metadata (`@creit.tech/xbull-wallet-connect`, `@hot-wallet/sdk`, `@near-wallet-selector/core`, `eyes`, `text-encoding-utf-8`). The project does not bundle or redistribute any of them, so they are allowlisted as test and example dependencies. Tenants who add the kit to their own sites take it from npm under its own terms. The other allowlisted packages are data, not code: `caniuse-lite` (CC-BY-4.0, browser support data for build tools; it reaches the console importer only through the shadcn CLI and is not bundled), `mdn-data` (CC0-1.0, CSS reference data for test and example tooling) and `webpki-root-certs` (CDLA-Permissive-2.0, root certificates in the Rust example app). Before 2026-11-13 these three were exceptions that applied wherever the package appeared; they are now accepted only while not shipped. **Changing the allowlist.** Add or update an entry in `not_shipped_allowlist` with the exact version and the licence string the report printed, a reason that says why the package is not shipped and what it is, and the review date. The reviewer checks the report's "pulled in by" line. A version bump of an allowlisted package fails the gate until the entry is reviewed again. ## 8. Results recorded on 2026-11-13 * Trivy and syft on all 12 platform images built from this tree and the 11 upstream images. All 12 platform images pass the gate after three fixes: `github.com/klauspost/compress` 1.20.1 to 1.20.4 (GO-2026-6865, called from the replayer), Caddy built with `golang.org/x/net` v0.62.0 (CVE-2026-78661 and others), and `postgres-ha` without the unused `gosu` (Go 1.24.6 stdlib, one critical and ten high) and with the Debian `zlib1g` security update. * Upstream images with fixable high findings that only their maintainers can fix: Umami 3.5.0 (`undici` 6.28.0), HAProxy 3.4.7 (`zlib1g`), etcd 3.6.16, Prometheus 3.15.1, Alertmanager 0.34.2, Grafana 13.2.6, Gatus 5.37.0, node\_exporter 1.12.1, postgres\_exporter 0.20.1, prometheus-nats-exporter 0.20.3. Section 8.1 records, per finding, the pin bump or the mitigation and its reasoning. * govulncheck: no called vulnerability in either module after the fixes (the example app moved to `golang.org/x/net` v0.62.0). One uncalled module finding remains (`golang.org/x/crypto/openpgp`, GO-2026-5932, fixed only in a pseudo-version; the package is not imported). * A throwaway build of `postgres-ha` with `gosu` left in fails the gate: 1 critical, 25 high, exit 1. ### 8.1 Upstream image findings: bump or mitigation (2026-11-13) Every fixable critical or high finding of the upstream images, rechecked on 2026-11-13 with Trivy 0.76.2 (`--scanners vuln`, registry images by digest, one at a time). For each image the registry was checked for a release that fixes it (plan section 5 pin policy: a younger minor line is taken when it carries a security fix). Where no release fixes a Go finding, the image's binary was checked with `govulncheck -mode=binary` (v1.9.0, database of 2026-11-11), which reports whether the vulnerable symbols are compiled into the binary at all. No critical finding exists in any upstream image. Nothing here was run against production, which does not exist yet. **Bumped (each rescanned after the change):** | Image | Was | Now | Findings before, after | Checks | | - | - | - | - | - | | HAProxy | `haproxy:3.4.7` (Debian trixie) `sha256:f6d2ee87...` | `haproxy:3.4.7-alpine` `sha256:b9fe316f718d5015c5cfebd8ca019591f3e44656f61d93cb8b59a1028961e530` | 2 high (`zlib1g` CVE-2026-76844, CVE-2026-85091; no rebuilt trixie image exists), 0 | Same HAProxy release, Alpine 3.24 base. Both builds run as uid 99 and are built with `PROMEX`; `haproxy -c` passes on a rendered core `haproxy.cfg` and on the edge and core relay configs with both builds | | Gatus | `v5.37.0` `sha256:094eb186...` | `v5.38.0` `sha256:085ed63da963728965d00a3c78aa6167e3bf135218485b893201758e079e8415` | 12 high, 5 high (the Go standard library and gRPC findings are gone) | The 5.38 line is 16 days old, taken for the fixes. The v5.37.0...v5.38.0 diff leaves `config/key/key.go`, `config/endpoint/result.go` and the statuses API that `internal/prober` and `cmd/sloreport` depend on unchanged. A rendered probe `gatus.yml` loads the same 5 endpoints and 20 external endpoints in both versions | Pins moved together in `infra/ansible/vars/pins.yml`, the observability role defaults, `deploy/images/upstream/` (`tests/pins` checks they agree) and `tests/observability/compose.yml`. **Mitigated (no fixed release exists on 2026-11-13):** | Image | Finding | Why it is accepted, and when to revisit | | - | - | - | | Umami 3.5.0 (newest release; 3.5.0 is the newest tag on GHCR) | `undici` 6.28.0: CVE-2026-109383 (TLS options dropped), CVE-2026-109869, CVE-2026-19534, CVE-2026-87018 (client-side denial of service) | All four are in undici's HTTP client. Umami makes no outbound request in this deployment: `PRIVATE_MODE`, `DISABLE_TELEMETRY` and `DISABLE_UPDATES` are set, and `TestWebPath` checks no egress from boot and during a UI session (`threat-model.md`). Inbound requests are served by Next.js, not undici. Revisit at the next Umami release through `make umami-upgrade`. | | etcd 3.6.16 (newest 3.6 release) | `golang.org/x/net` CVE-2026-78661 (HTML parser denial of service, GO-2026-6860); the symbol is in the binary | etcd's client and peer ports are published on the private address only, the firewall accepts the private network only, and the only client is Patroni. No etcd API takes HTML. The 3.7 line is not used until Patroni support is confirmed (plan section 5). Revisit at 3.6.17. | | Prometheus 3.15.1, Alertmanager 0.34.2 | `golang.org/x/net` CVE-2026-78661 | govulncheck: module present, vulnerable symbols not in either binary. The younger lines (Prometheus 3.16.0, Alertmanager 0.35.0, scanned) carry the same finding, so a bump would not fix it. | | node\_exporter 1.12.1 (newest) | 16 high: Go 1.26.5 standard library (CVE-2026-111943, -33818, -39821, -46600, -56853, -56858, -56859, -56860, -56862, -78667, -78669, -97031), `x/net` CVE-2026-78661 and -78669, `x/crypto` CVE-2026-56854, `x/text` CVE-2026-56851 | govulncheck: the `x/net/html`, `x/crypto`, `x/text`, CVE-2026-46600 and CVE-2026-78667 symbols are not in the binary. The rest are present and are denial of service in the HTTP server, TLS and parsers, or an x509 client issue. The exporter listens on the private address only (`--web.listen-address`), the firewall drops everything else, it has no TLS configuration and makes no outbound TLS connection, and only Prometheus scrapes it. An attacker must already be on the private network. Revisit at the next node\_exporter release. | | postgres\_exporter 0.20.1 (newest) | 19 high: Go 1.26.4 standard library (CVE-2026-111943, -33818, -39821, -39822, -46600, -56853, -56858, -56859, -56860, -56862, -78667, -78669, -97031), `x/net` CVE-2026-46600, -78661 and -78669, `x/crypto` CVE-2026-56854, `x/text` CVE-2026-56851 and -56852 | govulncheck: the `x/net/html`, `x/crypto`, `x/text`, CVE-2026-39821, -39822, -56859 and -78667 symbols are not in the binary. The rest are present. The exporter is on the private address of the PostgreSQL hosts only, scraped by Prometheus, and connects only to PostgreSQL through HAProxy on the private network as a role that holds `pg_monitor` and nothing else (`tasks/core.yml`). Revisit at the next release. | | Grafana 13.2.6 (newest 13.2) | 19 high, all in bundled plugin binaries under `data/plugins-bundled` (MySQL, Tempo, InfluxDB, Loki, Stackdriver, PostgreSQL, Pyroscope, Jaeger, MSSQL, OpenTSDB, Prometheus, Zipkin): `x/net`, Go standard library, and Tempo CVE-2026-28377 and CVE-2026-21728 | The only provisioned data source is Prometheus (`files/grafana/provisioning/datasources/prometheus.yml`); the other plugins handle a query only for a data source of their type, and none exists. The Prometheus plugin's finding is the HTML parser one, reachable only through responses of the platform's own Prometheus. Grafana is published on `127.0.0.1:3000` only (operators reach it through an SSH tunnel) with sign-up and anonymous access off. The `-slim` build has no findings but also no bundled Prometheus data source, so it cannot replace it. Revisit at 13.2.7 or when the 13.3 line is 30 days old. | | Gatus 5.38.0 | `go-jose` CVE-2026-116587 (JWK parsing), `klauspost/compress` CVE-2026-63209, `x/image` CVE-2026-46603, `x/net` CVE-2026-78661 and CVE-2026-78669 (HTTP/2 SETTINGS) | govulncheck: the `x/image` and `x/net/html` symbols are not in the binary. `go-jose` is used only by Gatus's OIDC login, which `gatus.yml` does not configure. Gatus's port is not published: only the probe node's Caddy reaches it on the compose network, and Caddy speaks HTTP/1.1 to it, so no outside client can send it HTTP/2 frames; its own checks call only the platform's hosts. Revisit at 5.39. | | prometheus-nats-exporter 0.20.3 (newest) | Alpine `musl` CVE-2026-107891 (`regcomp`), `zlib` CVE-2026-85091 | The exporter is a statically linked Go binary (go1.27.3), so it loads neither library. Only the image's `/entrypoint.sh` runs under BusyBox `sh` (linked to musl) to `exec` the exporter, with no regular expression and no input from outside. Revisit at the next release. | The hourly `security.yml` workflow keeps filing these findings until a fixed release exists. A fixable high finding that is not in this table has no recorded decision and is handled under the 24-hour rule (`docs/runbooks/security-patch.md`). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.