> ## Documentation Index > Fetch the complete documentation index at: https://docs.veristar.app/llms.txt > Use this file to discover all available pages before exploring further. # Secrets: SOPS with age > How secrets are encrypted with SOPS and age, and which files hold them. Status: first version (phase 15, stream b). Pins: SOPS 3.13.3, age 1.3.2 (plan section 5). ## 1. Layout ``` infra/secrets/ .sops.yaml creation rules, generated by sync-recipients.sh recipients/.txt age public keys per environment (input I14), one per line sync-recipients.sh regenerates .sops.yaml and re-encrypts to the lists new-secrets.py generates one environment's secrets file /secrets.enc.yaml the platform's secrets (Ansible), SOPS-encrypted yaml /ca.enc.yaml the internal CA of the private network (controller only) /tls.enc.yaml the CA bundle and one key and certificate per node (ansible.md section 9) /tofu.enc.env OpenTofu's credentials (opentofu.md section 1) ``` Production has a recipient list since 2026-11-13 (`recipients/production.txt`, the owner's identity) and therefore a creation rule; its `secrets.enc.yaml` and `tofu.enc.env` exist. Staging has no list, so `sops encrypt` for staging refuses with `no matching creation rules found`. That refusal is intended; nothing can be encrypted to an unreviewed key. Section 4 asks for at least two recipients per environment; production has one until a second person's key is added. ## 2. Who decrypts Only people. Each recipient is a person's age public key. The nodes never hold an age identity: the Ansible `secrets` role decrypts on the controller with the operator's identity (`SOPS_AGE_KEY_FILE`, else `~/.config/sops/age/keys.txt`) through `community.sops.load_vars`, and writes per-service env files on each node: * `/etc/veristar/env/` is `root:root 0700`, every file in it `root:root 0600`. * Each service gets only its own keys (`roles/secrets/templates/.env.j2`), read by Docker Compose's `env_file`. * Config files that must carry a credential (NATS, Patroni) are written by the edge and core roles with mode 0600; Patroni's is owned by the container's postgres user. * Every task that handles a value runs with `no_log`. The `secrets` role installs SOPS and age on the controller from their GitHub releases, checked against pinned SHA-256 sums (`infra/ansible/vars/pins.yml`). ## 3. Contents of `/secrets.enc.yaml` Generated (random, by `new-secrets.py`): PostgreSQL superuser and replication passwords, Patroni REST password, Umami database password, `APP_SECRET` and admin password, `UMAMI_USER_PASSWORD_KEY`, the Umami retention role password, `OAUTH_STATE_KEY`, the five platform role passwords, one NATS password per service user with its bcrypt hash, and the NATS route password with its hash, and for monitoring (phase 17) the Grafana admin password, the `postgres_exporter` password, the probe metrics token and the Gatus external-endpoint token. The hashes are generated once with the passwords, so the NATS server config does not change on every converge. A file made before the monitoring keys existed is refused by the `secrets` role, naming the missing key; add each with `sops edit` and a value from `openssl rand -hex 32`. Owner inputs (taken from the environment of `new-secrets.py`, never generated, left out when unset): `github_client_id` and `github_client_secret` (I2, as `SA_GITHUB_CLIENT_ID`, `SA_GITHUB_CLIENT_SECRET`), `cloudflare_api_token` (I6, as `SA_CLOUDFLARE_API_TOKEN`; the zone-scoped token with `Zone:Zone:Read` and `Zone:DNS:Edit`, [prerequisites.md](https://docs.veristar.app/deploy/prerequisites.md) step 5), `alertmanager_discord_webhook_url` and `gatus_discord_webhook_url` (I8, as `SA_ALERTMANAGER_DISCORD_WEBHOOK_URL`, `SA_GATUS_DISCORD_WEBHOOK_URL`). The `secrets` role refuses to converge a core node without I2 and any node using DNS-01 without I6, naming the missing key. The `observability` role, which `site.yml` runs on every edge and core node, refuses to run without the Alertmanager webhook (I8); the local converge passes `observability_test_webhook_base` instead (`docs/deploy/local-converge.md`). ## 4. Procedures Create an environment's file (after I14 is in place): ```sh theme={null} SA_GITHUB_CLIENT_ID=... SA_GITHUB_CLIENT_SECRET=... SA_CLOUDFLARE_API_TOKEN=... \ python3 infra/secrets/new-secrets.py staging \ | sops --config infra/secrets/.sops.yaml encrypt --filename-override infra/secrets/staging/secrets.enc.yaml \ --input-type yaml --output-type yaml /dev/stdin > infra/secrets/staging/secrets.enc.yaml ``` Add an input later: `sops --config infra/secrets/.sops.yaml edit infra/secrets/staging/secrets.enc.yaml`. **Adding a recipient.** The person generates an identity with `age-keygen` and sends the public key. Add it to `infra/secrets/recipients/.txt` with a comment naming the holder, run `infra/secrets/sync-recipients.sh` (it rewrites `.sops.yaml` and runs `sops updatekeys` on every file of that environment), review the diff and commit both. **Removing a recipient.** Delete the line, run `sync-recipients.sh`, commit. Re-encryption only stops the person reading future versions; they may have kept the old plaintext. Therefore also rotate every value they could read, and rotate I2 and I6 with their owners. What `site.yml` applies by itself after a value changes in the file: the five platform role passwords (`platformctl db roles`), the NATS passwords and hashes (env files and a NATS reload), and every env file. What it does not apply yet, because bootstrap only creates and never alters them: the Umami database password, the Umami admin password (`platformctl umami bootstrap` replaces only the seeded one), the PostgreSQL superuser, replication and Patroni REST passwords, and `UMAMI_USER_PASSWORD_KEY` (changing it changes every derived Umami password). Their rotation steps are in [../runbooks/secret-rotation.md](https://docs.veristar.app/runbooks/secret-rotation.md); removing a recipient is a complete rotation only once those steps are done too. **Losing every identity** makes the files unreadable. Each recipient keeps their identity in a password manager; there must be at least two recipients per environment. ## 5. Setting up an environment, step by step Staging is shown; production is the same with `production` in every path. Needs the controller tools of [prerequisites.md](https://docs.veristar.app/deploy/prerequisites.md) with `sops` and `age-keygen` on `PATH`, and inputs I14 (recipients) plus whatever owner inputs exist (I2, I5, I6, I8). 1. **Each recipient makes an identity** on their own machine and keeps the file private: ```sh theme={null} age-keygen -o ~/.config/sops/age/keys.txt ``` Expected: `Public key: age1...`. Only that public key is sent. 2. **List the recipients** in `infra/secrets/recipients/staging.txt`, one public key per line, each under a comment line naming the holder (at least two people). 3. **Generate the SOPS rules:** ```sh theme={null} SOPS=sops infra/secrets/sync-recipients.sh ``` Expected: `wrote .../infra/secrets/.sops.yaml with 1 rule(s)` (2 once production has a list too). The rule is `path_regex: (^|/)staging/[^/]+\.enc\.(yaml|env)$` with the listed keys. 4. **Create the platform secrets file** with the owner inputs that exist, typed into the environment of this one command (the values are never written in plain text to disk): ```sh theme={null} mkdir -p infra/secrets/staging read -rs SA_GITHUB_CLIENT_SECRET; export SA_GITHUB_CLIENT_SECRET SA_GITHUB_CLIENT_ID= \ ~/.local/share/veristar/ansible-venv/bin/python infra/secrets/new-secrets.py staging \ | sops --config infra/secrets/.sops.yaml encrypt --filename-override infra/secrets/staging/secrets.enc.yaml \ --input-type yaml --output-type yaml /dev/stdin > infra/secrets/staging/secrets.enc.yaml unset SA_GITHUB_CLIENT_SECRET ``` Do the same with `read -rs` for `SA_CLOUDFLARE_API_TOKEN` (I6), `SA_ALERTMANAGER_DISCORD_WEBHOOK_URL` and `SA_GATUS_DISCORD_WEBHOOK_URL` (I8) when you have them. The script names every input it left out on standard error, for example `new-secrets.py: SA_CLOUDFLARE_API_TOKEN unset, cloudflare_api_token left out`. The venv's Python is used because the script needs `bcrypt` and PyYAML. 5. **Check** that the file decrypts and holds every key: ```sh theme={null} sops decrypt infra/secrets/staging/secrets.enc.yaml | grep -o '^[a-z_]*:' ``` Expected: `sa_secrets_environment:` first, then every generated key of section 3 and the owner inputs you supplied. Never redirect the decrypted output to a file. 6. **Create the OpenTofu credentials file** `infra/secrets/staging/tofu.enc.env` with the variables of [opentofu.md](https://docs.veristar.app/deploy/opentofu.md) section 1 ("Secrets handed to OpenTofu"): ```sh theme={null} sops --config infra/secrets/.sops.yaml edit infra/secrets/staging/tofu.enc.env ``` SOPS opens an editor on an empty dotenv file; write one `NAME=value` line per variable (`CLOUDFLARE_API_TOKEN` and `TF_VAR_state_passphrase`; Azure needs no secret here, OpenTofu uses the operator's `az login`). Make the passphrase with `python3 -c 'import secrets; print(secrets.token_urlsafe(32))'`; it must be at least 16 characters and different per environment. 7. **Issue the private network's certificates** once the inventory exists ([opentofu.md](https://docs.veristar.app/deploy/opentofu.md) section 6), before the first converge: ```sh theme={null} make internal-ca ENV=staging ``` Expected: `N certificate(s) issued, ca new, 0 retired ca(s) still trusted`, one certificate per edge, core, backup and restore node plus the operators' one, and two new files, `infra/secrets/staging/ca.enc.yaml` and `tls.enc.yaml` ([ansible.md](https://docs.veristar.app/deploy/ansible.md) section 9). The `secrets` role refuses to converge a node without them. 8. **Commit** `infra/secrets/.sops.yaml`, the recipients list and the encrypted files. Steps 1 to 5 were rehearsed on 2026-11-13 in a scratch directory with a throwaway identity and a copy of `sync-recipients.sh` (never under `infra/secrets/`): the script wrote one rule, `new-secrets.py staging` piped into `sops encrypt` exited 0 and reported the seven owner inputs it left out, and the file decrypted to every generated key listed in section 3 (37 encrypted values). Steps 6 and 7 and every real recipient wait for input I14. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.