Skip to content

fix: add nginx to acme group so HTTP-01 challenges don't 403#4

Merged
josibake merged 5 commits into
mainfrom
fix/nginx-acme-group
May 19, 2026
Merged

fix: add nginx to acme group so HTTP-01 challenges don't 403#4
josibake merged 5 commits into
mainfrom
fix/nginx-acme-group

Conversation

@josibake
Copy link
Copy Markdown
Member

Summary

The order-renew script NixOS generates for webroot-mode ACME creates the leaf directory /var/lib/acme/acme-challenge/.well-known/acme-challenge/ with mode 0750 owned by acme:acme. Without nginx in the acme group, lego writes the challenge token but nginx can't read it back, Let's Encrypt POSTs to the challenge URL, gets 403, and the order fails with urn:ietf:params:acme:error:unauthorized.

NixOS's enableACME = true; nginx shorthand wires this automatically; the helper here uses webroot directly (the prior enableACME path asserted nginx had read access to the issued cert, which it shouldn't and which broke frigate's group-readable cert dir), so we have to add nginx to the acme group ourselves.

Caught live during the albatross.2140.dev → frigate.2140.dev cutover today: the first issuance after the hostname change failed 403 until the leaf dir was manually chgrp nginx'd.

Test plan

  • CI: existing regtest-preset / regtest-edge VM tests still pass (they exercise webroot-mode ACME with the test ACME server)
  • On a host where this preset is deployed: trigger a fresh security.acme.certs.<host> issuance (new hostname or --force-renewal); confirm acme-order-renew-<host>.service lands on Active: inactive (dead) with code=exited, status=0/SUCCESS instead of status=10
  • On the same host, confirm id nginx shows the acme group post-deploy

josibake and others added 5 commits May 19, 2026 18:27
`public-frigate` configured bitcoind + fulcrum + ZMQ + optional mesh
exposure inline. Once a backend-only preset becomes necessary (a host
that runs the same stack for a remote `frigate-edge` consumer, with
no frigate of its own — see following commit), both presets need the
same wiring. Move it to a private `_internal/bitcoin-stack.nix`
helper, mirroring the pattern already used for `frigate-tls-acme`.

The helper takes options under `services._roost.bitcoin-stack` (an
internal namespace, marked `internal = true` throughout), and parent
presets wire them from their own typed options.

No behavior change for existing consumers — `public-frigate`'s
external surface is identical (same options, same defaults, same
runtime config). Verified by `regtest-preset`, which exercises the
full preset and continues to pass against the refactored module.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`bitcoind-backend` is the sibling preset to `public-frigate`: bitcoind
+ fulcrum + ZMQ sequence publisher, exposed on a private interface,
with no frigate process on this box. Pairs with `frigate-edge` on
remote consumer hosts. Useful when the latency-sensitive consumer is
in a different DC from the existing public-frigate host (the
loopback cost is loopback; the remote-backend cost is per-call RTT)
and the consumer host can't afford the ~950 GB storage that
bitcoind+fulcrum needs.

Shares the bitcoind+fulcrum+expose logic with `public-frigate` via
the `_internal/bitcoin-stack.nix` helper from the preceding commit;
this preset is mostly an options surface and a small `mkIf` that
wires the helper.

Also adds:
  - `nixosModules.bitcoind-backend-host` — a batteries-included
    bundle (nix-bitcoin + the preset), so consumers needing only
    `roost` in their flake inputs can deploy a backend host.
  - `test/regtest-backend.nix` — single-VM nixosTest that boots
    `bitcoind-backend`, mines 101 regtest blocks, and verifies the
    backend ports listen on the configured mesh address, the
    configured rpcauth user can authenticate, and a wrong password
    is rejected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruff F541 fired in CI because the curl invocation in regtest-backend
was f-stringed but the only interpolations were Nix `${...}`, which
are resolved at Nix eval time. By the time Python sees the lines,
there are no `{}` placeholders left — drop the `f` prefix, and
collapse `{{ }}` back to `{ }` in the JSON bodies (those were only
escaped to survive f-string syntax).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The order-renew script that NixOS generates for `webroot`-mode ACME
creates the leaf directory `/var/lib/acme/acme-challenge/.well-known/
acme-challenge/` with mode 0750 owned by `acme:acme`. Without nginx
in the acme group, lego writes the challenge token but nginx can't
read it back, Let's Encrypt POSTs to the challenge URL, gets 403, and
the order fails with `urn:ietf:params:acme:error:unauthorized`.

NixOS's `enableACME = true;` nginx shorthand wires this automatically;
the helper here uses webroot directly (the prior `enableACME` path
asserted nginx had read access to the *issued cert*, which it
shouldn't), so we have to add nginx to the acme group ourselves.

Bit a live host today during the albatross.2140.dev → frigate.2140.dev
move: the first issuance after the hostname change failed 403 until
the leaf dir was manually chgrp'd to `nginx`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`openssl pkcs8 -out` hardcodes mode 0600 on the output regardless of
umask (defensive default for private-key files), so the `umask 0027`
preamble in postRun was a no-op for the actual file lego produced.
The resulting key.pem is acme:acme 0600 — frigate joins the `acme`
group via extraSupplementaryGroups but mode 0600 leaves the file
unreadable, and frigate fails on startup with

    SSL: failed to read private key /var/lib/acme/<host>/key.pem

Add an explicit `chmod 0640` after the conversion. Same bug shape
as the nginx-acme-group fix one commit earlier — both came up live
during the albatross.2140.dev -> frigate.2140.dev cutover.

Also drop the `umask 0027` line: it didn't do anything useful and
its presence suggests the perms are coming from the umask.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@josibake josibake merged commit 75ec6e1 into main May 19, 2026
2 checks passed
@josibake josibake deleted the fix/nginx-acme-group branch May 19, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant