Skip to content

Hetzner: cpx (new gen) defaults + capacity-aware placement#166

Open
barnabasbusa wants to merge 5 commits intomasterfrom
bbusa/swap-hetzner-defaults-to-cpx
Open

Hetzner: cpx (new gen) defaults + capacity-aware placement#166
barnabasbusa wants to merge 5 commits intomasterfrom
bbusa/swap-hetzner-defaults-to-cpx

Conversation

@barnabasbusa
Copy link
Copy Markdown
Contributor

@barnabasbusa barnabasbusa commented May 6, 2026

Summary

Two changes layered together to address recurring resource_unavailable and unsupported location for server type errors on terraform apply:

1. New default SKUs

Was Now nbg1 €/mo
supernode cax41 (16 ARM / 32 GB / 320 GB) cpx62 (16 AMD / 32 GB / 640 GB) €50.49
fullnode cax31 (8 ARM / 16 GB / 160 GB) cpx42 (8 AMD / 16 GB / 320 GB) €25.49

Hetzner ARM cax* capacity has been chronically tight, and the legacy cpx41/cpx51 SKUs are no longer creatable in some EU locations. The new-gen cpx42/cpx62 are widely available.

Cost vs DigitalOcean

cpx* is more expensive than cax*/cx*, but still dramatically cheaper than DigitalOcean for equivalent specs:

Spec Hetzner cpx DO equivalent
8 vCPU / 16 GB / ~300 GB cpx42€25/mo s-8vcpu-16gb ≈ €88/mo (~$96)
16 vCPU / 32 GB / ~600 GB cpx62€50/mo s-16vcpu-32gb ≈ €177/mo (~$192)

So we're roughly 3–4× cheaper than DO while staying on a SKU that's actually in stock. For a multi-month devnet that's hundreds of euros saved per supernode.

The arch label logic at hetzner.tf already keys off the ^cax regex, so labels automatically flip to arch:amd64.

2. Capacity-aware placement

At every plan/apply, query hcloud_datacenters and hcloud_server_type data sources, build the set of locations whose datacenters currently report both SKUs as available, and round-robin new servers only across those. Falls back to the full var.hetzner_regions list if every region is sold out (so plan doesn't error out — at that point you're stuck either way).

  • lifecycle { ignore_changes = [location] } on hcloud_server keeps existing placements pinned even if next plan's filter would prefer a different region (location is replacement-forced on hcloud_server otherwise).
  • hcloud_server_network now reads hcloud_server.main[each.key].location (the actual post-state location) so the network reference stays correct under any drift.

Test plan

  • Downstream devnet repos consuming this template plan cleanly
  • Ansible playbooks resolve amd64 binaries/images for affected client and tooling roles
  • First apply on a fresh devnet provisions servers without resource_unavailable / unsupported location errors
  • echo 'local.hetzner_available_locations' | terraform console returns the expected non-empty list when capacity is healthy

ARM cax* capacity at Hetzner has been chronically tight across all
EU locations, causing repeated `resource_unavailable` placement
failures on apply. Switch the supernode/fullnode defaults to the
spec-equivalent AMD shared-vCPU sizes which have far better
availability:
  - supernode: cax41 (16/32) -> cpx51 (16/32)
  - fullnode:  cax31 (8/16)  -> cpx41 (8/16)

The arch label logic already keys off the `^cax` regex, so labels
will automatically flip to `arch:amd64`.
Hetzner is rolling out a new CX line that replaces the CPX series
in many locations. CPX SKUs (cpx41/cpx51) are no longer creatable
in fsn1/nbg1/hel1, returning `unsupported location for server type
(invalid_input)` from the API.

Switch defaults to the new CX equivalents at identical specs and
slightly lower price:
  - supernode: cpx51 (16/32) -> cx53 (16/32) — €31.99 -> €22.99
  - fullnode:  cpx41 (8/16)  -> cx43 (8/16)  — €24.49 -> €12.49
@barnabasbusa barnabasbusa changed the title Swap default Hetzner sizes from cax (ARM) to cpx (AMD) Swap default Hetzner sizes from cax (ARM) to cx (new gen) May 6, 2026
Hetzner ARM (cax*) capacity has been chronically tight in EU and
the legacy CPX line (cpx41/cpx51) is no longer creatable in some
regions. Newer cpx42/cpx62 are widely available and still ~3-4x
cheaper than DigitalOcean for equivalent specs.

Two behavior changes layered together:

1. Default sizes flipped to the new-gen CPX:
   - supernode: cpx62 (16 vCPU / 32 GB / 640 GB)
   - fullnode:  cpx42 (8 vCPU / 16 GB / 320 GB)

2. Capacity-aware placement: query `hcloud_datacenters` and
   `hcloud_server_type` data sources at plan time, build the set
   of locations whose datacenters currently report both SKUs as
   `available`, and round-robin only across those. Falls back to
   the full region list if every region is sold out.

   - `lifecycle { ignore_changes = [location] }` keeps existing
     servers pinned where they were placed even if next plan's
     filter would prefer a different region (location is
     replacement-forced otherwise).
   - `hcloud_server_network` reads the actual server location
     post-state so the network reference stays correct after any
     drift between planned and real location.

Arch labels still flip to `arch:amd64` automatically via the
existing `^cax` regex.
@barnabasbusa barnabasbusa changed the title Swap default Hetzner sizes from cax (ARM) to cx (new gen) Hetzner: cpx (new gen) defaults + capacity-aware placement May 6, 2026
Capture why we landed on cpx42/cpx62 (capacity vs cax/legacy cpx,
cost vs DigitalOcean) so future operators don't have to relitigate
the choice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant