Skip to content

feat: publish and use nix-catalog #1995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
samrose merged 5 commits into from
Jan 8, 2026
+88 −5
Merged

feat: publish and use nix-catalog #1995

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
890c9ee
feat: publish and use nix-catalog
samrose Jan 5, 2026
08705bc
fix: catalog entries hash plus version
samrose Jan 5, 2026
01b77e0
chore: use OIDC to auth shared services bucket (#1997)
tomashley Jan 6, 2026
aebc9f7
fix: use aws cli to access catalog
samrose Jan 6, 2026
d235f25
chore: bump to release
samrose Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/ami-release-nix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,52 @@ jobs:
aws s3 cp /tmp/pg_binaries.tar.gz s3://${{ secrets.PROD_ARTIFACTS_BUCKET }}/upgrades/postgres/supabase-postgres-${{ steps.process_release_version.outputs.version }}/24.04.tar.gz
aws s3 cp /tmp/pg_binaries.tar.gz s3://${{ secrets.PROD_ARTIFACTS_BUCKET }}/upgrades/postgres/supabase-postgres-${{ steps.process_release_version.outputs.version }}/upgrade_bundle.tar.gz

- name: GitHub OIDC Auth
uses: aws-actions/configure-aws-credentials@v4.1.0
with:
aws-region: ap-southeast-1
role-to-assume: arn:aws:iam::279559813984:role/supabase-github-oidc-role
role-session-name: shared-services-jump

- name: Assume destination role
uses: aws-actions/configure-aws-credentials@v4.1.0
with:
aws-region: ap-southeast-1
role-to-assume: arn:aws:iam::279559813984:role/supabase-nix-catalog-artifacts-role-6387512
role-skip-session-tagging: true
role-session-name: upload-assets
role-chaining: true

- name: Update nix store path catalog
run: |
VERSION="${{ steps.process_release_version.outputs.version }}"
GIT_SHA="${{ github.sha }}"
PG_VERSION="${{ matrix.postgres_version }}"
SYSTEM="aarch64-linux"
Copy link

@coderabbitai coderabbitai bot Jan 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

x86_64 instances will fail to find their store path in the catalog.

SYSTEM is hardcoded to aarch64-linux, and the workflow matrix only includes ARM runners. When x86_64 instances attempt an upgrade, they will fail at line 333-338 in initiate.sh because the catalog won't contain an x86_64-linux entry.

Either:

  1. Add x86_64 builds to the matrix and generate catalog entries for both architectures, or
  2. Ensure x86_64 upgrades use a different mechanism.
#!/bin/bash
# Check if there's a separate x86_64 workflow or if x86 is expected to be supported
rg -n "x86_64" .github/workflows/ --type yaml
rg -n "amd64" .github/workflows/ --type yaml
🤖 Prompt for AI Agents 
In @.github/workflows/ami-release-nix.yml at line 162, The workflow hardcodes
SYSTEM="aarch64-linux", causing x86_64 instances to miss catalog entries and
fail in initiate.sh (lines ~333-338); update the workflow matrix to include
x86_64 (amd64) builds so catalog entries are generated for both architectures
and set SYSTEM dynamically from the matrix/runner architecture, or modify the
upgrade path to detect uname -m/runner.arch and map it to
"x86_64-linux"/"aarch64-linux" so initiate.sh finds the correct store path;
ensure the matrix keys and any catalog-generation steps produce entries for both
architectures and that SYSTEM is derived from the selected matrix value rather
than hardcoded.

Copy link
Collaborator Author

@samrose samrose Jan 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not flag issues related to AMI release that relevant to x86_64-linux system architecture, as we are not supporting that on the AMI release

Copy link

@coderabbitai coderabbitai bot Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samrose, understood! I've noted that x86_64-linux is not supported for AMI releases in this repository. I won't flag similar architecture-related issues in the future.

✏️ Learnings added 
Learnt from: samrose
Repo: supabase/postgres PR: 1995
File: .github/workflows/ami-release-nix.yml:162-162
Timestamp: 2026-01-07T14:50:16.501Z
Learning: In the supabase/postgres repository, AMI releases only support aarch64-linux architecture. x86_64-linux is not supported for AMI releases, so issues related to missing x86_64 support in AMI release workflows should not be flagged.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


# Get store path for this build
STORE_PATH=$(nix eval --raw ".#psql_${PG_VERSION}/bin.outPath")

# Each postgres version gets its own catalog file (no race conditions)
CATALOG_S3="s3://${{ secrets.SHARED_AWS_ARTIFACTS_BUCKET }}/nix-catalog/${GIT_SHA}-psql_${PG_VERSION}.json"

# Create catalog JSON for this version
jq -n \
--arg ver "$VERSION" \
--arg sha "$GIT_SHA" \
--arg sys "$SYSTEM" \
--arg path "$STORE_PATH" \
'{version: $ver, git_sha: $sha, ($sys): $path}' > /tmp/catalog.json

echo "Catalog for psql_${PG_VERSION}:"
cat /tmp/catalog.json

# Upload catalog
aws s3 cp /tmp/catalog.json "$CATALOG_S3" \
--content-type "application/json"

echo "Catalog uploaded to ${CATALOG_S3}"

Comment on lines +157 to +186
Copy link

@coderabbitai coderabbitai bot Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

# Search for bucket name references in workflow and script files
rg -n "SHARED_AWS_ARTIFACTS_BUCKET|supabase-internal-artifacts" --type yaml --type bash --type sh

Repository: supabase/postgres

Length of output: 89

🏁 Script executed:

# Also check the specific files mentioned
echo "=== Checking ami-release-nix.yml around line 168 ==="
fd "ami-release-nix.yml" -x cat -n {} | sed -n '160,190p'

echo "=== Checking initiate.sh around line 322 ==="
fd "initiate.sh" -x cat -n {} | sed -n '310,335p'

Repository: supabase/postgres

Length of output: 1482

🏁 Script executed:

# Find the workflow file
fd "ami-release-nix.yml" -x wc -l {}

echo "=== Full ami-release-nix.yml content ==="
fd "ami-release-nix.yml" -x cat -n {}

Repository: supabase/postgres

Length of output: 101

🏁 Script executed:

# List all workflow files
echo "=== Workflow files in .github/workflows/ ==="
fd . ".github/workflows/" -type f

echo "=== Search for SHARED_AWS_ARTIFACTS_BUCKET ==="
rg "SHARED_AWS_ARTIFACTS_BUCKET" --type yaml

Repository: supabase/postgres

Length of output: 340

🏁 Script executed:

# Search for any bucket references in workflows
rg "nix-catalog|SHARED_AWS" --type yaml -A 2 -B 2

Repository: supabase/postgres

Length of output: 43

🏁 Script executed:

# Check repository structure
echo "=== Root directory ==="
ls -la | head -20

echo "=== .github directory ==="
ls -la .github/ 2>/dev/null || echo ".github not found"

echo "=== Check for .github/workflows ==="
ls -la .github/workflows/ 2>/dev/null || echo ".github/workflows not found"

Repository: supabase/postgres

Length of output: 3353

🏁 Script executed:

# Search for the references more broadly
echo "=== Search for SHARED_AWS_ARTIFACTS_BUCKET ==="
rg "SHARED_AWS_ARTIFACTS_BUCKET"

echo "=== Search for nix-catalog ==="
rg "nix-catalog"

echo "=== Search for supabase-internal-artifacts ==="
rg "supabase-internal-artifacts"

Repository: supabase/postgres

Length of output: 679

🏁 Script executed:

# Read the ami-release-nix.yml file
echo "=== Full ami-release-nix.yml content with line numbers ==="
cat -n .github/workflows/ami-release-nix.yml

Repository: supabase/postgres

Length of output: 10429

Verify that secrets.SHARED_AWS_ARTIFACTS_BUCKET equals supabase-internal-artifacts.

The workflow uploads nix catalogs to a bucket specified by the SHARED_AWS_ARTIFACTS_BUCKET secret (line 168), but initiate.sh downloads from the hardcoded bucket supabase-internal-artifacts (line 322). If these don't match, pg_upgrade operations will fail when fetching catalogs.

Ensure the GitHub Actions secret is configured with the correct bucket name, or update one of the references to use a consistent source of truth.

🤖 Prompt for AI Agents 
In @.github/workflows/ami-release-nix.yml around lines 157 - 186, The workflow
uses secrets.SHARED_AWS_ARTIFACTS_BUCKET to construct CATALOG_S3 when uploading
the nix catalog, but initiate.sh downloads catalogs from the hardcoded bucket
name supabase-internal-artifacts, causing a mismatch; fix by making the source
of truth consistent: either set the GitHub secret SHARED_AWS_ARTIFACTS_BUCKET to
supabase-internal-artifacts, or update initiate.sh to read the bucket from the
same secret/environment variable (or parameter) used to build CATALOG_S3 so both
upload (CATALOG_S3 in the workflow) and download (the catalog fetch logic in
initiate.sh) reference the identical bucket name.

- name: Create release
uses: softprops/action-gh-release@v2
with:
Expand Down
41 changes: 39 additions & 2 deletions ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,11 +301,48 @@ EXTRA_NIX_CONF
fi
fi

echo "1.2. Installing flake revision: $NIX_FLAKE_VERSION"
echo "1.2. Fetching store path for flake revision: $NIX_FLAKE_VERSION"
# shellcheck disable=SC1091
source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
nix-collect-garbage -d > /tmp/pg_upgrade-nix-gc.log 2>&1 || true
PG_UPGRADE_BIN_DIR=$(nix build "github:supabase/postgres/${NIX_FLAKE_VERSION}#psql_${PGVERSION}/bin" --no-link --print-out-paths --extra-experimental-features nix-command --extra-experimental-features flakes)

# Determine system architecture
ARCH=$(uname -m)
if [ "$ARCH" = "aarch64" ]; then
SYSTEM="aarch64-linux"
elif [ "$ARCH" = "x86_64" ]; then
SYSTEM="x86_64-linux"
else
echo "ERROR: Unsupported architecture: $ARCH"
exit 1
fi

# Fetch store path from catalog (avoids expensive nix eval - prevents OOM on small instances)
# Each postgres version has its own catalog file: {git_sha}-psql_{version}.json
CATALOG_S3="s3://supabase-internal-artifacts/nix-catalog/${NIX_FLAKE_VERSION}-psql_${PGVERSION}.json"
CATALOG_LOCAL="/tmp/nix-catalog-${NIX_FLAKE_VERSION}-psql_${PGVERSION}.json"
echo "Fetching catalog from: $CATALOG_S3"

if ! aws s3 cp "$CATALOG_S3" "$CATALOG_LOCAL" --region ap-southeast-1; then
echo "ERROR: Failed to fetch catalog from $CATALOG_S3"
exit 1
fi
Comment on lines +320 to +329
Copy link

@coderabbitai coderabbitai bot Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for SHARED_AWS_ARTIFACTS_BUCKET references
echo "=== SHARED_AWS_ARTIFACTS_BUCKET references ==="
rg -n "SHARED_AWS_ARTIFACTS_BUCKET" .github/workflows/

echo ""
echo "=== supabase-internal-artifacts references ==="
rg -n "supabase-internal-artifacts" ansible/ .github/workflows/

echo ""
echo "=== Context around ami-release-nix.yml line 168 ==="
head -n 180 .github/workflows/ami-release-nix.yml | tail -n 20

Repository: supabase/postgres

Length of output: 1328

🏁 Script executed:

#!/bin/bash
# Check how initiate.sh is invoked
echo "=== References to initiate.sh ==="
rg -n "initiate.sh" .github/workflows/

echo ""
echo "=== Check if CATALOG_S3 or bucket is passed to initiate.sh ==="
rg -n "CATALOG_S3\|supabase-internal-artifacts" -B 5 -A 5 ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh | head -n 50

echo ""
echo "=== Check workflow invocation of script ==="
rg -n "initiate.sh\|ansible" .github/workflows/ami-release-nix.yml

Repository: supabase/postgres

Length of output: 203

🏁 Script executed:

#!/bin/bash
# Verify file exists and check its structure
echo "=== Check if file exists ==="
ls -la ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh

echo ""
echo "=== Get file info ==="
wc -l ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh

echo ""
echo "=== Check directory contents ==="
ls -la ansible/files/admin_api_scripts/pg_upgrade_scripts/

echo ""
echo "=== Look for any ansible playbook usage ==="
fd -e yml -e yaml . ansible/ | head -20

echo ""
echo "=== Check if initiate.sh is called from any other script ==="
rg -r "initiate" ansible/

Repository: supabase/postgres

Length of output: 15199

🏁 Script executed:

#!/bin/bash
# Check the context around lines 320-329 of initiate.sh
echo "=== Context around lines 320-329 ==="
sed -n '310,340p' ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh

echo ""
echo "=== Check script header/purpose ==="
head -n 30 ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh

echo ""
echo "=== Look for any parameterization of bucket ==="
grep -n "CATALOG\|bucket\|S3\|supabase-internal" ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh

Repository: supabase/postgres

Length of output: 2780

🏁 Script executed:

#!/bin/bash
# Check for secrets definition or documentation
echo "=== Look for GitHub secrets or documentation ==="
find . -name "*.md" -type f | xargs grep -l "SHARED_AWS_ARTIFACTS_BUCKET\|supabase-internal-artifacts" 2>/dev/null

echo ""
echo "=== Check for any environment or config files ==="
find . -name ".env*" -o -name "*secrets*" -o -name "*config*" 2>/dev/null | head -20

echo ""
echo "=== Search for how script gets its environment ==="
grep -n "NIX_FLAKE_VERSION\|PGVERSION" ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh | head -10

echo ""
echo "=== Check if script sources environment from anywhere ==="
sed -n '1,100p' ansible/files/admin_api_scripts/pg_upgrade_scripts/initiate.sh | grep -n "source\|export\|\.env"

Repository: supabase/postgres

Length of output: 1377

🏁 Script executed:

#!/bin/bash
# Check common.sh for NIX_FLAKE_VERSION
echo "=== Search for NIX_FLAKE_VERSION in common.sh ==="
grep -n "NIX_FLAKE_VERSION" ansible/files/admin_api_scripts/pg_upgrade_scripts/common.sh

echo ""
echo "=== Get context around NIX_FLAKE_VERSION definition ==="
grep -n -B 3 -A 3 "NIX_FLAKE_VERSION" ansible/files/admin_api_scripts/pg_upgrade_scripts/common.sh

echo ""
echo "=== Check if NIX_FLAKE_VERSION is passed from somewhere else ==="
grep -rn "NIX_FLAKE_VERSION" ansible/ .github/ 2>/dev/null | head -20

Repository: supabase/postgres

Length of output: 1100

🏁 Script executed:

#!/bin/bash
# Check the workflow more carefully to understand the relationship
echo "=== Full ami-release-nix.yml around line 168 ==="
sed -n '150,200p' .github/workflows/ami-release-nix.yml

echo ""
echo "=== Check if catalog is actually uploaded by workflow ==="
sed -n '1,250p' .github/workflows/ami-release-nix.yml | grep -n "s3 cp\|s3 upload\|CATALOG"

Repository: supabase/postgres

Length of output: 3514

Use consistent bucket reference between workflow and script.

The workflow uploads the catalog to s3://${{ secrets.SHARED_AWS_ARTIFACTS_BUCKET }}/nix-catalog/... (line 182), but the script reads from hardcoded s3://supabase-internal-artifacts/nix-catalog/... (line 322). If the secret resolves to a different bucket name, the catalog fetch fails. Additionally, the path prefixes differ: the workflow uses ${GIT_SHA} while the script uses ${NIX_FLAKE_VERSION}. Parameterize the bucket name in the script or hardcode it consistently in both places.


STORE_PATH=$(jq -r ".\"${SYSTEM}\"" "$CATALOG_LOCAL")

if [ -z "$STORE_PATH" ] || [ "$STORE_PATH" = "null" ]; then
echo "ERROR: Could not find store path in catalog for ${SYSTEM}"
echo "Catalog contents:"
jq . "$CATALOG_LOCAL"
exit 1
fi

echo "Store path: $STORE_PATH"

# Realize from binary cache (no nix evaluation needed!)
nix-store -r "$STORE_PATH"

PG_UPGRADE_BIN_DIR="$STORE_PATH"
PGSHARENEW="$PG_UPGRADE_BIN_DIR/share/postgresql"
fi

Expand Down
6 changes: 3 additions & 3 deletions ansible/vars.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ postgres_major:

# Full version strings for each major version
postgres_release:
postgresorioledb-17: "17.6.0.027-orioledb"
postgres17: "17.6.1.070"
postgres15: "15.14.1.070"
postgresorioledb-17: "17.6.0.028-orioledb"
postgres17: "17.6.1.071"
postgres15: "15.14.1.071"

# Non Postgres Extensions
pgbouncer_release: 1.19.0
Expand Down
Loading