Skip to content

fix(lakekeeper): set ACCESS_DELEGATION_MODE 'none' on Iceberg ATTACH #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fuziontech merged 1 commit into from
May 21, 2026
+29 −12
Merged

fix(lakekeeper): set ACCESS_DELEGATION_MODE 'none' on Iceberg ATTACH #600

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions server/iceberg/migration.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,35 +99,52 @@ func BuildLakekeeperSecretStmt(cfg Config) string {
)
}

// LakekeeperAccessDelegationNone disables Iceberg REST credential vending on
// the ATTACH. This MUST be set explicitly: DuckDB's iceberg extension defaults
// ACCESS_DELEGATION_MODE to 'vended_credentials', so simply *omitting* the
// option does NOT turn vending off (the lesson from the first live write test).
const LakekeeperAccessDelegationNone = "ACCESS_DELEGATION_MODE 'none'"

// BuildLakekeeperAttachStmt builds the ATTACH statement for the Iceberg
// REST catalog served by Lakekeeper.
//
// We deliberately do NOT use ACCESS_DELEGATION_MODE 'vended_credentials':
// Lakekeeper's STS vending overflows AWS's packed-policy limit
// (PackedPolicyTooLarge), and it's unnecessary — the worker already holds S3
// creds for the warehouse bucket via the iceberg_sigv4 secret (built from the
// duckling's brokered STS creds, same per-org role/bucket; see
// attachLakekeeperCatalog). DuckDB uses that secret to read/write table data;
// Lakekeeper serves only catalog metadata.
// We explicitly set ACCESS_DELEGATION_MODE 'none' (NOT 'vended_credentials',
// and crucially NOT omitted — see below). Two reasons vending is wrong here:
// - Lakekeeper's STS vending overflows AWS's packed-policy limit
// (PackedPolicyTooLarge), so vending is disabled on the warehouse anyway.
// - With vending disabled server-side, a delegation-requesting client still
// gets a per-table storage *config* (region/endpoint) back with NO
// credentials. DuckDB turns that into a path-scoped S3 secret with empty
// creds, and because its scope (the table's S3 prefix) is MORE SPECIFIC
// than iceberg_sigv4's (s3://), it SHADOWS our working secret — every data
// read/write then goes out anonymous and S3 returns 403. Verified live on
// `ben` in managed-warehouse-dev: omitting the option 403'd; 'none' fixed it.
//
// With delegation off, DuckDB falls back to the ambient iceberg_sigv4 secret,
// which holds the duckling's own brokered S3 creds for the warehouse bucket
// (built in attachLakekeeperCatalog from the same per-org role/bucket). DuckDB
// uses that to read/write table data; Lakekeeper serves only catalog metadata.
//
// When OAUTH2_SERVER_URI is empty (allowall mode), AUTHORIZATION_TYPE 'none'
// is set instead of SECRET. When OAuth2 is configured, SECRET references
// the secret built by BuildLakekeeperSecretStmt.
func BuildLakekeeperAttachStmt(cfg Config) string {
if cfg.LakekeeperOAuth2ServerURI == "" {
return fmt.Sprintf(
"ATTACH '%s' AS %s (TYPE ICEBERG, ENDPOINT '%s', AUTHORIZATION_TYPE 'none')",
"ATTACH '%s' AS %s (TYPE ICEBERG, ENDPOINT '%s', AUTHORIZATION_TYPE 'none', %s)",
escapeSQLStringLiteral(cfg.LakekeeperWarehouse),
CatalogName,
escapeSQLStringLiteral(cfg.LakekeeperEndpoint),
LakekeeperAccessDelegationNone,
)
}
return fmt.Sprintf(
"ATTACH '%s' AS %s (TYPE ICEBERG, ENDPOINT '%s', SECRET %s)",
"ATTACH '%s' AS %s (TYPE ICEBERG, ENDPOINT '%s', SECRET %s, %s)",
escapeSQLStringLiteral(cfg.LakekeeperWarehouse),
CatalogName,
escapeSQLStringLiteral(cfg.LakekeeperEndpoint),
LakekeeperSecretName,
LakekeeperAccessDelegationNone,
)
}

Expand Down
6 changes: 3 additions & 3 deletions server/iceberg/migration_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ func TestBuildLakekeeperAttachStmt_Allowall(t *testing.T) {
LakekeeperEndpoint: "http://lakekeeper-acme.lakekeeper.svc:8181/catalog",
LakekeeperWarehouse: "org-acme",
})
want := "ATTACH 'org-acme' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://lakekeeper-acme.lakekeeper.svc:8181/catalog', AUTHORIZATION_TYPE 'none')"
want := "ATTACH 'org-acme' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://lakekeeper-acme.lakekeeper.svc:8181/catalog', AUTHORIZATION_TYPE 'none', ACCESS_DELEGATION_MODE 'none')"
if got != want {
t.Fatalf("BuildLakekeeperAttachStmt (allowall) mismatch:\n got: %s\nwant: %s", got, want)
}
Expand All @@ -96,7 +96,7 @@ func TestBuildLakekeeperAttachStmt_OAuth2(t *testing.T) {
LakekeeperWarehouse: "org-acme",
LakekeeperOAuth2ServerURI: "http://oidc/token",
})
want := "ATTACH 'org-acme' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://lakekeeper-acme.lakekeeper.svc:8181/catalog', SECRET iceberg_oauth)"
want := "ATTACH 'org-acme' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://lakekeeper-acme.lakekeeper.svc:8181/catalog', SECRET iceberg_oauth, ACCESS_DELEGATION_MODE 'none')"
if got != want {
t.Fatalf("BuildLakekeeperAttachStmt (oauth2) mismatch:\n got: %s\nwant: %s", got, want)
}
Expand All @@ -107,7 +107,7 @@ func TestBuildLakekeeperAttachStmt_EscapesQuotes(t *testing.T) {
LakekeeperEndpoint: "http://h/cat?x='y'",
LakekeeperWarehouse: "wh'name",
})
want := "ATTACH 'wh''name' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://h/cat?x=''y''', AUTHORIZATION_TYPE 'none')"
want := "ATTACH 'wh''name' AS iceberg (TYPE ICEBERG, ENDPOINT 'http://h/cat?x=''y''', AUTHORIZATION_TYPE 'none', ACCESS_DELEGATION_MODE 'none')"
if got != want {
t.Fatalf("BuildLakekeeperAttachStmt did not escape quotes:\n got: %s\nwant: %s", got, want)
}
Expand Down
Loading