Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
* JSON output for single objects now uses standard `"key": "value"` spacing (matching list output and `encoding/json` defaults).

### Bundles
* Validate that resource keys do not contain variable references ([#5169](https://github.com/databricks/cli/pull/5169))
* engine/direct: Drop the deployment state entry on a recreate before the follow-up `Create`, so a `Create` failure no longer leaves a broken state with `invalid state: empty id` on the next `bundle plan` ([#5173](https://github.com/databricks/cli/pull/5173)).

### Dependency updates
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
bundle:
name: variable-in-resource-key

variables:
env:
description: The target environment
default: dev

resources:
jobs:
${var.env}_job:
Comment thread
andrewnester marked this conversation as resolved.
name: my-job

targets:
dev:
default: true
resources:
jobs:
${var.env}_job_2:
name: my-job
${var.env}: my-job-description

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions acceptance/bundle/variables/variable_in_resource_key/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Warning: unknown field: ${var.env}
at targets.dev.resources.jobs.${var.env}_job_2
in databricks.yml:21:11

Error: resource key "${var.env}_job" must not contain variable references
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add as a comment or on PR description what output this test had before this PR?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see from #5098 that it's panic stacktrace, but good to confirm.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did fail with the same error before the fix

at resources.jobs.${var.env}_job
in databricks.yml:12:7

Error: resource key "${var.env}_job_2" must not contain variable references
at targets.dev.resources.jobs.${var.env}_job_2
in databricks.yml:20:11


Exit code: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
$CLI bundle plan
5 changes: 3 additions & 2 deletions bundle/config/mutator/mutator.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ func DefaultMutators(ctx context.Context, b *bundle.Bundle) {
InitializeVariables(),
DefineDefaultTarget(),

// Note: This mutator must run before the target overrides are merged.
// See the mutator for more details.
// Note: These mutators must run before the target overrides are merged.
// See the mutators for more details.
validate.NoVariableReferenceInResourceKey(),
validate.UniqueResourceKeys(),
)
}
56 changes: 56 additions & 0 deletions bundle/config/validate/no_variable_reference_in_resource_key.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
package validate

import (
"context"
"fmt"

"github.com/databricks/cli/bundle"
"github.com/databricks/cli/libs/diag"
"github.com/databricks/cli/libs/dyn"
"github.com/databricks/cli/libs/dyn/dynvar"
)

type noVariableReferenceInResourceKey struct{}

// NoVariableReferenceInResourceKey validates that no resource key contains a variable reference.
// Resource keys are used as identifiers throughout the deployment pipeline and must be static strings.
func NoVariableReferenceInResourceKey() bundle.Mutator {
return &noVariableReferenceInResourceKey{}
}

func (m *noVariableReferenceInResourceKey) Name() string {
return "validate:no_variable_reference_in_resource_key"
}

func (m *noVariableReferenceInResourceKey) Apply(_ context.Context, b *bundle.Bundle) diag.Diagnostics {
var diags diag.Diagnostics

patterns := []dyn.Pattern{
dyn.NewPattern(dyn.Key("resources"), dyn.AnyKey(), dyn.AnyKey()),
Comment thread
andrewnester marked this conversation as resolved.
Comment thread
andrewnester marked this conversation as resolved.
dyn.NewPattern(dyn.Key("targets"), dyn.AnyKey(), dyn.Key("resources"), dyn.AnyKey(), dyn.AnyKey()),
}

for _, pattern := range patterns {
_, err := dyn.MapByPattern(
b.Config.Value(),
pattern,
func(p dyn.Path, v dyn.Value) (dyn.Value, error) {
key := p[len(p)-1].Key()
if dynvar.ContainsVariableReference(key) {
diags = append(diags, diag.Diagnostic{
Severity: diag.Error,
Summary: fmt.Sprintf("resource key %q must not contain variable references", key),
Locations: v.Locations(),
Paths: []dyn.Path{p},
})
}
return v, nil
},
)
if err != nil {
diags = append(diags, diag.FromErr(err)...)
}
}

return diags
}
Loading