CLI panics when trying to deploy bundle

[agagrins]

Describe the issue

When running in CircleCI with a Docker image cimg:python3.11, and doing databricks bundle deploy --force-lock, the CLI panics

Steps to reproduce the behavior

1. docker pull cimg/python:3.11
2. docker run --rm -ti --mount type=bind,source=./bundle.yaml,target=/home/circleci/project/bundle.yaml,readonly cimg/python:3.11
3. curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sudo sh
4. databricks bundle deploy --force-lock

Expected Behavior

The asset bundle should be deployed

Actual Behavior

The asset bundle is not deployed

OS and CLI version

Docker image cimg:python3.11 and latest CLI version. It's the same for Python 3.10, and for several other images

Is this a regression?

Seems to be same same for v0.299.0, v0.298.0, and v0.297.2. Versions before that fail with the openpgp: key expired error. But yes, this whole thing used to work

Debug Logs

The specifics of the bundle config have been redacted

The Databricks CLI unexpectedly had a fatal error.
Please report this issue to Databricks in the form of a GitHub issue at:
https://github.com/databricks/cli

CLI Version: 0.299.0

Panic Payload: merge node already set

Stack Trace:
goroutine 1 [running]:
runtime/debug.Stack()
        runtime/debug/stack.go:26 +0x64
github.com/databricks/cli/cmd/root.Execute.func1()
        github.com/databricks/cli/cmd/root/root.go:117 +0x9c
panic({0x1169da0?, 0x197f3f0?})
        runtime/panic.go:783 +0x120
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).loadMapping(0x4000510bf0, 0x4000561d60, {{0x4000050c30?, 0x8b314?}, 0x20?, 0x14706a0?})
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:113 +0x8f8
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).load(0x4000561cc0?, 0x4000561d60)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:51 +0x250
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).loadMapping(0x40005acbf0, 0x4000561c20, {{0x4000050c30?, 0x1?}, 0x4000523140?, 0x1b?})
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:127 +0x1ac
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).load(0x4000561b80?, 0x4000561c20)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:51 +0x250
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).loadMapping(0x40005acbf0, 0x400054cdc0, {{0x4000050c30?, 0x1?}, 0x4000011f50?, 0x6?})
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:127 +0x1ac
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).load(0x400054cd20?, 0x400054cdc0)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:51 +0x250
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).loadMapping(0x40005acbf0, 0x400054ca00, {{0x4000050c30?, 0x0?}, 0x40005108a8?, 0x11c398?})
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:127 +0x1ac
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).load(0x400054c960?, 0x400054ca00)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:51 +0x250
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).loadDocument(...)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:74
github.com/databricks/cli/libs/dyn/yamlloader.(*loader).load(0x4000510c00?, 0x400054c8c0)
        github.com/databricks/cli/libs/dyn/yamlloader/loader.go:47 +0x84
github.com/databricks/cli/libs/dyn/yamlloader.LoadYAML({0x4000050c30, 0x46}, {0x1987800, 0x400054ed80})
        github.com/databricks/cli/libs/dyn/yamlloader/yaml.go:21 +0x28c
github.com/databricks/cli/bundle/config.LoadFromBytes({0x4000050c30, 0x46}, {0x4000554000, 0x755c, 0x755d})
        github.com/databricks/cli/bundle/config/root.go:107 +0x98
github.com/databricks/cli/bundle/config.Load({0x4000050c30, 0x46})
        github.com/databricks/cli/bundle/config/root.go:100 +0x68
github.com/databricks/cli/bundle/config/loader.(*entryPoint).Apply(0x4000077858?, {0x199f9b8?, 0x400054e150?}, 0x4000077808)
        github.com/databricks/cli/bundle/config/loader/entry_point.go:27 +0x44
github.com/databricks/cli/bundle.ApplyContext({0x199f9b8, 0x4000337c50}, 0x4000077808, {0x1993b48, 0x28a71e0})
        github.com/databricks/cli/bundle/mutator.go:91 +0x194
github.com/databricks/cli/bundle.ApplySeqContext({0x199f9b8, 0x4000337c50}, 0x4000077808, {0x4000511198?, 0x4000511188?, 0xb99268?})
        github.com/databricks/cli/bundle/mutator.go:100 +0x60
github.com/databricks/cli/bundle/config/mutator.DefaultMutators({0x199f9b8, 0x4000337c50}, 0x4000077808)
        github.com/databricks/cli/bundle/config/mutator/mutator.go:14 +0x14c
github.com/databricks/cli/bundle/phases.LoadDefaultTarget({0x199f9b8, 0x4000337c50}, 0x4000077808)
        github.com/databricks/cli/bundle/phases/load.go:23 +0x44
github.com/databricks/cli/cmd/root.configureBundle(0x40002fb208, 0x4000077808)
        github.com/databricks/cli/cmd/root/bundle.go:148 +0x4c
github.com/databricks/cli/cmd/root.MustConfigureBundle(0x40002fb208)
        github.com/databricks/cli/cmd/root/bundle.go:203 +0x78
github.com/databricks/cli/cmd/bundle/utils.ProcessBundleRet(0x40002fb208, {0x0, 0x4000538a40, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0, 0x0, ...})
        github.com/databricks/cli/cmd/bundle/utils/process.go:100 +0x12c
github.com/databricks/cli/cmd/bundle/utils.ProcessBundle(...)
        github.com/databricks/cli/cmd/bundle/utils/process.go:83
github.com/databricks/cli/cmd/bundle.newDeployCommand.func1(0x40002fb208, {0x40004a10d0?, 0x4?, 0x1641114?})
        github.com/databricks/cli/cmd/bundle/deploy.go:47 +0x124
github.com/spf13/cobra.(*Command).execute(0x40002fb208, {0x40004a10b0, 0x1, 0x1})
        github.com/spf13/cobra@v1.10.2/command.go:1015 +0x7d4
github.com/spf13/cobra.(*Command).ExecuteC(0x40001acf08)
        github.com/spf13/cobra@v1.10.2/command.go:1148 +0x350
github.com/spf13/cobra.(*Command).ExecuteContextC(...)
        github.com/spf13/cobra@v1.10.2/command.go:1080
github.com/databricks/cli/cmd/root.Execute({0x199f980, 0x28a71e0}, 0x40001acf08)
        github.com/databricks/cli/cmd/root/root.go:146 +0x108
main.main()
        github.com/databricks/cli/main.go:17 +0x44

Exited with code exit status 1

[denik]

@agagrins could you try earlier versions (make sure to use latest patched one, e.g. 0.296.1 instead of 0.296.0).

Also, can you send the config that causes it to dabs-feedback@databricks.com

[agagrins]

Using version v.213.0 doesn't panic, when run locally. From v.214.0 they mostly seem to

1. docker pull cimg/python:3.11
2. docker run --rm -ti --mount type=bind,source=./bundle.yaml,target=/home/circleci/project/bundle.yaml,readonly cimg/python:3.11
3. curl -L -s -O https://github.com/databricks/cli/releases/download/v0.213.0/databricks_cli_0.213.0_linux_arm64.zip
4. unzip databricks_cli_0.213.0_linux_arm64.zip
5. ./databricks bundle deploy --force-lock

The bundle.yaml file is big and contains lots of sensitive information, so no. That one I cannot share