integration/bundle: install Terraform locally so tests bypass registry.terraform.io#5175
Draft
shreyas-goenka wants to merge 1 commit intomainfrom
Draft
integration/bundle: install Terraform locally so tests bypass registry.terraform.io#5175shreyas-goenka wants to merge 1 commit intomainfrom
shreyas-goenka wants to merge 1 commit intomainfrom
Conversation
The Databricks-internal CI runners that execute these tests cannot reach
registry.terraform.io, so `bundle deploy` inside tests like
TestGenerateFromExistingJobAndDeploy fails inside `terraform init` with:
Could not retrieve the list of available versions for provider
databricks/databricks: could not connect to registry.terraform.io: ... EOF
acceptance/install_terraform.py already exists and downloads Terraform 1.5.5
plus the matching Databricks provider into a local filesystem mirror
(driven from `acceptance` tests). Wire it into the integration/bundle
package via a new TestMain and a small testutil helper:
- internal/testutil/terraform.go: SetupTerraform() invokes the script
against acceptance/build/{GOOS}_{GOARCH} and exports
TF_CLI_CONFIG_FILE / DATABRICKS_TF_CLI_CONFIG_FILE /
DATABRICKS_TF_EXEC_PATH / TERRAFORM. The CLI's existing
getEnvVarWithMatchingVersion path picks these up (init.go:195) and
routes terraform init at the local mirror instead of registry.terraform.io.
- integration/bundle/main_test.go: TestMain calls SetupTerraform() once
per package run before any test executes. Same setup runs for `go test
./integration/bundle/...` on a developer laptop and in CI.
Verified locally: `go test ./integration/bundle/... -run TestNonExistent`
runs TestMain, downloads terraform + provider 1.113.0 to
acceptance/build/darwin_arm64/, and writes the .terraformrc with the
filesystem_mirror block. CLOUD_ENV-gated tests still skip without a
workspace, but TestMain has already staged Terraform by then.
Co-authored-by: Isaac
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Databricks-internal CI runners can't reach `registry.terraform.io`, so deploy-touching integration tests in `integration/bundle` (`TestGenerateFromExistingJobAndDeploy`, `TestGenerateFromExistingPipelineAndDeploy`, `TestSparkJarTaskDeployAndRunOn{Volumes,Workspace}`) fail inside `terraform init`:
```
Could not retrieve the list of available versions for provider
databricks/databricks: could not connect to registry.terraform.io: ... EOF
```
`acceptance/install_terraform.py` already exists and stages Terraform 1.5.5 + the matching Databricks provider into a local filesystem mirror, and the CLI already honors `DATABRICKS_TF_CLI_CONFIG_FILE` (see `bundle/deploy/terraform/init.go:195`). This PR wires that into the `integration/bundle` package via TestMain so deploy tests use the local mirror instead of going to registry.terraform.io.
Changes
Same setup works for `go test ./integration/bundle/...` on a laptop and in CI.
Test plan
This pull request was AI-assisted by Isaac.