Skip to content

feat(mixins): recurse into nested modules and only update existing copies#50

Open
oycyc wants to merge 6 commits into
mainfrom
fix/mixins-update-nested-instances-issue-47
Open

feat(mixins): recurse into nested modules and only update existing copies#50
oycyc wants to merge 6 commits into
mainfrom
fix/mixins-update-nested-instances-issue-47

Conversation

@oycyc
Copy link
Copy Markdown
Contributor

@oycyc oycyc commented May 17, 2026

Closes #47.

Summary

Make mixins:update-all recurse from the user's working directory and only update existing copies of the mixin. Drops the hard-coded ./components/ scope so the task works in any consuming repo regardless of layout, and a single run keeps the mixin in sync across every consumer (root modules, in-repo child modules, etc.).

Approach

  • Find existing copies of the mixin recursively from the current working directory and overwrite in place.
    • REASON BEING: A given mixin isn't universal — e.g. secrets.sops.tf only belongs in certain modules that consume SOPS. Copying it everywhere creates unused files that have to be deleted by hand. Existing presence on disk is the opt-in signal.
  • Search from . (the user's working directory) rather than ./components/, so the task works in repos that don't follow a ./components/ layout, and so a single run also keeps in-repo child modules in sync — not just top-level components.
  • Prune .terraform/, .terragrunt-cache/, and the source ./mixins/ directory so vendored child-module copies aren't overwritten and we don't try to copy the source mixin onto itself.

Test plan

Validated end-to-end against the internal mp-infra-spacelift-automation Terraform infra repo (masterpoint-internal/mp-infra-spacelift-automation), which has both a flat-and-nested root-module layout AND in-repo child modules that consume the same context.tf mixin from CloudPosse null-label. The same logic was first applied to that repo's local taskfiles/mixins/Taskfile.yml override and exercised before being ported here.

Commands run in mp-infra-spacelift-automation:

# Inventory after the prune change — picks up child-modules too
$ find . \( -type d \( -name .terraform -o -name .terragrunt-cache \) -o -path ./mixins \) -prune -o -type f -name context.tf -print | wc -l
26     # 24 root modules + 2 child modules

# Execute (--force because Task's source/generates cache would otherwise report up-to-date)
$ task --force l:mixins:update -- context
./mixins/context.tf.tpl -> ./root-modules/aws-iam/delegated-tf-role/context.tf
./mixins/context.tf.tpl -> ./root-modules/aws-iam/primary-tf-role/context.tf
./mixins/context.tf.tpl -> ./root-modules/demo/rds/context.tf
... 23 more lines ...
./mixins/context.tf.tpl -> ./child-modules/demo-rds/context.tf
./mixins/context.tf.tpl -> ./child-modules/demo-kms-key/context.tf

Verified:

  • Nested root-module instances picked up. All three nested instances (aws-iam/delegated-tf-role, aws-iam/primary-tf-role, demo/rds) appeared in the cp -v output.
  • In-repo child modules picked up. Both child-modules/demo-kms-key/context.tf and child-modules/demo-rds/context.tf were updated, which the previous ./components/-scoped version would have missed. This is the headline benefit of the recursive search.
  • Existing copies only. context.tf was overwritten in exactly the 26 directories that already had one on disk. Root modules without a context.tf (e.g. aws-organization, github-organization-secrets, github-repositories, tfstate-backend) were left untouched. git status showed no new files added.
  • Source ./mixins/ pruned. mixins/context.tf.tpl was confirmed unchanged after the run, so we didn't walk into the template source directory or attempt a self-copy.
  • .terraform/ pruned. A raw recursive scan would otherwise hit ~165 vendored main.tf entries; no mixin files were written into .terraform/ or .terragrunt-cache/.
  • Precondition behavior. Running with a non-existent mixin name fails cleanly with File does not exist: ./mixins/<file> and produces no copies.

@oycyc oycyc requested a review from a team as a code owner May 17, 2026 15:11
@oycyc oycyc requested a review from Gowiem May 17, 2026 15:11
@oycyc oycyc force-pushed the fix/mixins-update-nested-instances-issue-47 branch from fe1ae22 to 2bbc903 Compare May 17, 2026 15:13
@oycyc oycyc changed the title fix(mixins): recurse into nested components and only update existing copies fix(mixins): recurse into nested modules and only update existing copies May 17, 2026
oycyc added 2 commits May 17, 2026 11:15
…copies

Closes #47.

The `mixins:update-all` task previously assumed a flat `./components/*`
layout, so nested deployable instances like `aws-iam/delegated-tf-role`
were never updated.

It also unconditionally copied the mixin into every top-level component
directory, which is wrong: a given mixin is not universally applicable
(e.g. the `secrets.sops.tf` mixin only belongs in components that actually
consume SOPS secrets). Copying it everywhere creates unused files that
have to be deleted by hand.

Changes:

- Locate destinations by recursively finding existing copies of the named
  mixin under `./components/` (`find ... -type f -name <file>`). A
  component opts in to a mixin by having a copy of it on disk; this task
  only keeps those existing copies in sync. To adopt a mixin in a new
  component, copy it in manually once — that becomes the explicit opt-in
  and future `update-all` runs will track the template.
- Recursion picks up nested instance layouts (e.g.
  `components/aws-iam/delegated-tf-role/<file>`) automatically.
- Prune `.terraform/` and `.terragrunt-cache/` so vendored child-module
  copies that Terraform/Terragrunt drop on disk during init are not
  overwritten.
- Set `dir: "{{.USER_WORKING_DIR}}"` so `./mixins/` and `./components/`
  resolve from where the user invoked `task`, not from the included
  taskfile's own directory.
- Drop the `generates:` glob — it cannot express nested outputs and was
  causing Task to falsely treat the task as up-to-date.
- Rewrite the loop to avoid `read -d ''` (not supported by Task's
  mvdan/sh interpreter).
The remaining precondition for `mixins:update-all` is checking the source
template exists. The `./components/` directory check is unnecessary —
if it's missing, `find` simply produces no matches and the task no-ops
cleanly, which is the correct behavior. Removing the check also lets
the task succeed in repos that haven't created a `./components/`
directory yet.
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 17, 2026

📝 Walkthrough

Walkthrough

This PR updates the mixins:update-all task in lib/mixins/Taskfile.yml to support nested root-module instances. The task documentation now explicitly states it updates existing mixin copies under ./components/ without adding mixins to directories that lack them. The precondition was simplified from checking both directory existence and non-emptiness to only verifying the directory exists. The copy mechanism changed from a flat per-component iteration to a find-based approach that discovers main.tf files at any depth, excludes Terraform and Terragrunt cache directories, and copies the mixin template to each discovered location. The generates: declaration was removed.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The pull request fulfills all coding requirements from issue #47: recursively finds existing mixin copies using find with proper pruning, overwrites only existing files (not adding unwanted copies), and handles nested deployable instances as requested.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the mixin update task implementation; no out-of-scope modifications are present.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title accurately describes the main change: enabling recursive traversal into nested modules and restricting updates to only existing mixin copies, which directly addresses issue #47.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/mixins-update-nested-instances-issue-47

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/mixins/Taskfile.yml`:
- Around line 28-29: Quote the template variable {{.CLI_ARGS}} to prevent
word-splitting for filenames with spaces: update the find invocation to use
-name "{{.CLI_ARGS}}" and quote the source path in the cp command (e.g. cp -v
"./mixins/{{.CLI_ARGS}}" "$dest"); this fixes both usages in the shown find |
while ... block that reference {{.CLI_ARGS}}.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 682cbab8-2127-415d-ac88-9d156077d0c8

📥 Commits

Reviewing files that changed from the base of the PR and between dc2bbef and fe1ae22.

📒 Files selected for processing (1)
  • lib/mixins/Taskfile.yml

Comment thread lib/mixins/Taskfile.yml Outdated
@oycyc oycyc force-pushed the fix/mixins-update-nested-instances-issue-47 branch from 2bbc903 to 2fead6b Compare May 17, 2026 15:15
@oycyc oycyc changed the title fix(mixins): recurse into nested modules and only update existing copies feat(mixins): recurse into nested modules and only update existing copies May 17, 2026
Comment thread lib/mixins/Taskfile.yml Outdated
update-all:
desc: "Update a mixin across the components. Example: `task mixins:update-all -- context.tf`."
# Updates (by overwriting) every existing copy of the named mixin under
# ./components/ with the current ./mixins/<file> template.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to add this on, but since we don't really use ./components anymore -- Want to update this to be configurable so this is usable in mp-infra and new client repos?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gowiem I'm even thinking do it recursively based on the current user directory, rather than having user specify.

If a repo has internal custom child modules AND root modules that both use mixins (e.g. context null label), I would want both to be updated.

  • but then for huge monorepos, that's inefficient and they can't do it from root, they'd have to cd those directories. but then for monorepos, they have their dedicated directories for each "repo" so they be working from those directories I suppose e.g. repo/infra

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shape it how you want, let's just make sure that ./components is not the way. Maybe we force providing a directory so we can always ensure we're doing it in the right location? This isn't run all that often so some level of explicitness is fine.

oycyc added 3 commits May 17, 2026 11:29
Replaces the hard-coded `./components/` scope with a recursive search from
the current working directory. Existing copies of the mixin are discovered
wherever they live in the consuming repo, so a single `update-all` run
keeps the mixin in sync across every consumer — root modules, in-repo
child modules, and anything else that has opted in by having the file on
disk.

Motivation: not every consumer repo follows a `./components/` layout, and
even repos that do may also have in-repo child modules that consume the
same mixin (e.g. `context.tf` from CloudPosse null-label). Hard-coding
`./components/` left those out.

The source `./mixins/` directory is added to the prune list so we never
descend into it and accidentally copy the source onto itself. `.terraform/`
and `.terragrunt-cache/` continue to be pruned to avoid vendored
child-module copies.
The recursive search remains user-cwd-relative — invoking the task from a
subdirectory still scopes the update to that subtree — but the source
mixin path no longer depends on cwd. It's resolved from
`<repo-root>/mixins/<file>` via `git rev-parse --show-toplevel`, so the
task works from any directory inside the repo without the precondition
or `cp` source path breaking.

Also cleans up the long single-line `find` into a multi-line pipeline
with explicit prune/match clauses, factors repeated paths into
`vars.MIXIN_SRC`, and tightens the desc.
@oycyc oycyc requested a review from Gowiem May 17, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support nested root-module instances in mixin update task

2 participants