Skip to content

Update/verify dep vendoring#132

Open
Tim Hockin (thockin) wants to merge 11 commits into
agent-substrate:mainfrom
thockin:add-vendoring
Open

Update/verify dep vendoring#132
Tim Hockin (thockin) wants to merge 11 commits into
agent-substrate:mainfrom
thockin:add-vendoring

Conversation

@thockin
Copy link
Copy Markdown
Collaborator

This adds an update/verify script pair to check go mod vendor in the main module. Should we do this in sub-modules too?

Fixes #19

@dims
Copy link
Copy Markdown
Collaborator

Should we do this in sub-modules too?

yes please!

@thockin
Copy link
Copy Markdown
Collaborator Author

Now vendors all go.mod directories

@thockin
Copy link
Copy Markdown
Collaborator Author

I broke up the large commits into script-changes and run-the-script commits. the "run" commits are just the output of running the script.

Copy link
Copy Markdown
Collaborator

@dims Davanum Srinivas (dims) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

the individual vendor/ for each tool ... we'll get used to it.

@thockin
Copy link
Copy Markdown
Collaborator Author

rebased

@BenTheElder
Copy link
Copy Markdown
Collaborator

Benjamin Elder (BenTheElder) commented Jun 2, 2026

This adds an update/verify script pair to check go mod vendor in the main module. Should we do this in sub-modules too?

I'm not convinced. That adds a LOT more bloat, and reviewing tools-module vendor is not nearly as actionable.

We're using go mod as an implementation detail of obtaining those tools, we could just as well fetch binaries.

@BenTheElder
Copy link
Copy Markdown
Collaborator

Also, we shouldn't be dropping anything from LICENSES here, what's going on there?

@@ -1,27 +0,0 @@
// +build windows solaris
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code in LICENSES was here because the library is MPL and requires redistributing the sources. I think this commit broke something.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running go mod vendor causes go-licenses to NOT create these files. No logs, no warnings. Adding --include_tests does not bring it back (though we should probably add that).

Given that these are tests, they are not linked into our project, so I think the need for them is dubious.

@BenTheElder
Copy link
Copy Markdown
Collaborator

Github won't show me the full diff, even commit by commit.

@dims
Copy link
Copy Markdown
Collaborator

Benjamin Elder (@BenTheElder) i switched to diffshub for large commits

@BenTheElder
Copy link
Copy Markdown
Collaborator

Also, the tools modules are supposed to be updated by updating the version of the tool and letting the other deps be auto selected to the versions specified by the to upstream. We have a script for that. Which avoids breakage caused by patching those tools in unexpected ways.

So updating the tools will create a huge churn each time as ~all transitive deps update at once.

I would rather work out binary fetch for each than pretend to review their vendor.

If the concern is those tools stop building because the module proxy is missing a dependency, we should use binaries. If those are a concern, well we have bottom turtles we need to download for our images too.

@thockin
Copy link
Copy Markdown
Collaborator Author

One of the motivations for vendoring is "I am at the airport. I quickly clone the repo and get on the plane. I can do work." If I can't compile, that's a problem?

@BenTheElder
Copy link
Copy Markdown
Collaborator

We don't vendor the base images though.
The modules will be in your system module cache, just run a build before you leave.

This is needed when go-licenses fails to recognize standard library
packages as part of Go's core.

This breaks when using a versioned or managed Go toolchain (e.g., via Go
1.21+ automatic toolchain downloads) that places standard library files
outside your main GOROOT path.
@thockin
Copy link
Copy Markdown
Collaborator Author

just run a build before you leave is insufficient. You at least need to run hack/update-all. Anyway, I think I agree with you. Davanum Srinivas (@dims)?

I moved the commits to vendor tools to the end of this series. Other than that, I think it's good.

@thockin Tim Hockin (thockin) force-pushed the add-vendoring branch 2 times, most recently from 79177e4 to 8e0c849 Compare June 2, 2026 16:26
@thockin
Copy link
Copy Markdown
Collaborator Author

In fact, I pushed without the tools now. Let's debate that in a followup.

This removed go code from within LICENSES, but it was not caused by this
commit.  It would get removed as soon as we ran `go mod vendor` - I
guess go-licenses decided internally (no logs or anything) that
vendoring is good enough.  But these are tests - they are not linked in
anyway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should we vendor our dependencies?

3 participants