Skip to content

Commit 43eac2e

Browse files
authored
Merge pull request #40 from netresearch/docs/update-readme-and-adrs
docs: update README, add ADR-007 and ADR-008
2 parents 39b106c + 122418a commit 43eac2e

5 files changed

Lines changed: 331 additions & 30 deletions

File tree

README.md

Lines changed: 27 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ Example (abridged):
9999
Note: Not all of these are expected to be installed globally; the report simply surfaces what is present and how it compares upstream.
100100

101101
## Requirements
102-
- Python 3.9+
102+
- Python 3.14+
103103
- Network access to query GitHub/PyPI/crates.io/npm
104104

105105
## Quick Start
@@ -451,19 +451,29 @@ If `source_kind` is `skip`, upstream lookup is disabled for that tool.
451451
python3 -m pyflakes cli_audit.py
452452
```
453453

454-
- Run tests (n/a): This repo currently ships without tests. PRs welcome.
454+
- Run tests:
455+
```bash
456+
make test # Run all 490+ tests
457+
make test-unit # Unit tests only
458+
make test-coverage # With coverage report
459+
make test-parallel # Parallel via pytest-xdist
460+
```
455461

456462
## Installation scripts
457463

458-
Language-agnostic core tools and language-specific stacks are provided under `scripts/`:
464+
All 89 cataloged tools can be installed, upgraded, uninstalled, or reconciled via generic Make targets:
459465

460466
```bash
461-
make scripts-perms
467+
# Generic pattern targets - work for ANY cataloged tool
468+
make install-<tool> # e.g., make install-jq, make install-semgrep
469+
make upgrade-<tool> # e.g., make upgrade-ripgrep
470+
make uninstall-<tool> # e.g., make uninstall-bat
471+
make reconcile-<tool> # e.g., make reconcile-node
462472

463-
# Core simple tools (fd, fzf, ripgrep, jq, yq, bat, delta, just)
464-
make install-core
473+
# Group install by catalog tag
474+
make install-core # Core tools (fd, fzf, ripgrep, jq, yq, bat, delta, just)
465475

466-
# Language stacks
476+
# Language stacks (dedicated scripts with full lifecycle management)
467477
make install-python
468478
make install-node
469479
make install-go
@@ -478,7 +488,7 @@ make install-brew
478488
make install-rust
479489
```
480490

481-
These scripts prefer the most up-to-date sources (e.g., nvm for Node, vendor installers for AWS CLI and kubectl) when feasible.
491+
The pattern targets use a fallback chain: if a dedicated script exists (`scripts/install_<tool>.sh`), it is used; otherwise, the generic installer (`scripts/install_tool.sh`) reads the tool's catalog JSON entry and delegates to the appropriate method-specific installer. This means adding support for a new tool only requires creating a `catalog/<tool>.json` file.
482492

483493
### Role-focused quick checks (local-only)
484494

@@ -520,20 +530,19 @@ The audit attempts to identify how a tool was installed by inspecting the resolv
520530

521531
When ambiguous, the audit may report a generic bucket (e.g., `~/.local/bin`). The JSON output includes `installed_path_resolved` and `classification_reason` to aid debugging.
522532

523-
### Actions: install, update, uninstall, reconcile
533+
### Actions: install, upgrade, uninstall, reconcile
524534

525-
All scripts accept an action argument. Defaults to `install`.
535+
All pattern targets (`install-%`, `upgrade-%`, `uninstall-%`, `reconcile-%`) work for any of the 89 cataloged tools. They use a three-step fallback: dedicated script, then generic installer, then error.
526536

527537
```bash
528-
# Update existing toolchains
529-
make update-core
530-
make update-python
531-
make update-node
532-
make update-go
533-
make update-aws
534-
535-
# Uninstall
538+
# Upgrade any tool
539+
make upgrade-fd
540+
make upgrade-python
541+
make upgrade-node
542+
543+
# Uninstall any tool
536544
make uninstall-node
545+
make uninstall-semgrep
537546

538547
# Reconcile preferred method
539548
# Example: remove distro Node and switch to nvm-managed

catalog/README.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,11 @@ Install tools via system package managers (apt, brew, dnf, pacman).
6868
## Adding a New Tool
6969

7070
1. Create `catalog/<tool>.json` with appropriate metadata
71-
2. The tool will automatically use the generic installer for its method
71+
2. The tool will automatically be available via `make install-<tool>`, `make upgrade-<tool>`, `make uninstall-<tool>`, and `make reconcile-<tool>`
7272
3. No need to create a custom install script!
7373

74+
Currently **89 tools** are cataloged.
75+
7476
## Environment Variables
7577

7678
- `INSTALL_STRATEGY`: Where to install tools (USER, GLOBAL, CURRENT, PROJECT)
@@ -92,14 +94,8 @@ INSTALL_STRATEGY=GLOBAL scripts/install_tool.sh terraform
9294
make upgrade
9395
```
9496

95-
## Migration Status
96-
97-
Tools with catalog entries use the new system. Tools without catalog entries fall back to legacy `install_core.sh`.
97+
## Architecture
9898

99-
**Migrated:**
100-
- kubectl
101-
- terraform
102-
- aws
103-
- semgrep
99+
All 89 tools have catalog entries. The generic installer (`scripts/install_tool.sh`) reads a tool's catalog JSON and delegates to the appropriate method-specific installer under `scripts/installers/`. Tools with complex installation needs (python, node, docker, rust, etc.) use `install_method: "dedicated_script"` to route to their existing bespoke scripts.
104100

105-
**To migrate:** Add catalog entries for remaining tools from `install_core.sh`
101+
See [ADR-007](../docs/adr/ADR-007-generic-tool-installation-architecture.md) for the full architectural decision record.
Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# ADR-007: Generic Tool Installation Architecture
2+
3+
**Status:** Accepted
4+
**Date:** 2026-02-25
5+
**Deciders:** AI CLI Preparation Team
6+
**Tags:** installation, catalog, generic-installer, architecture
7+
8+
## Context
9+
10+
The project originally required a dedicated shell script (`scripts/install_<tool>.sh`) for each tool it managed. As the catalog grew to 89+ tools, this approach did not scale:
11+
12+
- Each new tool required writing a bespoke install script, even for tools that follow identical installation patterns (e.g., downloading a GitHub release binary, or `uv tool install <package>`).
13+
- Only ~15 tools had dedicated scripts. The remaining 75+ catalog-only tools could not be installed, upgraded, or uninstalled via Make targets.
14+
- Duplicated logic across scripts (downloading, extracting, verifying, installing to PATH) was difficult to maintain.
15+
16+
**Problem:** How do we support install/upgrade/uninstall for all 89+ cataloged tools without writing and maintaining a dedicated script for each one?
17+
18+
## Decision
19+
20+
Implement a **three-tier generic installation architecture**:
21+
22+
### Tier 1: Catalog JSON entries
23+
24+
Each tool is described by a `catalog/<tool>.json` file containing metadata and installation parameters:
25+
26+
```json
27+
{
28+
"name": "shfmt",
29+
"install_method": "github_release_binary",
30+
"binary_name": "shfmt",
31+
"github_repo": "mvdan/sh",
32+
"download_url_template": "https://github.com/mvdan/sh/releases/download/{version}/shfmt_{version}_linux_{arch}",
33+
"arch_map": {
34+
"x86_64": "amd64",
35+
"aarch64": "arm64",
36+
"armv7l": "arm"
37+
}
38+
}
39+
```
40+
41+
The `install_method` field determines which generic installer handles the tool.
42+
43+
### Tier 2: Method-specific installers
44+
45+
Generic installer scripts under `scripts/installers/` implement each installation method:
46+
47+
| Installer | Method | Example tools |
48+
|-----------|--------|---------------|
49+
| `github_release_binary.sh` | Download binary from GitHub releases | fd, ripgrep, bat, delta |
50+
| `hashicorp_zip.sh` | Download from releases.hashicorp.com | terraform |
51+
| `aws_installer.sh` | AWS official installer | aws |
52+
| `uv_tool.sh` | `uv tool install` for Python packages | semgrep, ruff, black |
53+
| `npm_global.sh` | `npm install -g` | eslint, prettier |
54+
| `package_manager.sh` | System package managers (apt/brew/dnf) | sponge, pipx |
55+
| `docker_plugin.sh` | Docker plugins | compose |
56+
| `github_clone.sh` | Clone and build from source | rbenv, ruby-build |
57+
| `npm_self_update.sh` | NPM self-update | npm |
58+
| `dedicated_script.sh` | Delegate to a tool-specific script | python, node, docker |
59+
| *(planned)* `go_install.sh` | Install via `go install` | templ |
60+
61+
**Note on `auto` install method:** 11 tools (including fd, ripgrep, bat, and hyperfine) use `"install_method": "auto"` instead of specifying a fixed installer. The `auto` method uses the reconciliation system (`scripts/lib/reconcile.sh`) to detect existing installations and choose the best available method from the tool's `available_methods` list in the catalog. This allows the system to adapt to the user's environment (e.g., preferring a GitHub binary release over cargo over apt, depending on what is already installed and available).
62+
63+
### Tier 3: Orchestrator
64+
65+
`scripts/install_tool.sh` is the central orchestrator that:
66+
67+
1. Reads `catalog/<tool>.json` to determine the install method
68+
2. Validates the catalog entry has required fields
69+
3. Delegates to the appropriate method-specific installer in `scripts/installers/`
70+
4. Handles universal concerns (uninstall detection, status reporting)
71+
72+
For tools with `install_method: "auto"`, the orchestrator uses the reconciliation system (`scripts/lib/reconcile.sh`) to detect existing installations and choose the best approach.
73+
74+
## Rationale
75+
76+
### Why a catalog-driven approach?
77+
78+
- **Declarative over imperative**: Tool metadata is data, not code. Adding a tool is a JSON file, not a shell script.
79+
- **Consistent behavior**: All tools using the same method share the same installer logic (download, extract, verify, install).
80+
- **Testable**: Catalog entries can be validated programmatically. Installers can be tested independently of individual tools.
81+
82+
### Why keep dedicated scripts?
83+
84+
Some tools have genuinely complex installation requirements that cannot be captured in a simple JSON entry:
85+
86+
- **Python** (`install_python.sh`): Manages uv installation, Python version selection, virtual environments
87+
- **Node** (`install_node.sh`): Manages nvm, node version selection, corepack
88+
- **Docker** (`install_docker.sh`): Repository setup, GPG keys, daemon configuration
89+
- **Rust** (`install_rust.sh`): Rustup installer, toolchain management
90+
91+
These tools use `install_method: "dedicated_script"` in their catalog entry, which routes to the existing script.
92+
93+
## Consequences
94+
95+
### Positive
96+
97+
- **Scalability**: Adding a new tool only requires creating a `catalog/<tool>.json` file
98+
- **Consistency**: All tools of the same method type behave identically
99+
- **Reduced maintenance**: Bug fixes in a method installer benefit all tools using that method
100+
- **Complete coverage**: All 89 cataloged tools now support install, upgrade, uninstall, and reconcile operations
101+
102+
### Negative
103+
104+
- **Abstraction cost**: Some tools may need special handling that does not fit neatly into a generic installer
105+
- **jq dependency**: The orchestrator requires `jq` to parse catalog JSON files
106+
107+
### Neutral
108+
109+
- **Coexistence**: Both dedicated scripts and generic installers coexist; the Makefile pattern targets handle the fallback transparently
110+
- **Migration path**: Tools can start with a dedicated script and migrate to catalog-only as patterns stabilize, or vice versa
111+
112+
## Implementation Notes
113+
114+
### Directory structure
115+
116+
```
117+
catalog/
118+
fd.json
119+
ripgrep.json
120+
semgrep.json
121+
... # 89 tool definitions
122+
scripts/
123+
install_tool.sh # Orchestrator
124+
install_python.sh # Dedicated script (complex tools)
125+
install_node.sh
126+
...
127+
installers/
128+
github_release_binary.sh
129+
hashicorp_zip.sh
130+
uv_tool.sh
131+
... # 10 method-specific installers
132+
```
133+
134+
### Adding a new tool
135+
136+
1. Create `catalog/<tool>.json` with `name`, `install_method`, and method-specific fields
137+
2. Optionally add `tags` for group installation (`make install-core`, etc.)
138+
3. The tool is immediately available via `make install-<tool>`, `make upgrade-<tool>`, etc.
139+
140+
## References
141+
142+
- **[ADR-008](ADR-008-makefile-pattern-target-fallback.md)** - Makefile pattern target fallback chain
143+
- **[ADR-001](ADR-001-context-aware-installation.md)** - Context-aware installation modes
144+
- **[ADR-002](ADR-002-package-manager-hierarchy.md)** - Package manager preference hierarchy
145+
- **[catalog/README.md](../../catalog/README.md)** - Catalog documentation
146+
147+
---
148+
149+
**Revision History:**
150+
151+
| Version | Date | Changes |
152+
|---------|------|---------|
153+
| 1.0 | 2026-02-25 | Initial decision accepted |

0 commit comments

Comments
 (0)