This guide covers operational aspects of AI CLI Preparation: Makefile targets, installation scripts, snapshot workflows, offline mode configuration, environment variables, CI/CD integration, and performance tuning.
# Render audit from snapshot (no network, fast)
make audit
# Update snapshot with latest data (verbose)
make update
# Auto-update if snapshot missing, then render
make audit-auto
# Run interactive upgrade guide
make upgradeRender audit table from snapshot (render-only mode, no network).
Environment:
CLI_AUDIT_RENDER=1- Render-only modeCLI_AUDIT_GROUP=0- No category groupingCLI_AUDIT_LINKS=1- Enable hyperlinksCLI_AUDIT_EMOJI=1- Use emoji status icons
Use Case: Fast local audit from cached snapshot, safe to run repeatedly.
Speed: <100ms (pure JSON read + format)
make auditOffline audit with cache-only mode (no upstream queries).
Environment:
- All settings from
make audit CLI_AUDIT_OFFLINE=1- Force offline mode
Use Case: Disconnected environments, air-gapped systems, CI cache validation.
make audit-offlineConditionally update snapshot if missing/stale, then render.
Logic:
- Check if
tools_snapshot.jsonexists - If missing: Run
CLI_AUDIT_COLLECT=1to fetch data - Render audit table from snapshot
Use Case: First-run scenarios, post-checkout, automated pipelines.
make audit-autoCollect-only mode: fetch upstream data, write snapshot, no output rendering.
Environment:
CLI_AUDIT_COLLECT=1- Collect-only modeCLI_AUDIT_DEBUG=1- Debug messagesCLI_AUDIT_PROGRESS=1- Progress updates
Use Case: Refresh snapshot with latest data, troubleshoot network issues.
Output: Writes tools_snapshot.json, prints verbose progress to stderr.
make updateInteractive remediation guide (renamed from guide).
Behavior:
- Runs full audit
- Identifies outdated/missing tools
- Prompts for installation/update actions
- Executes selected remediation steps
Use Case: One-stop tool for bringing system up to date.
make upgradeComplete system upgrade in 5 orchestrated stages.
Workflow:
- Stage 1: Refresh Data - Update version snapshot from upstream sources
- Stage 2: Upgrade Package Managers - Self-update system package managers (apt, brew, snap, flatpak)
- Stage 3: Upgrade Language Runtimes - Update core runtimes (Python, Node.js, Go, Ruby, Rust)
- Stage 4: Upgrade User Package Managers - Update language-specific managers (uv, pipx, npm, pnpm, yarn, cargo, composer, poetry)
- Stage 5: Upgrade Tools - Upgrade all CLI tools managed by each package manager
Features:
- Comprehensive logging to
logs/upgrade-YYYYMMDD-HHMMSS.log - Colored terminal output with progress tracking
- Version and location info for successful upgrades
- UV migration support (auto-migrates pip/pipx packages to uv tools)
- System package detection (skips system-managed tools, suggests reconcile)
- Statistics summary (upgraded/failed/skipped counts)
- Dry-run mode available
Use Case: Complete development environment upgrade, ensuring all tools across all package managers are current.
Duration: 5-15 minutes depending on number of outdated packages.
# Full system upgrade
make upgrade-all
# Preview what would be upgraded (dry-run)
make upgrade-all-dry-runOutput Example:
[1/5] Stage 1: Refresh version data
✓ Updated snapshot (64 tools checked)
[2/5] Stage 2: Upgrade package managers
✓ apt (upgraded 12 packages)
⏭ homebrew (not installed)
✓ snap (2.63 at /usr/bin/snap)
[3/5] Stage 3: Upgrade language runtimes
✓ python (3.12.7 → 3.12.8)
✓ node (20.10.0 → 20.11.0)
⏭ go (already latest: 1.22.0)
[4/5] Stage 4: Upgrade user package managers
✓ uv (0.4.30 → 0.5.0)
✓ pipx (1.7.1 → 1.8.0)
→ Migrating pip packages to uv...
✓ black migrated to uv tool
✓ ruff migrated to uv tool
[5/5] Stage 5: Upgrade CLI tools
✓ Upgraded 8 uv tools
✓ Upgraded 15 npm packages
✓ Upgraded 3 cargo binaries
Summary: 45 upgraded, 12 skipped, 2 failed
Duration: 8m 32s
Log: logs/upgrade-20251018-073045.log
Environment Variables:
DRY_RUN=1- Preview mode without making changesSKIP_SYSTEM=1- Skip system package managers (apt, brew, snap, flatpak)VERBOSE=1- Detailed output for debugging
Advanced:
# Direct script execution with options
DRY_RUN=1 VERBOSE=1 bash scripts/upgrade_all.sh
# Skip system package managers
SKIP_SYSTEM=1 make upgrade-all
# Check PATH configuration before upgrading
make check-pathValidate PATH configuration for all package managers.
Behavior:
- Checks if package manager binaries are in PATH
- Validates PATH ordering (user bins before system bins)
- Identifies potential shadowing issues
- Provides remediation suggestions
Use Case: Diagnose PATH issues before or after system upgrade.
make check-pathAudit single tool from snapshot.
# Examples
make audit-python
make audit-ripgrep
make audit-dockerAudit category subset in offline mode.
Available Categories:
agent-core- Core tools for AI agentspython-core- Python runtime and package managersnode-core- Node.js runtime and package managersgo-core- Go runtime and toolsinfra-core- Cloud/infrastructure toolssecurity-core- Security scanning toolsdata-core- Data processing tools
# Examples
make audit-offline-python-core
make audit-offline-security-core
make audit-offline-infra-core# Set script permissions (run once)
make scripts-perms
# Install core tools (fd, fzf, ripgrep, jq, yq, bat, delta, just)
make install-core
# Install language stacks
make install-python # pip, pipx, poetry, uv
make install-node # nvm, npm, pnpm, yarn
make install-go # Go toolchain
make install-rust # Rust via rustup
# Install cloud/infra tools
make install-aws # AWS CLI v2
make install-kubectl # Kubernetes CLI
make install-terraform # Terraform
make install-ansible # Ansible
make install-docker # Docker Engine
make install-brew # Homebrew (Linux/macOS)All install scripts support update action:
make update-core
make update-python
make update-node
make update-go
make update-aws
make update-rustmake uninstall-node
make uninstall-rust
make uninstall-dockerSwitch from system package to preferred method:
# Example: Remove distro Node, switch to nvm-managed
make reconcile-node
# Example: Remove distro Rust, switch to rustup
make reconcile-rustRun basic lint checks with pyflakes (optional).
make lintPlaceholder for formatting (no-op currently).
make fmtDisplay available targets and descriptions.
make helpAll scripts under scripts/ follow consistent patterns:
./scripts/install_COMPONENT.sh [ACTION] [TOOL]Actions:
install(default) - Install new toolsupdate- Update existing toolsuninstall- Remove toolsreconcile- Switch installation methods
Tool: Optional, install/update specific tool only
What it does:
- Detects OS and architecture
- Downloads latest upstream releases
- Installs to
~/.local/binor/usr/local/bin - Updates PATH if needed
- Verifies installation
Example:
./scripts/install_core.sh install
# Installs all core tools
./scripts/install_core.sh install fd
# Installs only fdWhat it does:
- Checks current installed version
- Fetches latest upstream version
- Skips if already up-to-date
- Downloads and replaces if outdated
- Preserves configuration
Example:
./scripts/install_python.sh update
# Updates pip, pipx, poetry, uv if outdatedWhat it does:
- Removes binaries from install directories
- Optionally removes config/data directories
- Cleans up PATH modifications
- Preserves user data by default
Example:
./scripts/install_node.sh uninstall
# Removes nvm, node, npm, pnpm, yarnWhat it does:
- Identifies installation method conflicts
- Removes system/distro packages
- Installs via preferred method
- Validates final state
Example:
./scripts/install_rust.sh reconcile
# Removes apt-installed rust, installs via rustupThe tool separates data collection (network) from rendering (local) for efficiency and offline support.
Full audit: collect + render in single command.
python3 cli_audit.pyFetch data, write snapshot, no output.
CLI_AUDIT_COLLECT=1 python3 cli_audit.pyOutput: Writes tools_snapshot.json
Use Case: CI cache updates, scheduled data refresh, troubleshooting.
Read snapshot, format output, no network.
CLI_AUDIT_RENDER=1 python3 cli_audit.pyUse Case: Repeated local checks, offline environments, fast queries.
Default Path: tools_snapshot.json (override with CLI_AUDIT_SNAPSHOT_FILE)
Schema:
{
"__meta__": {
"schema_version": 1,
"created_at": "2025-10-09T12:34:56Z",
"offline": false,
"count": 50,
"partial_failures": 2
},
"tools": [
{
"tool": "ripgrep",
"installed": "14.1.1 (150ms)",
"installed_version": "14.1.1",
"installed_method": "rustup/cargo",
"installed_path_resolved": "/home/user/.cargo/bin/rg",
"installed_path_selected": "/home/user/.cargo/bin/rg",
"classification_reason": "path-under-~/.cargo/bin",
"classification_reason_selected": "path-under-~/.cargo/bin",
"latest_upstream": "14.1.1 (220ms)",
"latest_version": "14.1.1",
"upstream_method": "github",
"status": "UP-TO-DATE",
"tool_url": "https://github.com/BurntSushi/ripgrep",
"latest_url": "https://github.com/BurntSushi/ripgrep/releases/tag/14.1.1"
}
]
}Run periodically or when tools change.
make updateUse audit-auto target to update only when missing.
make audit-autoUpdate snapshot nightly for fresh data.
# Crontab entry
0 2 * * * cd /path/to/project && make updateStore snapshot as CI artifact, update weekly.
# GitHub Actions example
- name: Restore snapshot cache
uses: actions/cache@v3
with:
path: tools_snapshot.json
key: tools-snapshot-${{ runner.os }}-${{ github.run_number }}
restore-keys: tools-snapshot-${{ runner.os }}-
- name: Update snapshot
run: make updateEnvironment Variable:
CLI_AUDIT_OFFLINE=1 python3 cli_audit.pyMakefile:
make audit-offlineWhat Changes:
- No upstream API calls (GitHub, PyPI, crates.io, npm)
- Uses
upstream_versions.jsonexclusively for version data - Marks upstream method as
manual - Displays
(offline)in readiness summary
What Still Works:
- Local version detection
- Installation method classification
- Status comparison (installed vs cached upstream)
- Full audit table/JSON output
- Update manual cache online:
make update
# Populates upstream_versions.json with current data- Commit manual cache:
git add upstream_versions.json
git commit -m "chore: update manual version cache"- Verify offline operation:
CLI_AUDIT_OFFLINE=1 python3 cli_audit.py --only pythonData Files:
upstream_versions.json- Latest available versions (committed)local_state.json- Machine-specific state (gitignored)
Update Commands:
python audit.py --update-baseline- Refresh upstream versionspython audit.py --update-local- Refresh local state
File: .env (gitignored)
# Mode Control
CLI_AUDIT_OFFLINE=0
CLI_AUDIT_COLLECT=0
CLI_AUDIT_RENDER=0
# Output
CLI_AUDIT_JSON=0
CLI_AUDIT_LINKS=1
CLI_AUDIT_EMOJI=1
CLI_AUDIT_TIMINGS=1
CLI_AUDIT_GROUP=1
# Performance
CLI_AUDIT_MAX_WORKERS=16
CLI_AUDIT_TIMEOUT_SECONDS=3
CLI_AUDIT_FAST=0
# Network
CLI_AUDIT_HTTP_RETRIES=2
CLI_AUDIT_BACKOFF_BASE=0.2
CLI_AUDIT_BACKOFF_JITTER=0.1
GITHUB_TOKEN=ghp_your_token_here
# Debugging (disable in production)
CLI_AUDIT_DEBUG=0
CLI_AUDIT_TRACE=0
CLI_AUDIT_TRACE_NET=0
CLI_AUDIT_PROGRESS=0
# Paths
CLI_AUDIT_SNAPSHOT_FILE=tools_snapshot.json
CLI_AUDIT_MANUAL_FILE=upstream_versions.json
# Cache
CLI_AUDIT_WRITE_MANUAL=1
CLI_AUDIT_MANUAL_FIRST=0File: .env.development
CLI_AUDIT_DEBUG=1
CLI_AUDIT_PROGRESS=1
CLI_AUDIT_TRACE=1
CLI_AUDIT_MAX_WORKERS=8
CLI_AUDIT_TIMEOUT_SECONDS=5Usage:
set -a; source .env.development; set +a
make auditFile: .env.ci
CLI_AUDIT_OFFLINE=0
CLI_AUDIT_JSON=1
CLI_AUDIT_TIMINGS=0
CLI_AUDIT_EMOJI=0
CLI_AUDIT_LINKS=0
CLI_AUDIT_MAX_WORKERS=4
CLI_AUDIT_TIMEOUT_SECONDS=10
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}File: .env.production
CLI_AUDIT_RENDER=1
CLI_AUDIT_OFFLINE=1
CLI_AUDIT_JSON=1
CLI_AUDIT_DEBUG=0
CLI_AUDIT_FAST=1name: Tool Audit
on:
schedule:
- cron: '0 8 * * 1' # Weekly on Monday
workflow_dispatch:
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Run audit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CLI_AUDIT_JSON: 1
run: |
python3 cli_audit.py > audit_results.json
- name: Check for outdated tools
run: |
jq -r '.[] | select(.status == "OUTDATED") | "\(.tool): \(.installed_version) → \(.latest_version)"' audit_results.json
- name: Upload results
uses: actions/upload-artifact@v3
with:
name: audit-results
path: audit_results.jsonname: Tool Audit with Cache
on:
push:
branches: [main]
pull_request:
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Restore snapshot cache
id: cache-snapshot
uses: actions/cache@v3
with:
path: tools_snapshot.json
key: tools-snapshot-${{ runner.os }}-${{ hashFiles('upstream_versions.json') }}
restore-keys: |
tools-snapshot-${{ runner.os }}-
- name: Update snapshot if cache miss
if: steps.cache-snapshot.outputs.cache-hit != 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: make update
- name: Render audit
run: make audit
- name: Save snapshot
uses: actions/cache/save@v3
if: always()
with:
path: tools_snapshot.json
key: tools-snapshot-${{ runner.os }}-${{ hashFiles('upstream_versions.json') }}name: Audit PR Comment
on:
pull_request:
types: [opened, synchronize]
jobs:
comment:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run audit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CLI_AUDIT_JSON: 1
run: python3 cli_audit.py > audit.json
- name: Generate summary
id: summary
run: |
echo "outdated=$(jq -r '[.[] | select(.status == "OUTDATED")] | length' audit.json)" >> $GITHUB_OUTPUT
echo "missing=$(jq -r '[.[] | select(.status == "NOT INSTALLED")] | length' audit.json)" >> $GITHUB_OUTPUT
- name: Comment PR
uses: actions/github-script@v6
with:
script: |
const outdated = ${{ steps.summary.outputs.outdated }};
const missing = ${{ steps.summary.outputs.missing }};
const body = `## Tool Audit Results\n\n- 🔄 Outdated: ${outdated}\n- ❌ Missing: ${missing}\n\nRun \`make upgrade\` to remediate.`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: body
})tool_audit:
stage: test
image: python:3.11
script:
- python3 cli_audit.py --only agent-core | python3 smart_column.py -s "|" -t
only:
- main
- merge_requests
tool_audit_json:
stage: test
image: python:3.11
script:
- CLI_AUDIT_JSON=1 python3 cli_audit.py > audit.json
- jq '.[] | select(.status != "UP-TO-DATE")' audit.json
artifacts:
paths:
- audit.json
expire_in: 1 week
only:
- schedulespipeline {
agent any
environment {
GITHUB_TOKEN = credentials('github-token')
}
stages {
stage('Audit Tools') {
steps {
sh '''
CLI_AUDIT_JSON=1 python3 cli_audit.py > audit.json
jq -r '.[] | select(.status == "OUTDATED") | .tool' audit.json > outdated.txt
'''
archiveArtifacts artifacts: 'audit.json,outdated.txt', fingerprint: true
}
}
stage('Report') {
steps {
script {
def outdated = readFile('outdated.txt').trim()
if (outdated) {
currentBuild.result = 'UNSTABLE'
echo "Outdated tools: ${outdated}"
}
}
}
}
}
}# Measure collection time
time make update
# Measure render time
time make audit
# Identify slow tools
CLI_AUDIT_TRACE=1 CLI_AUDIT_SLOW_MS=1000 make update 2>&1 | grep "slow"Default: 16 workers
# For powerful machines
CLI_AUDIT_MAX_WORKERS=32 make update
# For resource-constrained environments
CLI_AUDIT_MAX_WORKERS=4 make updateRecommendation: 16-20 workers optimal, diminishing returns above.
Default: 3 seconds per operation
# Faster but may miss slow tools
CLI_AUDIT_TIMEOUT_SECONDS=1 make update
# More patient for slow networks
CLI_AUDIT_TIMEOUT_SECONDS=10 make updateDefault: 2 retries with exponential backoff
# Aggressive (faster failure)
CLI_AUDIT_HTTP_RETRIES=1 CLI_AUDIT_BACKOFF_BASE=0.1 make update
# Conservative (better success rate)
CLI_AUDIT_HTTP_RETRIES=5 CLI_AUDIT_BACKOFF_BASE=0.5 make updateTry cache before network (cache-first strategy):
CLI_AUDIT_MANUAL_FIRST=1 make updateEffect: Reduces API calls, faster when cache is current.
Skip expensive operations:
CLI_AUDIT_FAST=1 make updateSkips:
- Docker image inspection (if enabled)
- Slow upstream APIs (GNU FTP)
- Deep path searches
Prevent rate limiting by limiting per-host requests:
# Default: 4 concurrent GitHub API requests
CLI_AUDIT_HOST_CAP_GITHUB_API=2 make update
# Default: 4 concurrent npm registry requests
CLI_AUDIT_HOST_CAP_NPM=8 make updateAvailable Caps:
CLI_AUDIT_HOST_CAP_GITHUB- github.com (releases)CLI_AUDIT_HOST_CAP_GITHUB_API- api.github.comCLI_AUDIT_HOST_CAP_NPM- registry.npmjs.orgCLI_AUDIT_HOST_CAP_CRATES- crates.ioCLI_AUDIT_HOST_CAP_GNU- GNU FTP mirrors
Fastest: Render from snapshot (no collection)
make audit # <100msUse Case: Repeated local checks, CI status gates.
| Scenario | Time | Workers | Network |
|---|---|---|---|
| Full audit (online) | ~10s | 16 | Yes |
| Full audit (cache-first) | ~5s | 16 | Partial |
| Full audit (offline) | ~3s | 16 | No |
| Render-only | <100ms | - | No |
| Single tool audit | ~300ms | 1 | Yes |
-
GitHub API Rate Limiting: 60 req/hour without token
- Fix: Set
GITHUB_TOKENenvironment variable - Effect: 5000 req/hour authenticated
- Fix: Set
-
Network Latency: Upstream APIs may be slow
- Fix: Use manual-first mode, increase timeout
- Effect: Better success rate, slightly slower
-
Subprocess Execution: 50+ version checks add up
- Fix: Increase workers, reduce timeout
- Effect: Faster completion, potential misses
-
Docker Inspection: Slow if many images
- Fix:
CLI_AUDIT_DOCKER_INFO=0 - Effect: Skip docker image details
- Fix:
High-Reliability (CI/CD):
CLI_AUDIT_MAX_WORKERS=8
CLI_AUDIT_TIMEOUT_SECONDS=10
CLI_AUDIT_HTTP_RETRIES=5
GITHUB_TOKEN=<token>High-Performance (Local Dev):
CLI_AUDIT_MAX_WORKERS=20
CLI_AUDIT_TIMEOUT_SECONDS=2
CLI_AUDIT_HTTP_RETRIES=1
CLI_AUDIT_MANUAL_FIRST=1Offline/Air-Gapped:
CLI_AUDIT_OFFLINE=1
CLI_AUDIT_RENDER=1
CLI_AUDIT_MANUAL_FIRST=1#!/bin/bash
# health_check.sh
set -euo pipefail
OUTDATED_COUNT=$(CLI_AUDIT_JSON=1 python3 cli_audit.py | jq '[.[] | select(.status == "OUTDATED")] | length')
MISSING_COUNT=$(CLI_AUDIT_JSON=1 python3 cli_audit.py | jq '[.[] | select(.status == "NOT INSTALLED")] | length')
if [ "$OUTDATED_COUNT" -gt 5 ] || [ "$MISSING_COUNT" -gt 2 ]; then
echo "WARN: $OUTDATED_COUNT outdated, $MISSING_COUNT missing tools"
exit 1
fi
echo "OK: System up to date"
exit 0#!/bin/bash
# metrics.sh
set -euo pipefail
CLI_AUDIT_JSON=1 python3 cli_audit.py > audit.json
cat <<EOF
# HELP cli_audit_tools_total Total number of tools audited
# TYPE cli_audit_tools_total gauge
cli_audit_tools_total $(jq 'length' audit.json)
# HELP cli_audit_outdated_total Number of outdated tools
# TYPE cli_audit_outdated_total gauge
cli_audit_outdated_total $(jq '[.[] | select(.status == "OUTDATED")] | length' audit.json)
# HELP cli_audit_missing_total Number of missing tools
# TYPE cli_audit_missing_total gauge
cli_audit_missing_total $(jq '[.[] | select(.status == "NOT INSTALLED")] | length' audit.json)
EOFSee TROUBLESHOOTING.md for detailed debugging information.
- ARCHITECTURE.md - System design and data flow
- API_REFERENCE.md - Function signatures and environment variables
- DEVELOPER_GUIDE.md - Contributing and development
- TROUBLESHOOTING.md - Common issues and solutions
- TOOL_ECOSYSTEM.md - Tool catalog and classifications