Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
1d8eae8
Initial plan
Copilot May 6, 2026
5ffd5c7
Audit and fix CLI flag bindings, add config gen command, regenerate e…
Copilot May 6, 2026
494d49e
docs: add config template generation guide to configuration documenta…
frjcomp May 6, 2026
985322f
test: add comprehensive configuration priority chain and coverage tests
frjcomp May 6, 2026
b93c6db
fix: extract CLI flag bindings to package-level vars and add enforcem…
frjcomp May 6, 2026
625b001
test: enforce flag bindings across remaining commands
frjcomp May 6, 2026
bd137b9
feat: simplify CLI flag handling with CommandSetup helpers
frjcomp May 6, 2026
1930c44
refactor: migrate all commands to CommandSetup builder pattern
frjcomp May 6, 2026
b835a9a
Fix config generation review feedback
frjcomp May 6, 2026
65870f5
fix: validate only leaf config paths, not intermediate containers
frjcomp May 7, 2026
1255667
docs: update config documentation with --output flag and config get/s…
frjcomp May 7, 2026
b987650
fix: prevent log output before validation errors in config commands
frjcomp May 7, 2026
9af3e3c
fix: allow section paths for config get
frjcomp May 7, 2026
bc738fa
Normalize URL flags and GitLab project/group terminology
frjcomp May 7, 2026
d5d1fea
Fix e2e regressions after flag refactor
frjcomp May 7, 2026
57c798f
fix: update tests for url flag rename and fix gosec findings
frjcomp May 7, 2026
1312cf4
test: reuse scan flagBindings in env binding tests
frjcomp May 11, 2026
ee11f3e
build: add pre-release guard automation
frjcomp May 11, 2026
e10f09d
build: remove pre-release guard (temporary validation only)
frjcomp May 11, 2026
fe015e1
fix: normalize config key handling
frjcomp May 11, 2026
1648a4d
test: add shared flag binding coverage helper
frjcomp May 11, 2026
ac5b5fb
chore: drop committed example config file
frjcomp May 11, 2026
208bf0b
refactor: remove low-value narration comments from source files
frjcomp May 11, 2026
389f6b4
fix: restrict ValidateURL to http/https, fix WithAutoBindings auto-de…
frjcomp May 11, 2026
657fe82
fix: update e2e test for new ValidateURL error message
frjcomp May 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ make serve-docs # Installs dependencies if needed, generates and serves docs
- Each command should have a corresponding test file
- Commands are organized by platform (gitlab, github, bitbucket, devops, gitea)
- Use consistent flag naming across commands
- **When adding or modifying command flags**: Update both `docs/introduction/configuration.md` and `pipeleek.example.yaml` to reflect the changes
- **When adding or modifying command flags**: Update `docs/introduction/configuration.md` and ensure `pipeleek config gen` output remains accurate

### Configuration Loading Pattern (MANDATORY)

Expand Down
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help build build-all build-gitlab build-github build-bitbucket build-devops build-gitea build-circle test test-unit test-e2e lint clean coverage coverage-html serve-docs
.PHONY: help build build-all build-gitlab build-github build-bitbucket build-devops build-gitea build-circle test test-unit test-e2e lint clean coverage coverage-html serve-docs release-guard

# Default target
help:
Expand All @@ -18,6 +18,7 @@ help:
@echo " make test-e2e - Run e2e tests (builds binary first)"
@echo " make coverage - Generate test coverage report"
@echo " make coverage-html - Generate and open HTML coverage report"
@echo " make release-guard - Compare against latest release and run pre-release safety checks"
@echo " make lint - Run golangci-lint"
@echo " make serve-docs - Generate and serve CLI documentation"
@echo " make clean - Remove built artifacts"
Expand Down Expand Up @@ -126,6 +127,13 @@ coverage-html: coverage
echo "Open coverage.html in your browser to view the report"; \
fi

# Compare current branch against latest release and run release-safety checks
# Set STRICT_ALLOWLIST=1 to fail if changed files fall outside ALLOWLIST_REGEX.
# Set FAST_MODE=1 to skip gosec and golangci-lint for faster iteration.
release-guard:
@echo "Running pre-release guard..."
./scripts/pre_release_guard.sh

# Run golangci-lint
lint:
@echo "Running golangci-lint..."
Expand Down
8 changes: 4 additions & 4 deletions docs/guides/gitlab.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ There are many reasons why credentials might be included in the job output. More
[Pipeleek](https://github.com/CompassSecurity/pipeleek) can be used to scan for credentials in the job outputs.

```bash
$ pipeleek gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com -c [gitlab session cookie]] -v -a -j 5 --confidence high-verified,high
$ pipeleek gl scan --token glpat-[redacted] --url https://gitlab.example.com -c [gitlab session cookie]] -v -a -j 5 --confidence high-verified,high
2024-09-26T13:47:09+02:00 debug Verbose log output enabled
2024-09-26T13:47:10+02:00 info Gitlab Version Check revision=2e166256199 version=17.5.0-pre
2024-09-26T13:47:10+02:00 debug Setting up queue on disk
Expand Down Expand Up @@ -236,7 +236,7 @@ Runners can be attached globally, on the group level or on individual projects.
Using pipeleek we can automate runner enumeration:

```bash
$ pipeleek gl runners --token glpat-[redacted] --gitlab https://gitlab.example.com -v list
$ pipeleek gl runners --token glpat-[redacted] --url https://gitlab.example.com -v list
2024-09-26T14:26:54+02:00 info group runner description=2-green.shared-gitlab-org.runners-manager.gitlab.example.com name=comp-test-ia paused=false runner=gitlab-runner tags=gitlab-org type=instance_type
2024-09-26T14:26:55+02:00 info group runner description=3-green.shared-gitlab-org.runners-manager.gitlab.example.com/dind name=comp-test-ia paused=false runner=gitlab-runner tags=gitlab-org-docker type=instance_type
2024-09-26T14:26:55+02:00 info group runner description=blue-3.saas-linux-large-amd64.runners-manager.gitlab.example.com/default name=comp-test-ia paused=false runner=gitlab-runner tags=saas-linux-large-amd64 type=instance_type
Expand All @@ -250,7 +250,7 @@ Pipeleek can generate a `.gitlab-ci.yml` or directly create a project and launch

```bash
# Manual creation
$ pipeleek gl runners --token glpat-[redacted] --gitlab https://gitlab.example.com -v exploit --tags saas-linux-small-amd64 --shell --dry
$ pipeleek gl runners --token glpat-[redacted] --url https://gitlab.example.com -v exploit --tags saas-linux-small-amd64 --shell --dry
2024-09-26T14:32:26+02:00 debug Verbose log output enabled
2024-09-26T14:32:26+02:00 info Generated .gitlab-ci.yml
2024-09-26T14:32:26+02:00 info ---
Expand All @@ -276,7 +276,7 @@ pipeleek-job-saas-linux-small-amd64:
2024-09-26T14:32:26+02:00 info Done, Bye Bye 🏳️‍🌈🔥

# Automated
$ pipeleek gl runners --token glpat-[redacted] --gitlab https://gitlab.example.com -v exploit --tags saas-linux-small-amd64 --shell
$ pipeleek gl runners --token glpat-[redacted] --url https://gitlab.example.com -v exploit --tags saas-linux-small-amd64 --shell
2024-09-26T14:33:48+02:00 debug Verbose log output enabled
2024-09-26T14:33:49+02:00 info Created project name=pipeleek-runner-exploit url=https://gitlab.example.com/[redacted]/pipeleek-runner-exploit
2024-09-26T14:33:50+02:00 info Created .gitlab-ci.yml file=.gitlab-ci.yml
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/renovate.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ Your goal is to abuse the Renovate bot's access level to merge a malicious `gitl
Using Pipeleek, you can monitor your repository for new Renovate branches. When a new one is detected, Pipeleek tries to add a new job into the `gitlab-ci.yml`. As this needs to exploit a race condition (adding new changes to the Renovate branch before the bot activates auto-merge), this might take a few attempts.

```bash
pipeleek gl renovate privesc -g https://gitlab.com -t glpat-[redacted] --repo-name company1/a-software-project --renovate-branches-regex 'renovate/.*' -v
pipeleek gl renovate privesc -g https://gitlab.com -t glpat-[redacted] --project company1/a-software-project --renovate-branches-regex 'renovate/.*' -v
2025-09-30T07:56:57Z debug Verbose log output enabled
2025-09-30T07:56:57Z info Ensure the Renovate bot does have a greater access level than you, otherwise this will not work, and is able to auto merge into the protected main branch
2025-09-30T07:56:58Z debug Testing push access level for default branch branch=main requiredAccessLevel=40 userAccessLevel=30
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/scanning.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ As shown, Pipeleek can detect secrets in job logs and build artifacts. Security
If you find a repository that looks particularly interesting e.g. `secret-pipelines`, you can scan all its job logs, not just the most recent ones:

```bash
pipeleek gl scan -g https://gitlab.com -t glpat-[redacted] --cookie [redacted] --artifacts --repo mygroup/my-secret-pipelines-project
pipeleek gl scan -g https://gitlab.com -t glpat-[redacted] --cookie [redacted] --artifacts --project mygroup/my-secret-pipelines-project
```
254 changes: 57 additions & 197 deletions docs/introduction/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,17 @@ Pipeleek can be configured via config files, environment variables, or CLI flags

## Quick Start

Create `~/.config/pipeleek/pipeleek.yaml`:
Generate a configuration template with all available options:

```bash
# Write to config file (recommended)
pipeleek config gen --output ~/.config/pipeleek/pipeleek.yaml
```


The generated template documents all settings, their defaults, CLI flags, and environment variable names for quick reference.

Then configure your needed object keys, for example:

```yaml
gitlab:
Expand All @@ -32,7 +42,7 @@ pipeleek gl scan

Configuration sources are resolved in this order (highest to lowest):

1. **CLI flags** - `--gitlab`, `--token`, etc.
1. **CLI flags** - `--url`, `--token`, etc.
2. **Environment variables** - `PIPELEEK_GITLAB_TOKEN`
3. **Config file** - `~/.config/pipeleek/pipeleek.yaml`
4. **Defaults**
Expand All @@ -52,199 +62,7 @@ Config keys follow the pattern: `<platform>.<subcommand>.<flag_name>`

Platform-level settings (like `url` and `token`) are inherited by all commands under that platform.

### GitLab

```yaml
gitlab:
url: https://gitlab.example.com # Shared across all gl commands
token: glpat-xxxxxxxxxxxxxxxxxxxx # Shared across all gl commands
cookie: "" # Optional: _gitlab_session cookie for dotenv artifacts

enum:
level: full # gl enum --level

cicd:
yaml:
project: group/project # gl cicd yaml --project

schedule: {} # gl schedule (inherits url/token)

secureFiles: {} # gl secureFiles (inherits url/token)

variables: {} # gl variables (inherits url/token)

jobToken:
exploit:
project: group/project # gl jobToken exploit --project

vuln: {} # gl vuln (inherits url/token)

runners:
list: {} # gl runners list (inherits url/token)

exploit:
tags: [docker, linux] # gl runners exploit --tags
shell: bash # gl runners exploit --shell
dry: false # gl runners exploit --dry
age_public_key: "" # gl runners exploit --age-public-key
repo_name: "" # gl runners exploit --repo-name

renovate:
enum:
owned: true # gl renovate enum --owned
member: true # gl renovate enum --member
repo: false # gl renovate enum --repo
namespace: false # gl renovate enum --namespace
search: "" # gl renovate enum --search
fast: false # gl renovate enum --fast
dump: false # gl renovate enum --dump

bots:
term: renovate # gl renovate bots --term

autodiscovery: {} # gl renovate autodiscovery (inherits url/token)

privesc: {} # gl renovate privesc (inherits url/token)

register:
username: newuser # gluna register --username
password: secret # gluna register --password
email: user@example.com # gluna register --email

shodan:
json: shodan_data.json # gluna shodan --json

scan_public:
search: "" # gluna scan --search
repo: "" # gluna scan --repo
namespace: "" # gluna scan --namespace
job_limit: 0 # gluna scan --job-limit
queue: "" # gluna scan --queue
artifacts: false # gluna scan --artifacts

scan:
threads: 10 # gl scan --threads (can override common.threads)

snippets:
scan:
project: group/project # gl snippets scan --project
namespace: group # gl snippets scan --namespace
search: "" # gl snippets scan --search
owned: false # gl snippets scan --owned
member: false # gl snippets scan --member
# Runtime scan settings come from common.*:
# common.threads, common.trufflehog_verification,
# common.confidence_filter, common.hit_timeout (duration, e.g. "120s")

tf:
output_dir: ./terraform-states # gl tf --output-dir
threads: 4 # gl tf --threads (can override common.threads)
# Note: artifacts, max_artifact_size, and owned do not apply to gl tf.
```

### GitHub

```yaml
github:
url: https://api.github.com
token: ghp_xxxxxxxxxxxxxxxxxxxx

ghtoken:
exploit:
repo: owner/repo # gh ghtoken exploit --repo

scan:
owner: myorg
repo: myrepo
```

### BitBucket

```yaml
bitbucket:
url: https://bitbucket.org
email: user@example.com
token: ATATTxxxxxx

scan:
workspace: myworkspace
repo_slug: myrepo
```

### Azure DevOps

```yaml
azure_devops:
url: https://dev.azure.com/myorg
token: ado-token

scan:
project: myproject
```

### Gitea

```yaml
gitea:
url: https://gitea.example.com
token: gitea-token

enum:
owner: myorg # gitea enum --owner

secrets:
owner: myorg # gitea secrets --owner
repo: myrepo # gitea secrets --repo

variables:
owner: myorg # gitea variables --owner
repo: myrepo # gitea variables --repo

scan:
owner: myorg # gitea scan --owner
repo: myrepo # gitea scan --repo (optional, scans all if not specified)
```

### Jenkins

```yaml
jenkins:
url: https://jenkins.example.com
username: admin
token: jenkins-api-token

scan:
folder: team-a # jenkins scan --folder (optional)
job: team-a/service-a # jenkins scan --job (optional)
max_builds: 25 # jenkins scan --max-builds
```

### CircleCI

```yaml
circle:
url: https://circleci.com
token: circleci-token

scan:
project: [my-org/my-repo] # circle scan --project (optional if org is set)
vcs: github # circle scan --vcs
org: my-org # circle scan --org (also enables org-wide discovery when project is omitted)
# --org accepts: my-org, github/my-org, circleci/my-org (required for native
# CircleCI orgs), or app URL forms like
# https://app.circleci.com/pipelines/github/my-org/my-repo
# Note: org-wide discovery requires token visibility to that org. If not,
# use explicit --project selectors instead.
branch: main # circle scan --branch
status: [success, failed] # circle scan --status
workflow: [build, deploy] # circle scan --workflow
job: [unit-tests, release] # circle scan --job
since: 2026-01-01T00:00:00Z # circle scan --since (RFC3339)
until: 2026-01-31T23:59:59Z # circle scan --until (RFC3339)
max_pipelines: 0 # circle scan --max-pipelines (0 = no limit)
tests: true # circle scan --tests
insights: true # circle scan --insights
```
To view a full example of the available keys run `pipeleek config gen`.

### Common Settings

Expand Down Expand Up @@ -306,7 +124,7 @@ pipeleek gh scan --owned # Uses GitHub config

```bash
# Use config token but different URL
pipeleek gl enum --gitlab https://gitlab-dev.company.com
pipeleek gl enum --url https://gitlab-dev.company.com

# Use config URL/token but different level
pipeleek gl enum --level minimal
Expand All @@ -326,9 +144,51 @@ gitlab:
pipeleek gl enum --token glpat-xxxxxxxxxxxxxxxxxxxx
```

## Managing Config Values

### Getting Config Values

Read configuration values from your config file:

```bash
# Get a specific value
pipeleek config get gitlab.token

# Get an entire section (returns YAML)
pipeleek config get gitlab

# Get a nested value
pipeleek config get gitlab.renovate.enum.fast

# Get all configuration
pipeleek config get
```

### Setting Config Values

Write configuration values to your config file:

```bash
# Set a string value
pipeleek config set gitlab.token "glpat-xxxxxxxxxxxxxxxxxxxx"

# Set a number
pipeleek config set common.threads 8

# Set a boolean
pipeleek config set common.trufflehog_verification false

# Set a list (YAML format)
pipeleek config set gitlab.runners.exploit.tags '[\"docker\", \"shared\"]'
```

## Full Example

See [`pipeleek.example.yaml`](https://github.com/CompassSecurity/pipeleek/blob/main/pipeleek.example.yaml) for a complete example with all platforms and commands documented.
Generate a complete example with all platforms and commands documented by running:

```bash
pipeleek config gen
```

## Troubleshooting

Expand Down
4 changes: 2 additions & 2 deletions docs/introduction/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ Pipeleek also provides platform-specific binaries that include only the commands
The most basic example to scan e.g. GitLab pipeline logs for secrets.

```bash
pipeleek gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com
pipeleek gl scan --token glpat-[redacted] --url https://gitlab.example.com
```

### Scanning Artifacts
Expand All @@ -142,5 +142,5 @@ In addition to logs, Pipeleek can also scan artifacts generated by pipelines.
> **💡Tip:** All `scan` commands must be configured to scan artifacts. This feature is disabled by default.

```bash
pipeleek gl scan --token glpat-[redacted] --gitlab https://gitlab.example.com --artifacts
pipeleek gl scan --token glpat-[redacted] --url https://gitlab.example.com --artifacts
```
Loading
Loading