Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,9 @@
- [x] Slice 17: Align test method naming
- [x] Slice 18: Fail evaluation when rehydration errors occur (instead of silently swallowing them)
- [x] Slice 19: Add Long descriptions, Example blocks, and docs feedback (policy contract hint, snyk trail example)

## kosli evaluate input

- [x] Slice 1: `evaluate input --input-file` with a file path
- [x] Slice 2: stdin support (omit --input-file to read stdin; `-` not supported by cobra)
- [x] Slice 3: help text and examples
9 changes: 7 additions & 2 deletions cmd/kosli/evaluate.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,17 @@ import (
"github.com/spf13/cobra"
)

const evaluateShortDesc = `Evaluate Kosli trail data against OPA/Rego policies.`
const evaluateShortDesc = `Evaluate data against OPA/Rego policies.`

// Backtick breaks (`"` + "`x`" + `"`) are needed to embed markdown
// inline code spans inside raw string literals.
const evaluateLongDesc = evaluateShortDesc + `
Fetch trail data from Kosli and evaluate it against custom policies written
Evaluate trail data or local JSON input against custom policies written
in Rego, the policy language used by Open Policy Agent (OPA).

Use ` + "`evaluate trail`" + ` or ` + "`evaluate trails`" + ` to fetch data from Kosli and evaluate it.
Use ` + "`evaluate input`" + ` to evaluate a local JSON file or stdin without any API calls.

The policy must use ` + "`package policy`" + ` and define an ` + "`allow`" + ` rule.
An optional ` + "`violations`" + ` rule (a set of strings) can provide human-readable denial reasons.
The command exits with code 0 when allowed and code 1 when denied.`
Expand All @@ -28,6 +32,7 @@ func newEvaluateCmd(out io.Writer) *cobra.Command {
cmd.AddCommand(
newEvaluateTrailCmd(out),
newEvaluateTrailsCmd(out),
newEvaluateInputCmd(out),
)

return cmd
Expand Down
113 changes: 113 additions & 0 deletions cmd/kosli/evaluateInput.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
package main

import (
"encoding/json"
"fmt"
"io"
"os"

"github.com/spf13/cobra"
)

type evaluateInputOptions struct {
inputFile string
policyFile string
output string
showInput bool
}
Comment on lines +12 to +17
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit / future consideration: evaluateTrailOptions embeds commonEvaluateOptions and calls addFlags() to register the shared flags (policy, output, show-input). This struct re-declares those same three fields, and lines 62-64 duplicate the registration.

One option to avoid the duplication while keeping evaluate input free of the API-specific flags:

type evaluateInputOptions struct {
	commonEvaluateOptions
	inputFile string
}

Then in newEvaluateInputCmd, call o.addFlags(cmd, "...") and hide the flags that don't apply:

o.addFlags(cmd, "Path to a Rego policy file to evaluate against the input.")
cmd.Flags().StringVarP(&o.inputFile, "input-file", "i", "", "...")
// hide API-only flags
cmd.Flags().Lookup("flow").Hidden = true
cmd.Flags().Lookup("attestations").Hidden = true

Override RequireFlags to only require "policy" (not "flow").

This is the same observation from the earlier review comment — just fleshing out a concrete approach. Not blocking.


const evaluateInputShortDesc = `Evaluate a local JSON input against a Rego policy.`

const evaluateInputLongDesc = evaluateInputShortDesc + `
Read JSON from a file or stdin and evaluate it against a Rego policy using OPA.
The input can contain any JSON structure — the shape is defined by your policy.

The policy must use ` + "`package policy`" + ` and define an ` + "`allow`" + ` rule.
An optional ` + "`violations`" + ` rule (a set of strings) can provide human-readable denial reasons.
The command exits with code 0 when allowed and code 1 when denied.

When ` + "`--input-file`" + ` is omitted, JSON is read from stdin.`

const evaluateInputExample = `
# evaluate a local JSON file against a policy:
kosli evaluate input \
--input-file trail-data.json \
--policy policy.rego

# evaluate and show the data passed to the policy:
kosli evaluate input \
--input-file trail-data.json \
--policy policy.rego \
--show-input \
--output json

# read input from stdin:
cat trail-data.json | kosli evaluate input \
--policy policy.rego`

func newEvaluateInputCmd(out io.Writer) *cobra.Command {
o := new(evaluateInputOptions)
cmd := &cobra.Command{
Use: "input",
Short: evaluateInputShortDesc,
Long: evaluateInputLongDesc,
Example: evaluateInputExample,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
return o.run(out)
},
}

cmd.Flags().StringVarP(&o.inputFile, "input-file", "i", "", "[optional] Path to a JSON input file. Reads from stdin if omitted.")
cmd.Flags().StringVarP(&o.policyFile, "policy", "p", "", "Path to a Rego policy file to evaluate against the input.")
cmd.Flags().StringVarP(&o.output, "output", "o", "table", outputFlag)
cmd.Flags().BoolVar(&o.showInput, "show-input", false, "[optional] Include the policy input data in the output.")

err := RequireFlags(cmd, []string{"policy"})
if err != nil {
logger.Error("failed to configure required flags: %v", err)
}

return cmd
}

func (o *evaluateInputOptions) run(out io.Writer) error {
var input map[string]interface{}
var err error

if o.inputFile == "" {
input, err = loadInput(os.Stdin)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: os.Stdin is a package-level global, which makes this path harder to test in isolation (the suite test for "missing --input-file reads from stdin" relies on the process stdin being empty). Consider accepting the reader as a parameter to run() — e.g. injecting cmd.InOrStdin() — so a future test could supply a non-empty stdin without touching the global.

Not blocking for this slice, since the empty-stdin error path is already tested.

} else {
input, err = loadInputFromFile(o.inputFile)
}
if err != nil {
return err
}

return evaluateAndPrintResult(out, o.policyFile, input, o.output, o.showInput)
}

func loadInputFromFile(filePath string) (result map[string]interface{}, err error) {
f, err := os.Open(filePath)
if err != nil {
return nil, fmt.Errorf("failed to read input file: %w", err)
}
defer func() {
if cerr := f.Close(); cerr != nil && err == nil {
err = cerr
}
}()
return loadInput(f)
}

func loadInput(r io.Reader) (map[string]interface{}, error) {
data, err := io.ReadAll(r)
if err != nil {
return nil, fmt.Errorf("failed to read input: %w", err)
}
var input map[string]interface{}
if err := json.Unmarshal(data, &input); err != nil {
return nil, fmt.Errorf("failed to parse input: %w", err)
}
return input, nil
}
89 changes: 89 additions & 0 deletions cmd/kosli/evaluateInput_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package main

import (
"strings"
"testing"

"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)

type EvaluateInputCommandTestSuite struct {
suite.Suite
}

func (suite *EvaluateInputCommandTestSuite) TestEvaluateInputCmd() {
tests := []cmdTestCase{
{
wantError: true,
name: "missing --policy flag fails",
cmd: "evaluate input",
golden: "Error: required flag(s) \"policy\" not set\n",
},
{
name: "allow-all policy with input file returns ALLOWED",
cmd: "evaluate input --input-file testdata/evaluate/trail-input.json --policy testdata/policies/allow-all.rego",
goldenRegex: `RESULT:\s+ALLOWED`,
},
{
wantError: true,
name: "deny-all policy with input file returns DENIED",
cmd: "evaluate input --input-file testdata/evaluate/trail-input.json --policy testdata/policies/deny-all.rego",
goldenRegex: `RESULT:\s+DENIED`,
},
{
wantError: true,
name: "non-existent input file returns error",
cmd: "evaluate input --input-file testdata/evaluate/no-such-file.json --policy testdata/policies/allow-all.rego",
goldenRegex: `failed to read input file:`,
},
{
wantError: true,
name: "invalid JSON input file returns error",
cmd: "evaluate input --input-file testdata/policies/allow-all.rego --policy testdata/policies/allow-all.rego",
goldenRegex: `failed to parse input:`,
},
{
wantError: true,
name: "missing --input-file reads from stdin (empty stdin fails)",
cmd: "evaluate input --policy testdata/policies/allow-all.rego",
goldenRegex: `failed to parse input:`,
},
{
name: "JSON output with allow-all policy",
cmd: "evaluate input --input-file testdata/evaluate/trail-input.json --policy testdata/policies/allow-all.rego --output json",
goldenJson: []jsonCheck{
{"allow", true},
},
},
{
name: "show-input includes input in JSON output",
cmd: "evaluate input --input-file testdata/evaluate/trail-input.json --policy testdata/policies/allow-all.rego --output json --show-input",
goldenJson: []jsonCheck{
{"allow", true},
{"input.trail.name", "test-trail"},
},
},
}
runTestCmd(suite.T(), tests)
}

func TestLoadInput(t *testing.T) {
reader := strings.NewReader(`{"trail": {"name": "from-reader"}}`)
input, err := loadInput(reader)
require.NoError(t, err)
trail, ok := input["trail"].(map[string]interface{})
require.True(t, ok)
require.Equal(t, "from-reader", trail["name"])
}

func TestLoadInputInvalidJSON(t *testing.T) {
reader := strings.NewReader(`not json`)
_, err := loadInput(reader)
require.Error(t, err)
require.Contains(t, err.Error(), "failed to parse input")
}

func TestEvaluateInputCommandTestSuite(t *testing.T) {
suite.Run(t, new(EvaluateInputCommandTestSuite))
}
1 change: 1 addition & 0 deletions cmd/kosli/testdata/evaluate/trail-input.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"trail": {"name": "test-trail", "compliance_status": {}}}
Loading