Skip to content

Conversation

@SamMorrowDrums
Copy link
Collaborator

Summary

Exploratory spike to evaluate finer-grained instruction control by associating instructions with specific sets of tools rather than whole toolsets.

Design

Core Types

  • InstructionRule - Associates an instruction with a set of tool names
  • InstructionResolver - Resolves which instructions apply using the most-specificity rule

Specificity Algorithm

  1. A rule matches when ALL its tools are present in active tools
  2. When Rule A's tools are a proper superset of Rule B's tools (A ⊃ B), A shadows B
  3. Partial overlaps (neither is superset) → both rules apply
  4. Output is sorted for determinism

Example

rules := []InstructionRule{
    NewInstructionRule("issues-all", "All issues instruction", "get_issue", "list_issues", "create_issue"),
    NewInstructionRule("issues-read", "Read issues instruction", "get_issue", "list_issues"),
    NewInstructionRule("create-only", "Create only instruction", "create_issue"),
}

// Active tools: {get_issue, list_issues, create_issue}
// Result: Only "issues-all" applies (shadows the other two)

// Active tools: {get_issue, list_issues}  
// Result: Only "issues-read" applies

Testing

All tests pass:

  • TestInstructionResolver_BasicMatching
  • TestInstructionResolver_SupersetShadowing
  • TestInstructionResolver_PartialOverlapNoShadowing
  • TestInstructionResolver_ComplexHierarchy
  • TestInstructionResolver_EmptyRules
  • TestInstructionResolver_EmptyActiveTools

Next Steps (if we proceed)

  • Integrate with existing GenerateInstructions() or create a V2
  • Define actual instruction rules based on tool combinations
  • Consider adding priority/weight for equal-size rule conflicts

Add InstructionRule and InstructionResolver types that enable associating
instructions with specific sets of tools rather than whole toolsets.

Key features:
- Rules match when ALL specified tools are present in active tools
- Most-specific rule wins: superset rules shadow subset rules
- Partial overlaps (neither superset) result in both rules applying
- Deterministic output via sorted results

This is exploratory work to evaluate finer-grained instruction control.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants