Skip to content

Proposition priority and authority model #13

@jimador

Description

@jimador

Observation

All propositions in PropositionRepository are effectively peers. confidence and importance exist, but there's no framework-level concept of which propositions matter most — which should be retained vs. evicted under pressure, which should win in conflicts, or which an LLM should comply with more strongly.

This matters when:

  • Token budgets are limited — not all propositions fit in a prompt; which get included?
  • Knowledge conflicts arise — two conflicting propositions exist; which wins?
  • Memory pressure — too many propositions; which get evicted?
  • Compliance directives — some facts must be respected ("patient is allergic to penicillin") while others are suggestions

What DICE already has

  • confidence: ZeroToOne — LLM's extraction-time certainty. Doesn't evolve post-extraction (except through effectiveConfidence() decay).
  • importance: ZeroToOne — LLM's assessment of how much a fact matters. Set at extraction time, doesn't change.
  • decay: ZeroToOne — staleness rate. Feeds effectiveConfidence() but doesn't drive behavior beyond that.
  • reinforceCount: Int — tracks merge/reinforcement frequency. Stored but not acted on.
  • status: PropositionStatus — ACTIVE, SUPERSEDED, CONTRADICTED, PROMOTED. Lifecycle state, not priority.

The gap: confidence and importance are extraction-time snapshots. There's no lifecycle property that evolves based on reinforcement, decay, conflict resolution, or operator action.

The question

Is confidence + importance sufficient, or do propositions need a priority model that evolves over their lifecycle?

Some possibilities:

  1. Make existing fields dynamic — instead of new fields, let confidence and importance be updated by reinforcement, decay, and revision. reinforceCount already tracks frequency; a policy could boost confidence based on it.

  2. Add a rank field — an integer priority score (e.g., 0 = unranked, higher = more important). Influences retrieval ordering, eviction priority, conflict resolution. Evolves over time based on reinforcement and decay.

  3. Add an authority level — an enum indicating compliance strength, e.g.:

    Level Semantics
    PROVISIONAL May consider — unverified, low confidence
    RELIABLE Should respect — well-established
    CANON Must respect — ground truth, never auto-demoted

    Authority transitions: reinforcement promotes (PROVISIONAL → RELIABLE), decay demotes (RELIABLE → PROVISIONAL), CANON requires explicit operator action.

  4. Rank + authority together — rank = relative importance within an authority tier. Authority = how strongly the LLM should comply. Orthogonal dimensions.

Why not just use importance?

importance is a 0-1 float set at extraction time reflecting the LLM's assessment. Rank and authority would be lifecycle properties that evolve based on what happens after extraction:

  • importance: "How important did the extractor think this was?"
  • rank: "How important is this proposition now, given everything since extraction?"
  • authority: "How strongly should an LLM comply with this?"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions