Content-aware decay and revision for propositions

[jimador]

Observation

Proposition has decay, reinforceCount, and effectiveConfidence() - but these fields are largely passive, and they treat all knowledge identically.

The decay/reinforcement problem: effectiveConfidence() is a pure function on a single proposition. Nothing periodically acts on decayed confidence. reinforceCount is incremented during revision (RevisionResult.Reinforced), but there's no policy for what reinforcement means. MemoryMaintenanceOrchestrator runs consolidate → abstract → retire, and the retire phase prunes below a confidence threshold - but the decay formula is hardcoded and there's no reinforcement lifecycle.

The content-awareness problem: not all extracted knowledge ages or conflicts the same way:

• "The user's name is James" - persists until explicitly changed, shouldn't decay on a time curve
• "Always respond in formal English" - a constraint, not a claim; shouldn't decay at all
• "What's the status of the deployment?" - expires when answered, not when forgotten
• "I want to learn Kotlin" - active until completed or abandoned

These all get the same exp(-decay * k * age_days) treatment and the same PropositionReviser logic today.

What DICE already has

The building blocks are mostly there:

• Derivation already separates confidence, importance, and decay - the semantics just aren't content-aware
• PropositionReviser already produces Merged, Reinforced, Contradicted, Generalized - but the classification doesn't consider what kind of knowledge is being revised
• metadata: Map<String, Any> exists on Proposition and could carry content-kind signals without schema changes
• ExtractionPerspective already distinguishes whose knowledge - content-kind would distinguish what kind

The questions

1. Should decay and reinforcement be policy-driven?

Some possibilities:

1. Keep the current formula, add policies on top - effectiveConfidence() stays as-is. A DecayPolicy wraps it and adds threshold-based actions (demote below 0.3, archive below 0.1). The retire phase in MemoryMaintenanceOrchestrator delegates to this policy.

2. Make effectiveConfidence() pluggable - in*stead of hardcoding exp(-decay * k * age_days), delegate to a decay strategy interface. Content kind (however determined) selects the strategy.

3. ReinforcementPolicy SPI - when a proposition is reinforced (RevisionResult.Reinforced), a policy decides what happens: confidence boost, status change, authority promotion (if Proposition priority and authority model #13 is adopted). Makes reinforceCount functionally meaningful.

4. Scheduled decay sweeps - MemoryMaintenanceOrchestrator or a @Scheduled task periodically applies decay across all propositions in a context, archiving those below threshold. Currently, decay is only calculated on read (via effectiveConfidence()), never written back.*

2. Should decay and revision be content-aware?

Some possibilities, from least to most invasive:

1. Smarter extraction prompts - Tune prompts to set decay: 0.0 for directives, importance: 1.0 for state assertions, etc. No code changes. Relies on the LLM to infer content kind and express it through existing fields.

2. Metadata convention - Extraction prompts populate metadata["contentKind"] with a value like assertion, directive, state, question. No schema changes. Consumers (including PropositionReviser) can branch on it.

3. Pluggable decay strategy - Make effectiveConfidence() delegate to a strategy interface rather than hardcoding the exponential formula. Content kind (however determined) selects the strategy.

4. Type discriminator on Proposition - Add a contentKind: ContentKind enum field, defaulting to ASSERTION. PropositionReviser and PropositionAbstractor become type-aware.

5. Semantic unit generalization - Introduce a SemanticUnit type that Proposition specializes. Derivation already provides the shared base (confidence, importance, decay, grounding). A sealed hierarchy rooted at SemanticUnit would make content kind a first-class modeling concept rather than a field on Proposition:

   SemanticUnit (extends Derivation)
   ├── Assertion    - facts, claims, observations (current Proposition behavior)
   ├── Directive    - rules, constraints, policies
   ├── Intention    - goals, tasks, plans
   └── State        - entity attributes, system variables
   

   Each subtype could carry type-specific fields (e.g., Intention.completionCondition, Directive.scope) and override decay/revision behavior structurally rather than through branching. The taxonomy is small (~4 types) and clusters naturally from conversational content. PropositionReviser, PropositionAbstractor, and PropositionRepository would operate on SemanticUnit, with Proposition as the default/migration path.

These aren't mutually exclusive - (1) could validate whether the distinction matters before investing in (2)-(5).

Where content-kind would change behavior

Content Kind	Decay	Revision	Abstraction
Assertion (fact/claim)	Time-based (current)	Contradiction / reinforcement	Groups naturally
Directive (rule/constraint)	None	Supersession	Shouldn't merge with assertions
State (entity attribute)	Until changed	Replacement	Could roll up into entity profiles
Question	Event-based (answered)	N/A	N/A

Open questions

• Should decay be applied eagerly or lazily? Currently effectiveConfidence() calculates on read. Eager application (periodic sweeps) simplifies queries but loses the original confidence value.
• What does reinforcement actually do? Increment a counter? Boost confidence? Promote authority? Reset the decay clock? These are policy decisions that may vary by use case.
• Is LLM classification reliable enough? "James is a software engineer" - assertion or state? The boundaries are fuzzy. If classification is noisy, the value of branching on it drops.
• Is smarter prompt tuning sufficient? If extraction prompts can reliably set appropriate decay/importance values based on content kind, the explicit type discriminator may be unnecessary.
• How does this interact with ExtractionPerspective? "Always use formal English" from the user is a directive; from the agent it's a commitment. Should content-kind and perspective be orthogonal or coupled?
• What about abstraction? Should PropositionAbstractor respect content-kind boundaries, or is cross-kind synthesis valid (e.g., multiple state assertions → one assertion)?

[igordayen]

@jimador - summary of inquiry: rather than having generic decay function, consider decay function as content-type driven. please confirm, thank you

[jimador]

@igordayen Great question. The decay function itself should be generic (the exponential formula doesn't change). What becomes content-type-driven is the policy that assigns the decay value (e.g., stable facts like names get low decay, transient states like mood get high decay). This would be pluggable, so content-type (contentKind in the issue) is one strategy but you could also use constant values, rule-based logic, or user overrides. This issue is my attempt to capture the open design and direction questions.

[igordayen]

This issue is my attempt to capture the open design and direction questions.

@jimador - on board, same intent