feat(permanent): chunking for values > 600 bytes (book/page envelope)

[corvid-agent]

Summary

fledge-plugin-memory previously rejected any permanent save whose encrypted envelope exceeded ~882 bytes (the @corvidlabs/ts-algochat hard cap), or whose tx note exceeded 1024 bytes (Algorand's per-tx limit). Real-world team-knowledge memories regularly hit these limits — a bulk import of 1,034 corvid-agent memories saw ~37% drop on this basis.

This adds transparent multi-tx chunking on the permanent tier. Mutable (ARC-69 ASA) chunking is intentionally deferred — single PR keeps the review small.

How it works

• New src/chunking.ts splits values into ≤600-byte UTF-8-safe chunks. Multi-byte codepoints (emoji, CJK, RTL) are never cut mid-character — the splitter walks back to the prior leading byte on a continuation cut.
• permanentSave emits N transactions for an N-chunk value. Each carries the same key + ISO-8601 created timestamp plus envelope fields book (= key today), page (1..N), total (N). Single-chunk saves stay on the legacy envelope shape so existing readers and indexers see no change.
• permanentRecall / permanentList group by (key, created) inside a new reassemble step, require all total pages to be present (saves with missing pages are silently dropped, never partially reconstructed), and join in page order.
• Tombstones (permanentDelete) cover all pages for a key without needing per-page tombstones — the latest-round-wins rule already picks the tombstone over the older multi-page write.

Envelope shape (single chunk — unchanged)

{
  "type": "permanent-memory",
  "key": "team-humans",
  "value": "<encrypted ≤882 bytes>",
  "user": "XHG33...",
  "created": "2026-05-18T22:00:00Z"
}

Envelope shape (multi-chunk — new)

{
  "type": "permanent-memory",
  "key": "team-interaction-guide",
  "value": "<encrypted chunk N>",
  "user": "XHG33...",
  "created": "2026-05-18T22:00:00Z",
  "book": "team-interaction-guide",
  "page": 2,
  "total": 4
}

Tests (19, all pass)

• test/chunking.test.ts — boundary cases (exactly MAX_CLEARTEXT_PER_CHUNK bytes, +1 byte), round-trip preservation including emoji + CJK + RTL, needsChunking heuristic.
• test/permanent-reassemble.test.ts — legacy single-chunk pass-through, multi-page join in order, missing-page drop, two-save dedup with different created timestamps, contiguous-page enforcement, mixed single+multi handling.

Test plan

• bun test (26/26 across 3 files)
• E2E save+recall of a 3030-byte payload against live localnet (running in parallel with this PR — will update if it surfaces anything)
• Round-trip preserves UTF-8 with emoji and CJK
• Single-chunk legacy envelope shape unchanged
• Reviewer to consider whether total cap is needed (e.g. reject saves > 100 chunks to bound recall cost)

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request implements a chunking system for Algorand transaction notes, which are limited to 1024 bytes. It introduces logic to split strings into UTF-8-safe segments and reassemble them, along with comprehensive unit tests for various character types and boundary conditions. A potential infinite loop was identified in the chunkValue function if the maximum chunk size is configured to be smaller than a single multi-byte character, and a fix was suggested to ensure the loop always progresses.

[gemini-code-assist]

The chunkValue function is susceptible to an infinite loop if MAX_CLEARTEXT_PER_CHUNK is set to a value smaller than the byte length of a single UTF-8 character (e.g., if it were reduced to 2 for testing or future changes). If end walks back all the way to offset, the offset will never advance. While 600 bytes is plenty for any valid UTF-8 character (max 4 bytes), adding a guard ensures robustness against configuration changes or malformed input.

  while (offset < bytes.length) {
    let end = Math.min(offset + MAX_CLEARTEXT_PER_CHUNK, bytes.length);
    // Walk back if we landed in the middle of a UTF-8 continuation byte
    // (0b10xxxxxx, i.e. (byte & 0xC0) === 0x80). We stop walking once
    // we hit a leading byte; this caps regression at 3 bytes.
    while (end > offset && end < bytes.length && (bytes[end] & 0xc0) === 0x80) end--;
    
    // If the chunk size is too small to fit even one full character, 
    // force advance to avoid an infinite loop.
    if (end === offset) end = Math.min(offset + MAX_CLEARTEXT_PER_CHUNK, bytes.length);

    chunks.push(bytes.slice(offset, end).toString("utf-8"));
    offset = end;
  }