Skip to content

feat: route document uploads through ChittyStorage#85

Merged
chitcommit merged 1 commit intomainfrom
feat/storage-ingest
Apr 8, 2026
Merged

feat: route document uploads through ChittyStorage#85
chitcommit merged 1 commit intomainfrom
feat/storage-ingest

Conversation

@chitcommit
Copy link
Copy Markdown
Contributor

@chitcommit chitcommit commented Apr 8, 2026

Summary

  • Add SVC_STORAGE service binding
  • Upload route calls storage_ingest MCP tool (content-addressed, entity-linked)
  • Fallback to direct R2 if SVC_STORAGE unavailable
  • Batch upload also routes through ChittyStorage

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Summary by CodeRabbit

  • Breaking Changes

    • Removed evidence/dispute linking from document uploads
    • Batch upload endpoint now returns only file status and content hash (no longer includes skipped items or document objects)
  • Improvements

    • Document uploads now use deterministic content-based file identification instead of filenames
    • Added optional entity_slug and origin parameters to upload requests
    • Improved file deduplication and storage handling

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Apr 8, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
chittycommand-ui 66b8da6 Apr 08 2026, 06:55 AM

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

  1. @coderabbitai review
  2. @copilot review
  3. @codex review
  4. @claude review
    Adversarial review request: evaluate security, policy bypass paths, regression risk, and merge-gating bypass attempts.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 8, 2026

Warning

Rate limit exceeded

@chitcommit has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 17 minutes and 37 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 17 minutes and 37 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2970e4cd-8444-40cb-8e8c-e597d0327f13

📥 Commits

Reviewing files that changed from the base of the PR and between b765879 and 66b8da6.

📒 Files selected for processing (3)
  • src/index.ts
  • src/routes/documents.ts
  • wrangler.toml
📝 Walkthrough

Walkthrough

Updated document upload flow to use ChittyStorage service via MCP JSON-RPC instead of direct evidence integration. Added SVC_STORAGE binding to environment type and wrangler configuration. Modified upload routes to compute SHA-256 hashes for deterministic deduplication and accept new entity_slug and origin parameters, with R2 fallback on service unavailability.

Changes

Cohort / File(s) Summary
Environment & Configuration
src/index.ts, wrangler.toml
Added SVC_STORAGE: Fetcher to Env type and configured ChittyStorage service binding in wrangler configuration.
Upload Route Logic
src/routes/documents.ts
Removed evidence integration and UUID validation; replaced with SHA-256-based deduplication and MCP JSON-RPC calls to ChittyStorage service. Added entity_slug and origin request parameters. Modified /upload/batch to use per-file hashing instead of filename-based dedup. Condensed /gaps query logic. Implemented fallback to direct R2 storage on service unavailability.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Worker as /upload Handler
    participant ChittyStorage as SVC_STORAGE<br/>(ChittyStorage)
    participant R2 as R2 Storage<br/>(Fallback)
    participant DB as Database<br/>(cc_documents)

    Client->>Worker: POST /upload (file, entity_slug, origin)
    activate Worker
    Worker->>Worker: Compute SHA-256 hash<br/>Derive chittyId = scan-{prefix}
    
    alt SVC_STORAGE Available
        Worker->>ChittyStorage: MCP storage_ingest call<br/>(chitty_id, filename, content,<br/>mime_type, origin, entity_slugs)
        activate ChittyStorage
        ChittyStorage-->>Worker: {r2_key, ...}
        deactivate ChittyStorage
        
        Worker->>DB: INSERT cc_documents<br/>(r2_key, processing_status='synced',<br/>metadata={content_hash, chitty_id})
        activate DB
        DB-->>Worker: Success
        deactivate DB
    else SVC_STORAGE Unavailable
        Worker->>R2: Store file<br/>(r2Key = sha256/{hash})
        activate R2
        R2-->>Worker: Success
        deactivate R2
        
        Worker->>DB: INSERT cc_documents<br/>(r2_key=sha256/{hash},<br/>processing_status='pending',<br/>source='manual')
        activate DB
        DB-->>Worker: Success
        deactivate DB
    end
    
    Worker-->>Client: Upload response<br/>(status, content_hash, chitty_id)
    deactivate Worker
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Hopping through hashes so fine,
SHA-256 dedup aligns,
ChittyStorage takes the lead,
Documents ingest at speed,
With fallback plans all in line!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: route document uploads through ChittyStorage' directly and clearly summarizes the main change: routing document uploads through the ChittyStorage service via the storage_ingest MCP tool.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/storage-ingest

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector
Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

- Add SVC_STORAGE service binding to chittystorage worker
- Upload route now calls storage_ingest MCP tool (content-addressed, entity-linked)
- Fallback to direct R2 if SVC_STORAGE unavailable (legacy path)
- Batch upload also routes through ChittyStorage
- Removes evidenceClient fire-and-forget (ChittyStorage handles pipeline)
- Adds entity_slug and origin params to upload form
- Content hash computed locally for chitty_id until ChittyIdentity integration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@chitcommit chitcommit force-pushed the feat/storage-ingest branch from b765879 to 66b8da6 Compare April 8, 2026 06:54
@chitcommit chitcommit merged commit 060a357 into main Apr 8, 2026
11 of 13 checks passed
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

  1. @coderabbitai review
  2. @copilot review
  3. @codex review
  4. @claude review
    Adversarial review request: evaluate security, policy bypass paths, regression risk, and merge-gating bypass attempts.

@chitcommit chitcommit deleted the feat/storage-ingest branch April 8, 2026 06:54
@chatgpt-codex-connector
Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b76587923d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +96 to +97
INSERT INTO cc_documents (doc_type, source, filename, r2_key, processing_status)
VALUES ('upload', 'manual', ${safeName}, ${r2Key}, 'pending')
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve linked_dispute_id when inserting uploads

The /upload path no longer persists linked_dispute_id, so documents uploaded from dispute context are silently detached from their dispute. The UI still sends this field (ui/src/lib/api.ts), and dispute detail fetches documents with WHERE linked_dispute_id = :id (src/routes/disputes.ts), so these uploads stop appearing in dispute timelines after this change.

Useful? React with 👍 / 👎.

Comment on lines +127 to +130
await c.env.SVC_STORAGE.fetch('https://internal/mcp', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject batch success when storage ingest returns HTTP error

This batch branch awaits fetch but never checks response.ok (or MCP error payload), so a 4xx/5xx from ChittyStorage still falls through and is reported as status: 'ok'. That creates silent data-loss behavior where clients believe uploads succeeded even though ingest was rejected.

Useful? React with 👍 / 👎.

const contentHash = Array.from(new Uint8Array(hashBuf)).map(b => b.toString(16).padStart(2, '0')).join('');

if (c.env.SVC_STORAGE) {
const content_base64 = btoa(String.fromCharCode(...bytes));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid spreading file bytes into fromCharCode

String.fromCharCode(...bytes) passes one argument per byte, which exceeds JS argument limits for normal document sizes and throws RangeError before ingest. In this batch path that means otherwise valid files fail whenever SVC_STORAGE is enabled; use chunked/base64 encoding that does not spread the whole array into function arguments.

Useful? React with 👍 / 👎.

customMetadata: { filename: safeName, source: 'chittycommand' },
});
}
results.push({ filename: safeName, status: 'ok', content_hash: contentHash });
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Insert batch-ingested files into cc_documents

After a successful batch ingest, the handler only appends to results and never writes a cc_documents row. Since document list and gap endpoints read from cc_documents, batch-uploaded files become invisible to dashboard/gap workflows despite returning success.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant