Skip to content

Backend: Add cursor-based pagination for high-volume campaign lists #318

@CelestinaBeing

Description

@CelestinaBeing

Summary

The backend uses offset/limit pagination (?page=1&limit=10). At scale with thousands of campaigns, offset pagination becomes slow (full table scan + skip) and unreliable when records are inserted between pages (page drift). Cursor-based pagination is the scalable solution for production.

Problem

In backend/src/pagination.js:

// offset/limit approach
items.slice(offset, offset + limit)

Issues:

  • SELECT * OFFSET N requires scanning and skipping N rows — O(N) cost
  • Campaign insertions between page fetches cause duplicate or missing items
  • No stable cursor for re-fetching from a known position

Acceptance Criteria

  • Add cursor-based pagination support in paginateItems(): if ?cursor= is provided, use it instead of page/offset
  • Cursor encodes: { id, createdAt } base64-encoded so it's opaque to clients
  • Implement in sqliteCampaignRepository.list(): WHERE (createdAt, id) < (cursor.createdAt, cursor.id) with ORDER BY createdAt DESC, id DESC LIMIT N+1
  • Response includes: { items, nextCursor, hasMore }
  • Keep backward compat: offset/limit pagination still works when ?cursor= is absent
  • Update frontend/src/components/Pagination.tsx to support cursor-mode (load more / infinite scroll option)
  • Add unit tests for cursor encoding/decoding, boundary conditions (first page, last page, empty), and consistency under concurrent inserts
  • Update backend/openapi.yaml

References

  • backend/src/pagination.js
  • backend/src/dal/sqliteCampaignRepository.js
  • frontend/src/components/Pagination.tsx

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave program

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions