feat(rerank): per-pool rerank callback for vector search

.changeset/rerank-callback.md

@@ -0,0 +1,8 @@
+---
+"payloadcms-vectorize": minor
+"@payloadcms-vectorize/pg": minor
+"@payloadcms-vectorize/cf": minor
+"@payloadcms-vectorize/mongodb": minor
+---
+
+Add optional `rerank` callback on `EmbeddingConfig` for per-pool reranking. When configured, the plugin fetches `Math.floor(limit * multiplier)` candidates from the adapter, hands them to the user-supplied callback (`(query, results) => Promise<results>`), and trims the callback's output back down to the caller's `limit`. Provider-agnostic — bring your own Voyage / Cohere / local cross-encoder. `multiplier` must be a finite number `>= 1`; invalid configs are rejected at plugin init. Callback errors propagate to the caller.

README.md

@@ -19,6 +19,7 @@ A Payload CMS plugin that adds vector search capabilities to your collections. P
 - 🎯 [**Flexible Chunking**](#chunkers) — drive chunk creation yourself with `toKnowledgePool` functions so you can combine any fields or content types.
 - 🧩 **Extensible Schema** — attach custom [`extensionFields`](#knowledge-pool-config) to the embeddings collection and persist values per chunk for querying.
 - 🌐 [**REST API**](#rest-endpoints) — built-in vector-search endpoint with Payload-style [`where` filtering](#metadata-filtering-where) and configurable limits.
+- 🎚️ [**Optional Reranking**](#reranking-optional) — bring your own reranker (Voyage, Cohere, local cross-encoder) to reorder candidates after the vector search.
 - 🏊 [**Multiple Knowledge Pools**](#knowledge-pool-config) — separate knowledge pools with independent configurations.
 - 🌍 [**Localization (i18n)**](#localization-i18n) — first-class pattern for embedding and searching multi-locale Payload content.
 
@@ -34,6 +35,7 @@ A Payload CMS plugin that adds vector search capabilities to your collections. P
   - [Adapter Configuration](#adapter-configuration)
   - [CollectionVectorizeOption](#collectionvectorizeoption)
 - [Metadata Filtering (`where`)](#metadata-filtering-where)
+- [Reranking (optional)](#reranking-optional)
 - [Chunkers](#chunkers)
 - [Localization (i18n)](#localization-i18n)
 - [Bulk Embeddings API](#bulk-embeddings-api)
@@ -383,6 +385,40 @@ References to fields that don't exist on the embeddings table are silently dropp
 
 > **Adapter parity.** All operators are implemented in `@payloadcms-vectorize/pg`. The Cloudflare Vectorize adapter has narrower native filtering — see [@payloadcms-vectorize/cf → Known Limitations](./adapters/cf/README.md#metadata-filtering) for what is and isn't supported there. The MongoDB adapter splits the clause into a native `$vectorSearch` pre-filter and a JS post-filter — `like`/`contains`/`all` and any mixed-pre/post `or` are post-filtered, so they may return fewer than `limit` rows. See [@payloadcms-vectorize/mongodb → WHERE clause behavior](./adapters/mongodb/README.md#where-clause-behavior).
 
+## Reranking (optional)
+
+Pass a `rerank` config on a pool's `embeddingConfig` to reorder candidates with your own reranker (e.g. Voyage, Cohere, a local cross-encoder):
+
+```ts
+embeddingConfig: {
+  version: 'v1',
+  queryFn,
+  realTimeIngestionFn,
+  rerank: {
+    // DB fetches Math.floor(limit * multiplier) candidates before reranking.
+    // Higher multiplier = better recall, more latency, more cost.
+    multiplier: 4,
+    callback: async (query, results) => {
+      const ranked = await myReranker.rerank({
+        query,
+        documents: results.map((r) => r.chunkText),
+      })
+      return ranked
+        .map((r) => results[r.index])
+        .filter((r): r is (typeof results)[number] => r !== undefined)
+    },
+  },
+}
+```
+
+**Multiplier.** Use `1` when you only want reordering of the same candidate set. Use a higher value (3–5 is typical) when you also want to expand the candidate pool before reranking — higher = better recall, more latency, more cost.
+
+**Limit.** The plugin trims the callback's output to the caller's `limit`. If the callback returns fewer than `limit`, the smaller count is returned as-is. If `limit` is omitted, the rerank branch uses `10` for fetch sizing — note this differs from the non-rerank path, where an omitted `limit` is forwarded to the adapter and the adapter picks its own default.
+
+**Errors.** Errors thrown by the callback propagate to the caller — there is no silent fallback to unranked results.
+
+**Validation.** `multiplier` must be a finite number `>= 1` and `callback` must be a function; invalid configs are rejected at plugin init.
+
 ## Chunkers
 
 Use chunker helpers (see `dev/helpers/chunkers.ts`) to keep `toKnowledgePool` implementations focused on orchestration. A `toKnowledgePool` can combine multiple chunkers, enrich each chunk with metadata, and return everything the embeddings collection needs.
@@ -832,7 +868,7 @@ if (vectorizedPayload) {
 
 #### `vectorizedPayload.search(params)`
 
-Perform vector search programmatically without making an HTTP request. Parameters and result shape are identical to [POST `/api/vector-search`](#post-apivector-search).
+Perform vector search programmatically without making an HTTP request. Parameters and result shape are identical to [POST `/api/vector-search`](#post-apivector-search). If the pool has a [`rerank`](#reranking-optional) config, this call goes through the same rerank pipeline as the REST endpoint.
 
 **Returns:** `Promise<Array<VectorSearchResult>>` — the array that the REST endpoint wraps in `{ results }`.
 

dev/specs/rerankValidation.spec.ts

@@ -0,0 +1,81 @@
+import { describe, expect, test } from 'vitest'
+import { buildConfig } from 'payload'
+import { postgresAdapter } from '@payloadcms/db-postgres'
+import { makeDummyEmbedDocs, makeDummyEmbedQuery, testEmbeddingVersion } from 'helpers/embed.js'
+import { createMockAdapter } from 'helpers/mockAdapter.js'
+import payloadcmsVectorize, { type RerankFn } from 'payloadcms-vectorize'
+import { DIMS } from './constants.js'
+
+const dbName = 'rerank_validation_test'
+
+const buildWithRerank = async (rerank: any) =>
+  buildConfig({
+    collections: [{ slug: 'posts', fields: [{ name: 'title', type: 'text' }] }],
+    db: postgresAdapter({
+      pool: {
+        connectionString: `postgresql://postgres:password@localhost:5433/${dbName}`,
+      },
+    }),
+    plugins: [
+      payloadcmsVectorize({
+        dbAdapter: createMockAdapter(),
+        knowledgePools: {
+          default: {
+            collections: {
+              posts: {
+                toKnowledgePool: async (doc) => (doc.title ? [{ chunk: doc.title }] : []),
+              },
+            },
+            embeddingConfig: {
+              version: testEmbeddingVersion,
+              queryFn: makeDummyEmbedQuery(DIMS),
+              realTimeIngestionFn: makeDummyEmbedDocs(DIMS),
+              rerank,
+            },
+          },
+        },
+      }),
+    ],
+    secret: 'rerank-validation-secret',
+    jobs: { tasks: [] },
+  })
+
+const validCallback: RerankFn = async (_q, r) => r
+
+describe('rerank config validation', () => {
+  test('multiplier = 0 throws', async () => {
+    await expect(buildWithRerank({ multiplier: 0, callback: validCallback })).rejects.toThrow(
+      /multiplier/i,
+    )
+  })
+
+  test('multiplier = -1 throws', async () => {
+    await expect(buildWithRerank({ multiplier: -1, callback: validCallback })).rejects.toThrow(
+      /multiplier/i,
+    )
+  })
+
+  test('multiplier = NaN throws', async () => {
+    await expect(buildWithRerank({ multiplier: NaN, callback: validCallback })).rejects.toThrow(
+      /multiplier/i,
+    )
+  })
+
+  test('multiplier = Infinity throws', async () => {
+    await expect(
+      buildWithRerank({ multiplier: Infinity, callback: validCallback }),
+    ).rejects.toThrow(/multiplier/i)
+  })
+
+  test('callback not a function throws', async () => {
+    await expect(buildWithRerank({ multiplier: 2, callback: 'nope' as any })).rejects.toThrow(
+      /callback/i,
+    )
+  })
+
+  test('valid config does not throw', async () => {
+    await expect(
+      buildWithRerank({ multiplier: 1.5, callback: validCallback }),
+    ).resolves.toBeTruthy()
+  })
+})