Skip to content

feat(orm): implement postgres full-text search#2653

Open
ymc9 wants to merge 1 commit intodevfrom
feat/orm-full-text-search
Open

feat(orm): implement postgres full-text search#2653
ymc9 wants to merge 1 commit intodevfrom
feat/orm-full-text-search

Conversation

@ymc9
Copy link
Copy Markdown
Member

@ymc9 ymc9 commented May 6, 2026

Summary

  • Adds a Prisma-style full-text search capability behind a new field-level @fullText ZModel attribute. PostgreSQL only — MySQL/SQLite throw NotSupported. Mirrors the existing @fuzzy design.
  • Filter operator fts: where: { title: { fts: { search, config? } } } emits to_tsvector(field) @@ to_tsquery(query) (with an inline ::regconfig cast when config is provided; otherwise Postgres falls back to default_text_search_config).
  • OrderBy operator _ftsRelevance: { fields, search, config?, sort } emits a single ts_rank(...). Multi-field combines fields with concat_ws(' ', ...) so AND queries (e.g. cat & dog) match rows where the terms are split across fields — matches Prisma's behavior.
  • Type-level + Zod gating: the fts operator and _ftsRelevance orderBy appear only on String fields annotated with @fullText, only when the provider is postgresql. The 'FullText' filter kind plugs into existing slicing.
  • Cursor pagination is rejected when combined with _ftsRelevance (parallel to _fuzzyRelevance).
  • Refactors buildOrderBy into per-branch dispatchers: applyScalarOrderBy, applyAggregationOrderBy, applyRelationOrderBy, applyFuzzyRelevanceOrderBy, applyFtsRelevanceOrderBy.

Surface

// schema
model Article {
  id    Int    @id @default(autoincrement())
  title String @fullText
  body  String @fullText
  notes String?
}

// usage
const results = await db.article.findMany({
  where: { title: { fts: { search: 'cat & dog', config: 'english' } } },
  orderBy: {
    _ftsRelevance: { fields: ['title', 'body'], search: 'cat & dog', sort: 'desc' },
  },
});

Test plan

  • pnpm build (full graph)
  • pnpm lint
  • e2e typecheck (pnpm test:typecheck)
  • New full-text-search.test.ts — 31 tests covering: basic fts, &/|/!/<-> operators, config (english stems, simple does not, per-query isolation, DB-default fallback), composition with other filters, _ftsRelevance single + multi-field (incl. AND-across-fields), pagination + cursor guard, mutations (updateMany/deleteMany), @fullText gating (Zod-precise error messages), malformed query passthrough, OrArray contract, and three filter-kind slicing tests
  • No regressions in fuzzy-search.test.ts (64 tests)
  • No regressions in broader orderBy-touching suites (find, filter, aggregate, group-by, relation)

Notes

  • Single-field SQL is identical to Prisma's. Multi-field uses concat_ws (Prisma parity) rather than summing per-field ts_rank — chosen because the SUM form silently scores 0 on AND queries when terms span fields.
  • Out of scope (future work): per-field weighting via setweight(), schema-level default config (@fullText(config: "english")), GIN-index hint generation, and stored tsvector columns. Wrapping under fts (vs. Prisma's flat search: string) leaves room for these.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Introduced full-text search for PostgreSQL, allowing users to annotate string fields and perform advanced text queries with relevance-based result ordering, boolean operators, configurable search options, and field-specific filtering capabilities. Supports pagination and result mutations.
  • Tests

    • Added comprehensive end-to-end test suite covering full-text search queries, ordering, pagination, and mutations.

Introduces a Prisma-style full-text search capability gated by a new
field-level `@fullText` ZModel attribute. PostgreSQL only — MySQL/SQLite
throw NotSupported. Mirrors the existing `@fuzzy` design.

- Filter operator: `where: { title: { fts: { search, config? } } }`
  emits `to_tsvector(field) @@ to_tsquery(query)` (or with a `::regconfig`
  cast when `config` is provided; otherwise Postgres uses the database's
  `default_text_search_config`).
- OrderBy operator: `_ftsRelevance: { fields, search, config?, sort }`
  emits a single `ts_rank(...)`. Multi-field combines fields with
  `concat_ws(' ', ...)` so AND queries match terms across fields
  (matches Prisma's behavior).
- Type-level gating: the `fts` operator and `_ftsRelevance` orderBy
  appear only on String fields annotated with `@fullText` and only when
  the schema's provider is `postgresql`. Slicing's `'FullText'` filter
  kind controls availability of the runtime operator.
- Cursor pagination is rejected when combined with `_ftsRelevance`
  (parallel to `_fuzzyRelevance`).

Also refactors `buildOrderBy` to dispatch to small per-branch helpers
(`applyScalarOrderBy`, `applyAggregationOrderBy`, `applyRelationOrderBy`,
`applyFuzzyRelevanceOrderBy`, `applyFtsRelevanceOrderBy`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 6, 2026

📝 Walkthrough

Walkthrough

A new full-text search feature is added to the ORM stack with PostgreSQL support. Changes include a @fullText() attribute declaration, validation, type system extensions for full-text queries, dialect-level implementations for PostgreSQL with stubs for other providers, query builder integration, and comprehensive test coverage.

Changes

Full-Text Search Feature Implementation

Layer / File(s) Summary
Language Definition
packages/language/res/stdlib.zmodel
New @fullText() attribute declared for StringField, placed after the fuzzy attribute.
Language Validation
packages/language/src/validators/attribute-application-validator.ts
New _checkFullText validator method ensures @fullText is only applied with PostgreSQL provider.
Schema Type System
packages/schema/src/schema.ts, packages/orm/src/client/constants.ts
FieldDef extended with optional fullText boolean field; FILTER_PROPERTY_TO_KIND mapping adds fts: 'FullText' entry.
Core Type Definitions
packages/orm/src/client/crud-types.ts
New types introduced: FullTextFields, FullTextFilterPayload, FtsRelevanceOrderBy, and ProviderSupportsFullText feature flag. String field filters composed with nested full-text and fuzzy augmentations; orderBy extended for _ftsRelevance.
Dialect Base Infrastructure
packages/orm/src/client/crud/dialects/base-dialect.ts
Abstract methods buildFullTextFilter() and buildFtsRelevanceOrderBy() added to BaseCrudDialect. Cursor pagination validation updated to reject _fuzzyRelevance and _ftsRelevance.
PostgreSQL Implementation
packages/orm/src/client/crud/dialects/postgresql.ts
New methods implement full-text filtering via to_tsvector/to_tsquery with optional regconfig and relevance ordering via ts_rank. Helper normalizeFullTextOptions() validates FTS payload shape.
MySQL & SQLite Stubs
packages/orm/src/client/crud/dialects/mysql.ts, packages/orm/src/client/crud/dialects/sqlite.ts
Both dialects add buildFullTextFilter() and buildFtsRelevanceOrderBy() overrides that throw not-supported errors.
Query Builder Integration
packages/orm/src/client/zod/factory.ts
String filter schema generation extended with optional fts operator when provider supports full-text search; makeStringFilterSchema() and makePrimitiveFilterSchema() accept new withFullText parameter; orderBy schema includes _ftsRelevance when supported.
SDK Code Generation
packages/sdk/src/ts-schema-generator.ts
createDataFieldObject() updated to emit fullText: true when @fullText attribute is present.
Tests & Schema
tests/e2e/orm/schemas/full-text-search/schema.zmodel, tests/e2e/orm/schemas/full-text-search/schema.ts, tests/e2e/orm/client-api/full-text-search.test.ts
New PostgreSQL test schema with Article model featuring @fullText fields. Comprehensive test suite covers FTS queries, boolean operators, config options, field restrictions, relevance ordering, pagination, and gating behavior.

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 Full-text whispers now dance through the code,
From language to dialect, a feature bestowed,
PostgreSQL's powerful search comes to play,
While MySQL and SQLite wait for their day,
With types, tests, and validation so tight—
The rabbit's warren now searches right! 🔍

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat(orm): implement postgres full-text search' accurately and concisely summarizes the main change—adding PostgreSQL full-text search support to the ORM via the new @fullText attribute and fts/ftsRelevance operators.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/orm-full-text-search

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 6, 2026

Claude finished @ymc9's task —— View job


PR Review: Full-Text Search Implementation

I've conducted a comprehensive review of the full-text search implementation. Overall, this is a well-architected and secure implementation that follows good patterns.

Review Tasks:

  • Analyze core implementation files
  • Review SQL generation and injection safety
  • Evaluate type safety and error handling
  • Assess test coverage and quality
  • Check performance considerations
  • Review architecture and code organization

Strengths

Security & SQL Injection Prevention

  • Excellent parameterization: All user inputs are properly parameterized using sql.val() in packages/orm/src/client/crud/dialects/postgresql.ts:665-670
  • No string concatenation: The implementation correctly avoids string concatenation for SQL generation
  • Proper validation: Input validation in normalizeFullTextOptions() prevents malformed queries from reaching the database

Type Safety

  • Strong typing: Full-text search operators only appear on @fullText annotated String fields
  • Provider gating: Type-level restrictions ensure fts operators only exist when provider = "postgresql"
  • Zod validation: Comprehensive runtime validation in packages/orm/src/client/zod/factory.ts:1093-1098

Architecture & Code Organization

  • Consistent patterns: Mirrors existing @fuzzy implementation design
  • Proper separation: Clean separation between SQL generation, type generation, and validation
  • Provider abstraction: Other providers (MySQL/SQLite) correctly throw NotSupported errors

Comprehensive Test Coverage

The test suite in tests/e2e/orm/client-api/full-text-search.test.ts covers:

  • Basic single-term searches
  • Boolean operators (&, |, !, <->)
  • Configuration support with different text search configs
  • Multi-field relevance ordering
  • Error cases and malformed queries
  • Integration with other filters

🔍 Areas for Consideration

Error Handling

  • Malformed query passthrough: The implementation correctly passes malformed to_tsquery syntax to PostgreSQL, which will throw at execution time. This is acceptable but could potentially be improved with client-side validation.

Performance Considerations

  • Index recommendations: Consider documenting that users should create GIN indexes on @fullText fields for optimal performance
  • Multi-field concatenation: Using concat_ws(' ', ...) for multi-field relevance is correct for AND queries but may have different performance characteristics than individual field ranking

Documentation

  • Missing index guidance: The implementation doesn't provide guidance on creating supporting indexes
  • Configuration examples: More examples of different text search configurations could be helpful

🏗️ Technical Implementation Details

SQL Generation (packages/orm/src/client/crud/dialects/postgresql.ts)

// Clean parameterized approach - no injection risk
if (options.config === undefined) {
    return sql<SqlBool>`to_tsvector(${fieldRef}) @@ to_tsquery(${query})`;
}
const cfg = sql.val(options.config);
return sql<SqlBool>`to_tsvector(${cfg}::regconfig, ${fieldRef}) @@ to_tsquery(${cfg}::regconfig, ${query})`;

Type Safety (packages/orm/src/client/crud-types.ts)

The type system ensures fts only appears on properly annotated fields:

  • Fields must have @fullText attribute
  • Provider must be postgresql
  • Proper integration with filter slicing system

Multi-field Relevance

The concat_ws approach for multi-field ranking is well-chosen:

  • Handles NULL values correctly
  • Enables AND queries to match across fields
  • Maintains Prisma compatibility

🎯 Minor Suggestions

  1. Consider adding index hints: Future enhancement could include automatic GIN index recommendations
  2. Enhanced error messages: Could provide more specific error messages for common tsquery syntax errors
  3. Documentation: Add examples of optimal index creation in the schema documentation

📋 Summary

This is a high-quality implementation that:

  • ✅ Properly prevents SQL injection through parameterization
  • ✅ Maintains strong type safety with provider and attribute gating
  • ✅ Follows established architectural patterns
  • ✅ Includes comprehensive test coverage
  • ✅ Handles edge cases appropriately

The implementation is ready for production and follows security best practices. The code quality is excellent and maintains consistency with the existing codebase.


@ymc9 ymc9 changed the title feat(orm): add @fullText attribute and Postgres full-text search feat(orm): implement postgres full-text search May 6, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/orm/src/client/crud-types.ts`:
- Around line 416-467: The current AddFuzzyFilterIfSupported and
AddFullTextFilterIfSupported intersect the extra operator bag directly into Base
causing union distribution that drops primitive shorthand (e.g., string); change
the augmentation to only apply to object branches by replacing "Base & { ... } :
Base" with a distributive conditional that checks whether Base extends object
(or Record<string, unknown>) and only then adds the fuzzy/fts operator (i.e.,
Base extends object ? Base & { fuzzy?: FuzzyFilterPayload } : Base for
AddFuzzyFilterIfSupported and similarly Base extends object ? Base & { fts?:
FullTextFilterPayload } : Base for AddFullTextFilterIfSupported), keeping the
existing guards that check ProviderSupportsFuzzy/ProviderSupportsFullText and
GetModelField[...]['fuzzy'|'fullText'].

In `@packages/orm/src/client/crud/dialects/postgresql.ts`:
- Around line 709-719: The single-field branch builds `document` from
`fieldRefs[0]` but doesn't coalesce nullable strings, so
`to_tsvector(${document})` can yield NULL and make `ts_rank` return NULL; update
the single-field branch that defines `document` (the `document` variable used
with `to_tsvector`/`ts_rank` and in `query.orderBy`) to coalesce the single
field to an empty string (same null-skipping behavior as the multi-field
`concat_ws` path) before passing it to `to_tsvector` and `ts_rank`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9913c800-a26e-4f36-a060-f28be32e840f

📥 Commits

Reviewing files that changed from the base of the PR and between 9d147b9 and d0dd954.

📒 Files selected for processing (14)
  • packages/language/res/stdlib.zmodel
  • packages/language/src/validators/attribute-application-validator.ts
  • packages/orm/src/client/constants.ts
  • packages/orm/src/client/crud-types.ts
  • packages/orm/src/client/crud/dialects/base-dialect.ts
  • packages/orm/src/client/crud/dialects/mysql.ts
  • packages/orm/src/client/crud/dialects/postgresql.ts
  • packages/orm/src/client/crud/dialects/sqlite.ts
  • packages/orm/src/client/zod/factory.ts
  • packages/schema/src/schema.ts
  • packages/sdk/src/ts-schema-generator.ts
  • tests/e2e/orm/client-api/full-text-search.test.ts
  • tests/e2e/orm/schemas/full-text-search/schema.ts
  • tests/e2e/orm/schemas/full-text-search/schema.zmodel

Comment on lines 416 to +467
type AddFuzzyFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> =
GetModelFieldType<Schema, Model, Field> extends 'String'
? 'Fuzzy' extends AllowedKinds
? ProviderSupportsFuzzy<Schema> extends true
? GetModelField<Schema, Model, Field>['fuzzy'] extends true
? Base & {
/**
* Performs a fuzzy search on the string field. Only available when
* the schema's provider is `postgresql` (requires `pg_trgm` extension)
* and the field is annotated with `@fuzzy` in the ZModel schema.
* See {@link FuzzyFilterPayload} for the full options reference.
*/
fuzzy?: FuzzyFilterPayload;
}
: Base
: Base
> = 'Fuzzy' extends AllowedKinds
? ProviderSupportsFuzzy<Schema> extends true
? GetModelField<Schema, Model, Field>['fuzzy'] extends true
? Base & {
/**
* Performs a fuzzy search on the string field. Only available when
* the schema's provider is `postgresql` (requires `pg_trgm` extension)
* and the field is annotated with `@fuzzy` in the ZModel schema.
* See {@link FuzzyFilterPayload} for the full options reference.
*/
fuzzy?: FuzzyFilterPayload;
}
: Base
: Base
: Base;

/**
* Conditionally augments a string-field filter with the `fts` operator when:
* 1. The `FullText` filter kind is allowed for this field, AND
* 2. The schema's provider supports full-text search (postgres only), AND
* 3. The field is annotated with `@fullText` in the ZModel schema.
*
* Caller is responsible for only invoking this on String-typed fields
* (the gate lives in `FieldFilter`).
*/
type AddFullTextFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> = 'FullText' extends AllowedKinds
? ProviderSupportsFullText<Schema> extends true
? GetModelField<Schema, Model, Field>['fullText'] extends true
? Base & {
/**
* Performs a full-text search on the string field. Only available when
* the schema's provider is `postgresql` and the field is annotated with
* `@fullText` in the ZModel schema.
* See {@link FullTextFilterPayload} for the full options reference.
*/
fts?: FullTextFilterPayload;
}
: Base
: Base;
: Base
: Base;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

wc -l packages/orm/src/client/crud-types.ts

Repository: zenstackhq/zenstack

Length of output: 106


🏁 Script executed:

head -n 50 packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 1493


🏁 Script executed:

sed -n '380,480p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 4853


🏁 Script executed:

grep -n "type PrimitiveFilter" packages/orm/src/client/crud-types.ts

Repository: zenstackhq/zenstack

Length of output: 89


🏁 Script executed:

sed -n '200,350p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 6609


🏁 Script executed:

sed -n '558,700p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 5876


🏁 Script executed:

sed -n '78,126p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 2543


🏁 Script executed:

cat > /tmp/test-intersection.ts << 'EOF'
// Simplified test of the intersection behavior

type NullableIf<T, N extends boolean> = N extends true ? T | null : T;

// Mimics StringFilter structure
type StringFilterBase<Nullable extends boolean> =
  | (Nullable extends true ? string | null : string)
  | { equals?: string; contains?: string };

// Mimics the current implementation
type WithFuzzy<Base> = Base & { fuzzy?: { pattern: string } };

// Test case 1: Check if scalar branch survives
type Result1 = WithFuzzy<StringFilterBase<false>>;
// This should be: (string & { fuzzy?: ... }) | ({ equals?: string } & { fuzzy?: ... })
// The first part resolves to: never (because string can't have fuzzy property)
// The second part resolves to: { equals?: string; fuzzy?: ... }

// So the plain string shorthand "foo" should not work anymore

// Mimics the suggested fix
type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;
type WithFuzzyFixed<Base> = Base extends (infer Branch)[]
  ? Branch[]
  : Base extends (A | B)
  ? ExtendObjectBranch<A, { fuzzy?: { pattern: string } }> | ExtendObjectBranch<B, { fuzzy?: { pattern: string } }>
  : ExtendObjectBranch<Base, { fuzzy?: { pattern: string } }>;

// Let me try a simpler approach
type FixedWithFuzzy<Base> = Base extends object
  ? Base & { fuzzy?: { pattern: string } }
  : Base;

// This should preserve the scalar branch:
// - string → string (not object, so no intersection)
// - { equals?: string } → { equals?: string; fuzzy?: ... }

type Result2 = FixedWithFuzzy<StringFilterBase<false>>;
// Should be: string | { equals?: string; fuzzy?: ... }

EOF
cat /tmp/test-intersection.ts

Repository: zenstackhq/zenstack

Length of output: 1655


🏁 Script executed:

find packages/orm -name "*.test.ts" -o -name "*.spec.ts" | head -10

Repository: zenstackhq/zenstack

Length of output: 137


🏁 Script executed:

rg -l "AddFuzzyFilterIfSupported|AddFullTextFilterIfSupported" packages/orm/

Repository: zenstackhq/zenstack

Length of output: 101


🏁 Script executed:

sed -n '148,180p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 2174


🏁 Script executed:

sed -n '148,430p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 13649


🏁 Script executed:

grep -n "where.*title.*foo\|where.*{.*:.*['\"]" packages/orm/test/*.test.ts | head -20

Repository: zenstackhq/zenstack

Length of output: 45


🏁 Script executed:

rg "where\s*{" packages/orm/test -A 2 | head -30

Repository: zenstackhq/zenstack

Length of output: 145


🏁 Script executed:

cat > /tmp/verify-intersection.ts << 'EOF'
// Verify the union narrowing claim
type StringFilter = string | { equals?: string; contains?: string };

// Current implementation - intersects with whole union
type CurrentImpl<Base> = Base & { fuzzy?: string };

// Test: does plain string survive?
type Test1 = CurrentImpl<StringFilter>;
// Expected: string | { equals?: string; contains?: string; fuzzy?: string }
// Actual: { equals?: string; contains?: string; fuzzy?: string } (string disappears)

// Proposed fix
type FixedImpl<Base> = Base extends object ? Base & { fuzzy?: string } : Base;

// Test: does plain string survive?
type Test2 = FixedImpl<StringFilter>;
// Expected: string | { equals?: string; contains?: string; fuzzy?: string }
// Actual: string | { equals?: string; contains?: string; fuzzy?: string } (string preserved)

// Verify the broken case at the value level
const test1: Test1 = "hello"; // Should error - string is not in the union anymore
const test1b: Test1 = { equals: "world" }; // Should work

const test2: Test2 = "hello"; // Should work - string is preserved
const test2b: Test2 = { equals: "world" }; // Should work
EOF

cat /tmp/verify-intersection.ts

Repository: zenstackhq/zenstack

Length of output: 1172


🏁 Script executed:

find packages/orm/test -type f -name "*.ts" -exec grep -l "where.*:" {} \; | head -5

Repository: zenstackhq/zenstack

Length of output: 45


🏁 Script executed:

sed -n '1,100p' packages/orm/test/crud.test.ts | grep -A 5 "where"

Repository: zenstackhq/zenstack

Length of output: 137


🏁 Script executed:

find packages/orm/test -type f -name "*.test.ts" | head -5

Repository: zenstackhq/zenstack

Length of output: 137


🏁 Script executed:

ls -la packages/orm/test/*.ts | head -10

Repository: zenstackhq/zenstack

Length of output: 235


🏁 Script executed:

ls -la packages/orm/test/

Repository: zenstackhq/zenstack

Length of output: 367


🏁 Script executed:

cat packages/orm/test/procedures.test.ts

Repository: zenstackhq/zenstack

Length of output: 4375


🏁 Script executed:

rg "where.*:" packages/orm -A 1 -t ts | head -50

Repository: zenstackhq/zenstack

Length of output: 2942


🏁 Script executed:

rg "equals\|contains\|startsWith" packages/orm/src -B 2 | head -50

Repository: zenstackhq/zenstack

Length of output: 45


🏁 Script executed:

cat > /tmp/test-union-narrowing.ts << 'EOF'
// Test TypeScript's union narrowing behavior with intersections

// Simple union with scalar and object branch
type StringFilter = string | { equals?: string; contains?: string };

// Test 1: Current implementation (intersect entire union)
type AddFuzzyBroken<Base> = Base & { fuzzy?: string };

// What does TypeScript do with (string | object) & Extra?
// Result: (string & Extra) | (object & Extra)
// But string & { fuzzy } = never (string can't have fuzzy property)
// So: never | (object & Extra) = object & Extra
// Result loses the string branch!

type Result1 = AddFuzzyBroken<StringFilter>;
// Hovers reveal: { equals?: string | undefined; contains?: string | undefined; fuzzy?: string | undefined }
// The `string` branch is gone!

// Test 2: Proposed fix (only intersect object branches)
type AddFuzzyFixed<Base> = Base extends object ? Base & { fuzzy?: string } : Base;

// What happens here?
// When Base = (string | {equals, contains})
// This applies to the WHOLE union type, not distributed
// So: (string | object) extends object? NO
// Result: Base (unchanged) = string | object
// Wait, that's not right either...

// We need to distribute properly
type AddFuzzyFixedCorrect<Base> =
  Base extends (infer U)[]
    ? (U extends object ? U & { fuzzy?: string } : U)[]
    : Base extends object
      ? Base & { fuzzy?: string }
      : Base;

// Actually, the issue is that when you pass a union to a generic,
// it doesn't auto-distribute. Let me test this:

type TestUnion<T> = T extends object ? T & { fuzzy?: string } : T;

// When called with a union:
type Result2 = TestUnion<StringFilter>;
// This does NOT distribute!
// StringFilter (string | object) extends object? NO
// Result: StringFilter (unchanged)

// To make it work properly, we need to ensure distribution:
type AddFuzzyFixedProper<Base> = Base extends infer U
  ? U extends object
    ? U & { fuzzy?: string }
    : U
  : never;

type Result3 = AddFuzzyFixedProper<StringFilter>;
// Now it distributes! Result: string | { equals?: string; contains?: string; fuzzy?: string }

EOF
cat /tmp/test-union-narrowing.ts

Repository: zenstackhq/zenstack

Length of output: 2129


🏁 Script executed:

python3 << 'EOF'
# Let's analyze the TypeScript logic
logic = """
Key insight: In TypeScript, when you have a union like (A | B) and you intersect it:
- (A | B) & C = (A & C) | (B & C)  [distribution rule]

So for: (string | object) & { fuzzy }
Result: (string & { fuzzy }) | (object & { fuzzy })
       = never | (object & { fuzzy })  [because string can't have fuzzy property]
       = object & { fuzzy }

This LOSES the string branch!

With the fix:
type Fix<Base> = Base extends object ? Base & { fuzzy } : Base;

For Base = (string | {equals})
- This does NOT automatically distribute to each union member
- Need to use: Base extends infer U ? (U extends object ? U & {fuzzy} : U) : never

But the actual fix in the suggestion uses:
type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;

This only works if Base is ALREADY ONE OF THE UNION MEMBERS, not the union itself.

The real issue is that the helper is receiving a UNION as Base,
but the conditional doesn't distribute over the union automatically.
"""

print(logic)
EOF

Repository: zenstackhq/zenstack

Length of output: 1037


🏁 Script executed:

sed -n '233,260p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 1450


🏁 Script executed:

sed -n '233,260p' packages/orm/src/client/crud-types.ts

Repository: zenstackhq/zenstack

Length of output: 1254


🏁 Script executed:

cat > /tmp/verify-fix.ts << 'EOF'
// Verify the fix works with conditional distribution

type StringFilter = string | { equals?: string; contains?: string };

// Original broken approach
type AddFuzzyBroken = StringFilter & { fuzzy?: string };
// Result: { equals?: string; contains?: string; fuzzy?: string }
// String branch is LOST!

// Proposed fix using conditional that distributes
type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;
type AddFuzzyFixed = ExtendObjectBranch<StringFilter, { fuzzy?: string }>;

// When Base = string | object, and you use: Base extends object ? X : Base
// TypeScript distributes the conditional:
// = (string extends object ? string & Extra : string) | (object extends object ? object & Extra : object)
// = string | (object & Extra)
// String branch is PRESERVED!

// Verify at value level:
const test1: AddFuzzyBroken = "hello"; // ERROR: 'string' is not assignable
const test1b: AddFuzzyBroken = { equals: "world" }; // OK

const test2: AddFuzzyFixed = "hello"; // OK - string is preserved!
const test2b: AddFuzzyFixed = { equals: "world" }; // OK
const test2c: AddFuzzyFixed = { equals: "world", fuzzy: { pattern: "test" } }; // OK
EOF

cat /tmp/verify-fix.ts

Repository: zenstackhq/zenstack

Length of output: 1234


🏁 Script executed:

sed -n '636,656p' packages/orm/src/client/crud-types.ts | cat -n

Repository: zenstackhq/zenstack

Length of output: 1023


🏁 Script executed:

cat > /tmp/final-verification.ts << 'EOF'
// Complete verification of the review comment's claim

// StringFilter definition (from lines 636-656):
type StringFilter<Nullable extends boolean> =
  | (Nullable extends true ? string | null : string)  // <- scalar shorthand branch
  | { equals?: string; contains?: string };  // <- object branch with operators

// Current implementation (lines 278 and 301):
type AddFuzzyBroken<Base> = Base & { fuzzy?: string };

// When Base = StringFilter<false> = string | { equals?, contains? }
// TypeScript union distribution during intersection:
// (string | object) & { fuzzy }
// = (string & { fuzzy }) | (object & { fuzzy })
// = never | (object & { fuzzy })  [string can't have fuzzy property]
// = object & { fuzzy }
// Result: The string branch DISAPPEARS!

type ResultBroken = AddFuzzyBroken<StringFilter<false>>;
// Actual type: { equals?: string; contains?: string; fuzzy?: string }
// NO longer accepts plain string!

// Proposed fix (from the suggestion):
type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;

// When Base = StringFilter<false> = string | { equals?, contains? }
// TypeScript distributes the conditional to each union member:
// (string extends object ? string & { fuzzy } : string) 
// | (object extends object ? object & { fuzzy } : object)
// = string | (object & { fuzzy })
// The string branch is PRESERVED!

type ResultFixed = ExtendObjectBranch<StringFilter<false>, { fuzzy?: string }>;
// Actual type: string | { equals?: string; contains?: string; fuzzy?: string }
// Still accepts plain string!

// Value-level verification:
const test1: ResultBroken = "hello";  // ❌ ERROR
const test1b: ResultBroken = { equals: "world" };  // ✓ OK

const test2: ResultFixed = "hello";  // ✓ OK (fixed!)
const test2b: ResultFixed = { equals: "world" };  // ✓ OK
const test2c: ResultFixed = { equals: "world", fuzzy: { pattern: "test" } };  // ✓ OK
EOF

cat /tmp/final-verification.ts

Repository: zenstackhq/zenstack

Length of output: 1958


🏁 Script executed:

# Check if there are actual uses of string filters in a real test or example
rg "title|description|name" packages/orm/test -A 3 | grep -A 3 "where\|equals\|contains" | head -20

Repository: zenstackhq/zenstack

Length of output: 45


🏁 Script executed:

# Look for actual schema examples to see if string filters are used
find packages -name "*.zmodel" -o -name "*schema*" | head -10

Repository: zenstackhq/zenstack

Length of output: 577


Preserve the scalar shorthand when adding fuzzy/fts operators.

The current helpers intersect the extra operator bag directly into the entire StringFilter union, which causes TypeScript to distribute: (string | object) & Extra becomes (string & Extra) | (object & Extra). Since primitives cannot have extra properties, string & Extra resolves to never, leaving only the object branch. On fields annotated with @fuzzy or @fullText, this breaks the plain-string shorthand syntax where: { title: 'foo' } at the type level, even though runtime validation still accepts it.

Apply the augmentation only to object branches using a conditional that distributes across union members:

Suggested fix
+type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;
+
 type AddFuzzyFilterIfSupported<
     Schema extends SchemaDef,
     Model extends GetModels<Schema>,
     Field extends GetModelFields<Schema, Model>,
     AllowedKinds extends FilterKind,
     Base,
 > = 'Fuzzy' extends AllowedKinds
     ? ProviderSupportsFuzzy<Schema> extends true
         ? GetModelField<Schema, Model, Field>['fuzzy'] extends true
-            ? Base & {
+            ? ExtendObjectBranch<
+                  Base,
+                  {
                       fuzzy?: FuzzyFilterPayload;
-              }
+                  }
+              >
             : Base
         : Base
     : Base;
 
 type AddFullTextFilterIfSupported<
     Schema extends SchemaDef,
     Model extends GetModels<Schema>,
     Field extends GetModelFields<Schema, Model>,
     AllowedKinds extends FilterKind,
     Base,
 > = 'FullText' extends AllowedKinds
     ? ProviderSupportsFullText<Schema> extends true
         ? GetModelField<Schema, Model, Field>['fullText'] extends true
-            ? Base & {
+            ? ExtendObjectBranch<
+                  Base,
+                  {
                       fts?: FullTextFilterPayload;
-              }
+                  }
+              >
             : Base
         : Base
     : Base;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
type AddFuzzyFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> =
GetModelFieldType<Schema, Model, Field> extends 'String'
? 'Fuzzy' extends AllowedKinds
? ProviderSupportsFuzzy<Schema> extends true
? GetModelField<Schema, Model, Field>['fuzzy'] extends true
? Base & {
/**
* Performs a fuzzy search on the string field. Only available when
* the schema's provider is `postgresql` (requires `pg_trgm` extension)
* and the field is annotated with `@fuzzy` in the ZModel schema.
* See {@link FuzzyFilterPayload} for the full options reference.
*/
fuzzy?: FuzzyFilterPayload;
}
: Base
: Base
> = 'Fuzzy' extends AllowedKinds
? ProviderSupportsFuzzy<Schema> extends true
? GetModelField<Schema, Model, Field>['fuzzy'] extends true
? Base & {
/**
* Performs a fuzzy search on the string field. Only available when
* the schema's provider is `postgresql` (requires `pg_trgm` extension)
* and the field is annotated with `@fuzzy` in the ZModel schema.
* See {@link FuzzyFilterPayload} for the full options reference.
*/
fuzzy?: FuzzyFilterPayload;
}
: Base
: Base
: Base;
/**
* Conditionally augments a string-field filter with the `fts` operator when:
* 1. The `FullText` filter kind is allowed for this field, AND
* 2. The schema's provider supports full-text search (postgres only), AND
* 3. The field is annotated with `@fullText` in the ZModel schema.
*
* Caller is responsible for only invoking this on String-typed fields
* (the gate lives in `FieldFilter`).
*/
type AddFullTextFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> = 'FullText' extends AllowedKinds
? ProviderSupportsFullText<Schema> extends true
? GetModelField<Schema, Model, Field>['fullText'] extends true
? Base & {
/**
* Performs a full-text search on the string field. Only available when
* the schema's provider is `postgresql` and the field is annotated with
* `@fullText` in the ZModel schema.
* See {@link FullTextFilterPayload} for the full options reference.
*/
fts?: FullTextFilterPayload;
}
: Base
: Base;
: Base
: Base;
type ExtendObjectBranch<Base, Extra> = Base extends object ? Base & Extra : Base;
type AddFuzzyFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> = 'Fuzzy' extends AllowedKinds
? ProviderSupportsFuzzy<Schema> extends true
? GetModelField<Schema, Model, Field>['fuzzy'] extends true
? ExtendObjectBranch<
Base,
{
/**
* Performs a fuzzy search on the string field. Only available when
* the schema's provider is `postgresql` (requires `pg_trgm` extension)
* and the field is annotated with `@fuzzy` in the ZModel schema.
* See {`@link` FuzzyFilterPayload} for the full options reference.
*/
fuzzy?: FuzzyFilterPayload;
}
>
: Base
: Base
: Base;
/**
* Conditionally augments a string-field filter with the `fts` operator when:
* 1. The `FullText` filter kind is allowed for this field, AND
* 2. The schema's provider supports full-text search (postgres only), AND
* 3. The field is annotated with `@fullText` in the ZModel schema.
*
* Caller is responsible for only invoking this on String-typed fields
* (the gate lives in `FieldFilter`).
*/
type AddFullTextFilterIfSupported<
Schema extends SchemaDef,
Model extends GetModels<Schema>,
Field extends GetModelFields<Schema, Model>,
AllowedKinds extends FilterKind,
Base,
> = 'FullText' extends AllowedKinds
? ProviderSupportsFullText<Schema> extends true
? GetModelField<Schema, Model, Field>['fullText'] extends true
? ExtendObjectBranch<
Base,
{
/**
* Performs a full-text search on the string field. Only available when
* the schema's provider is `postgresql` and the field is annotated with
* `@fullText` in the ZModel schema.
* See {`@link` FullTextFilterPayload} for the full options reference.
*/
fts?: FullTextFilterPayload;
}
>
: Base
: Base
: Base;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/orm/src/client/crud-types.ts` around lines 416 - 467, The current
AddFuzzyFilterIfSupported and AddFullTextFilterIfSupported intersect the extra
operator bag directly into Base causing union distribution that drops primitive
shorthand (e.g., string); change the augmentation to only apply to object
branches by replacing "Base & { ... } : Base" with a distributive conditional
that checks whether Base extends object (or Record<string, unknown>) and only
then adds the fuzzy/fts operator (i.e., Base extends object ? Base & { fuzzy?:
FuzzyFilterPayload } : Base for AddFuzzyFilterIfSupported and similarly Base
extends object ? Base & { fts?: FullTextFilterPayload } : Base for
AddFullTextFilterIfSupported), keeping the existing guards that check
ProviderSupportsFuzzy/ProviderSupportsFullText and
GetModelField[...]['fuzzy'|'fullText'].

Comment on lines +709 to +719
// Document expression: a single field, or `concat_ws` of all fields when
// multi-field. `concat_ws` skips NULL arguments natively. Multi-field uses
// a single ts_rank over the combined document (matches Prisma; ensures
// AND queries match terms spread across fields).
const document =
fieldRefs.length === 1 ? fieldRefs[0]! : sql`concat_ws(' ', ${sql.join(fieldRefs)})`;

if (config === undefined) {
// No regconfig — Postgres uses default_text_search_config.
return query.orderBy(sql`ts_rank(to_tsvector(${document}), to_tsquery(${q}))`, sort);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

# First, let's locate and examine the PostgreSQL dialect file
find . -name "postgresql.ts" -type f | head -5

Repository: zenstackhq/zenstack

Length of output: 172


🏁 Script executed:

# Let's look at the actual code section mentioned
cat -n packages/orm/src/client/crud/dialects/postgresql.ts | sed -n '700,730p'

Repository: zenstackhq/zenstack

Length of output: 1451


🏁 Script executed:

# Search for the buildFtsRelevanceOrderBy function
rg -A 30 "buildFtsRelevanceOrderBy" packages/orm/src/client/crud/dialects/postgresql.ts

Repository: zenstackhq/zenstack

Length of output: 1234


🏁 Script executed:

# Check if optional string fields with `@fullText` are actually supported
# Look for full-text field validation or configuration
rg -B 5 -A 5 "@fullText" packages/orm/src/ --type ts

Repository: zenstackhq/zenstack

Length of output: 6838


🌐 Web query:

PostgreSQL ts_rank NULL handling ts_tsvector NULL behavior

💡 Result:

If the inputs to ts_rank are NULL, the result is NULL due to normal SQL NULL propagation. 1) NULL tsvector / NULL textsearch vector - PostgreSQL documentation explicitly warns that to_tsvector(NULL) returns NULL and recommends COALESCE when a field may be null. [1] - So if you call ts_rank with a NULL tsvector argument (or you create a NULL tsvector via to_tsvector(NULL)), you should expect ts_rank to return NULL (because the function cannot compute a rank from a NULL document vector). 2) NULL tsquery / NULL search query - Similarly, if you pass a NULL tsquery into ts_rank, the function cannot compute a match score; in practice, this yields NULL via normal SQL NULL propagation. 3) Practical takeaway (how to avoid NULL ranks) - Ensure your tsvector expression never becomes NULL by coalescing nullable inputs to empty strings when building the tsvector, e.g.: - setweight(to_tsvector(coalesce(title, '')), 'A') || setweight(to_tsvector(coalesce(body, '')), 'B') - PostgreSQL docs show this COALESCE pattern when fields might be null. [1] 4) Difference vs “rank 0” - A non-NULL tsvector that contains no matching lexemes will produce a score of 0 (or it will not match @@ and therefore won’t be ranked in the usual query pattern). [1] - NULL inputs are different: they produce NULL, not 0. Sources: PostgreSQL documentation (text search controls) explicitly notes to_tsvector(NULL) returns NULL and recommends COALESCE for nullable fields. [1]

Citations:


🌐 Web query:

PostgreSQL ORDER BY DESC NULL values sorting position

💡 Result:

In PostgreSQL, when using ORDER BY column DESC without specifying NULLS position, NULL values are placed first (at the beginning of the result set), before all non-NULL values. This is the default behavior for DESC order.[1][2][3][4] PostgreSQL treats NULL values as larger than any non-NULL value for sorting purposes. Therefore: - For ASC (or default ASC): NULLS LAST (NULLs at the end).[1][2] - For DESC: NULLS FIRST (NULLs at the start).[1][2] You can explicitly control this with NULLS FIRST or NULLS LAST clauses: SELECT * FROM table ORDER BY column DESC NULLS LAST; This would place NULLs after non-NULL values even in DESC order.[1][2] This behavior is consistent across recent versions, including 15, 18, and devel (as of 2026).[1][2][5][4]

Citations:


Coalesce nullable fields before computing ts_rank in single-field branch.

When a String field with @fullText is optional, to_tsvector(NULL) returns NULL, causing ts_rank to also return NULL. PostgreSQL's DESC ordering places NULLs first, so documents with missing field values rank ahead of actual matches. Apply the same null-skipping logic as the multi-field branch by coalescing the single field to an empty string.

Suggested fix
-        const document =
-            fieldRefs.length === 1 ? fieldRefs[0]! : sql`concat_ws(' ', ${sql.join(fieldRefs)})`;
+        const document =
+            fieldRefs.length === 1 ? sql`coalesce(${fieldRefs[0]!}, '')` : sql`concat_ws(' ', ${sql.join(fieldRefs)})`;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/orm/src/client/crud/dialects/postgresql.ts` around lines 709 - 719,
The single-field branch builds `document` from `fieldRefs[0]` but doesn't
coalesce nullable strings, so `to_tsvector(${document})` can yield NULL and make
`ts_rank` return NULL; update the single-field branch that defines `document`
(the `document` variable used with `to_tsvector`/`ts_rank` and in
`query.orderBy`) to coalesce the single field to an empty string (same
null-skipping behavior as the multi-field `concat_ws` path) before passing it to
`to_tsvector` and `ts_rank`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant