Allow optional output schema fields

[gnzjgo]

Summary

Fixes #147.

• Adds .optional() to Tinybird type validators so endpoint output fields can be marked as absent from some responses without changing their Tinybird/ClickHouse type.
• Updates InferOutput and InferOutputRow so optional output validators become optional object properties.
• Allows schema validation to accept missing optional output columns while still flagging missing required columns.
• Documents optional output fields in the README with a templated endpoint example.

Documentation / changelog note

Endpoint output schemas now support optional fields for templated responses where a column may or may not be selected depending on request params. The SDK does not generate the template condition automatically; .optional() documents and types the response contract so schema validation accepts both shapes.

Example:

export const topEvents = defineEndpoint("top_events", {
  params: {
    include_debug: p.string().optional("0"),
  },
  nodes: [
    node({
      name: "endpoint",
      sql: `
        SELECT
          event_name,
          count() AS event_count
          {% if include_debug == "1" %}
          , concat('debug:', event_name) AS debug_info
          , toNullable(concat('debug:', event_name)) AS nullable_debug_info
          {% end %}
        FROM events
        GROUP BY event_name
      `,
    }),
  ],
  output: {
    event_name: t.string(),
    event_count: t.uint64(),
    debug_info: t.string().optional(),
    nullable_debug_info: t.string().nullable().optional(),
  },
});

This infers:

type Row = {
  event_name: string;
  event_count: number;
  debug_info?: string;
  nullable_debug_info?: string | null;
};

How to test

Run with Node 20 via nvm:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
nvm use 20
pnpm vitest run src/schema/types.test.ts src/schema/pipe.test.ts src/cli/utils/schema-validation.test.ts
pnpm typecheck

Manual reviewer check

Create an endpoint output schema with optional fields backed by templated SQL as above.

Both rows should type-check:

type Row = InferOutputRow<typeof topEvents>;

const withoutDebug: Row = {
  event_name: "page_view",
  event_count: 42,
};

const withDebug: Row = {
  event_name: "page_view",
  event_count: 42,
  debug_info: "debug:page_view",
  nullable_debug_info: null,
};

Live Tinybird validation

Validated against Tinybird Cloud workspace gnzoptionalsdk with deployed resources:

• sdk_optional_output_events
• sdk_optional_output_endpoint

The endpoint was deployed with a templated include_debug param and queried both ways.

include_debug: "0" response shape:

{
  "keys": ["event_count", "event_name"],
  "has_debug_info": false,
  "has_nullable_debug_info": false
}

include_debug: "1" response shape:

{
  "keys": ["debug_info", "event_count", "event_name", "nullable_debug_info"],
  "has_debug_info": true,
  "has_nullable_debug_info": true
}

Additional regression covered

After testing an upgrade scenario, we also verified that param metadata/default rewriting is only applied to Tinybird typed parameter template functions (String, DateTime, Int32, etc.). Non-param helper functions such as split_to_array(...) and column(...) must be preserved as-is because Tinybird rejects generated keywords like required=False or description=... for those helpers.

Regression example:

params: {
  siteIds: p.string().optional().describe("Comma-separated list of framerSiteId. Only used for test isolation"),
},
sql: `{{ split_to_array(siteIds) }}`,

Expected generated SQL keeps the helper unchanged:

{{ split_to_array(siteIds) }}

[juliavallina]

Tested and working as expected.
I bumped the version to generate a new release