Add payload transforms documentation

documentation/configuration/configuration-utils/_http.config.json

@@ -266,5 +266,9 @@
   "http.redirect.1": {
     "default": "/ -> /index.html",
     "description": "Example redirect configuration. Format is 'source -> destination'."
+  },
+  "http.ingest.max.request.size": {
+    "default": "5M",
+    "description": "Maximum request body size for the `/ingest` endpoint used by [payload transforms](/docs/ingestion/payload-transforms/). The entire request body is held in memory during processing. Requests exceeding this limit receive an HTTP 413 (Payload Too Large) response."
   }
 }

documentation/ingestion/payload-transforms.md

@@ -0,0 +1,195 @@
+---
+title: Payload transforms
+sidebar_label: Payload Transforms
+description:
+  Guide to payload transforms in QuestDB, which parse and insert HTTP payloads
+  into tables using SQL expressions without middleware.
+---
+
+Payload transforms define how incoming HTTP payloads are parsed, transformed, and
+inserted into a QuestDB table. You define a transform once with a SQL `SELECT`
+expression, then POST data directly to QuestDB. No middleware or intermediary
+service is required.
+
+Use cases include webhook ingestion, IoT device data, and external API responses
+where you want to skip building a dedicated ingestion service.
+
+For full SQL syntax, see
+[CREATE PAYLOAD TRANSFORM](/docs/query/sql/create-payload-transform/) and
+[DROP PAYLOAD TRANSFORM](/docs/query/sql/drop-payload-transform/).
+
+## Example: Binance order book snapshots
+
+Store full order book snapshots from the Binance depth API, which returns JSON
+like:
+
+```json
+{
+  "bids": [["65000.01","0.5"], ["64999.99","1.2"]],
+  "asks": [["65000.02","0.3"], ["65000.05","0.8"]]
+}
+```
+
+Create the target table and the transform:
+
+```questdb-sql title="Table and transform definition"
+CREATE TABLE order_book (
+    ts TIMESTAMP,
+    symbol SYMBOL,
+    bids DOUBLE[][],
+    asks DOUBLE[][]
+) TIMESTAMP(ts) PARTITION BY DAY WAL;
+
+CREATE PAYLOAD TRANSFORM binance_depth
+INTO order_book
+DLQ dlq_errors PARTITION BY DAY TTL 7 DAYS
+AS DECLARE OVERRIDABLE @symbol := 'BTCUSDT'
+SELECT
+    now() AS ts,
+    @symbol AS symbol,
+    json_extract(payload(), '$.bids')::DOUBLE[][] AS bids,
+    json_extract(payload(), '$.asks')::DOUBLE[][] AS asks;
+```
+
+Ingest a snapshot:
+
+```shell title="POST a payload"
+curl -s "https://api.binance.com/api/v3/depth?symbol=BTCUSDT&limit=5" | \
+  curl -X POST "http://localhost:9000/ingest?transform=binance_depth" -d @-
+```
+
+Response:
+
+```json
+{"status": "ok", "rows_inserted": 1}
+```
+
+### Overriding variables
+
+The `@symbol` variable is declared `OVERRIDABLE`, so you can override it per
+request via URL query parameters:
+
+```shell title="Override a variable"
+curl -s "https://api.binance.com/api/v3/depth?symbol=ETHUSDT&limit=5" | \
+  curl -X POST "http://localhost:9000/ingest?transform=binance_depth&symbol=ETHUSDT" -d @-
+```
+
+Any URL query parameter other than `transform` is matched to a
+`DECLARE OVERRIDABLE` variable by name. Variables not marked `OVERRIDABLE`
+cannot be overridden - attempting to do so returns an error.
+
+### Inspecting failed payloads
+
+When a payload fails (bad JSON, type mismatch, missing columns), QuestDB writes
+the original payload, the error stage, and the error message to the DLQ table
+configured in the transform:
+
+```questdb-sql title="Query the DLQ"
+SELECT ts, transform_name, stage, error FROM dlq_errors;
+```
+
+| ts | transform_name | stage | error |
+| :--- | :--- | :--- | :--- |
+| 2026-03-23T14:00:00.000000Z | binance_depth | transform | column not found in target table [column=extra] |
+| 2026-03-23T14:01:00.000000Z | other_transform | transform | bad JSON payload |
+
+Multiple transforms can share the same DLQ table. See
+[CREATE PAYLOAD TRANSFORM](/docs/query/sql/create-payload-transform/#dead-letter-queue-schema)
+for the full DLQ schema.
+
+## HTTP endpoint
+
+**POST** `/ingest`
+
+### Query parameters
+
+| Parameter | Required | Description |
+| :--- | :--- | :--- |
+| `transform` | Yes | Name of the payload transform to execute |
+| Any other | No | Overrides a `DECLARE OVERRIDABLE` variable by name |
+
+The request body is the raw payload, accessible via `payload()` in the transform
+SQL.
+
+### Responses
+
+Success:
+
+```json
+{"status": "ok", "rows_inserted": 1}
+```
+
+Error:
+
+```json
+{"status": "error", "message": "..."}
+```
+
+## Permissions
+
+In QuestDB Open Source, any user with access to the HTTP endpoint can create
+transforms and invoke `/ingest`.
+
+:::note Enterprise
+
+In [QuestDB Enterprise](/enterprise/) deployments with
+[RBAC](/docs/security/rbac/) enabled, the following grants are required:
+
+| Action | Required grants |
+| :----- | :-------------- |
+| Create a transform | `CREATE PAYLOAD TRANSFORM` and `INSERT` on the target table (and DLQ table, if configured) |
+| Replace a transform (`OR REPLACE`) | `CREATE PAYLOAD TRANSFORM` and `DROP PAYLOAD TRANSFORM` |
+| Drop a transform | `DROP PAYLOAD TRANSFORM` |
+| Invoke `/ingest` | `HTTP` endpoint grant and `INSERT` on the target table |
+
+```questdb-sql title="Typical Enterprise setup"
+-- Admin who manages transforms
+GRANT CREATE PAYLOAD TRANSFORM, DROP PAYLOAD TRANSFORM TO ingest_admin;
+GRANT INSERT ON order_book, dlq_errors TO ingest_admin;
+
+-- Service account that calls /ingest
+GRANT HTTP TO ingest_service;
+GRANT INSERT ON order_book TO ingest_service;
+```
+
+:::
+
+## Request size limit
+
+The `/ingest` endpoint rejects request bodies that exceed a configurable maximum
+size. The default limit is 5 MB. To change it, set the
+`http.ingest.max.request.size` property in `server.conf`:
+
+```ini title="server.conf"
+http.ingest.max.request.size=10M
+```
+
+Requests exceeding the limit receive an HTTP 413 (Payload Too Large) response.
+The entire request body is held in memory during processing, so set this limit
+based on available memory and expected payload sizes.
+
+## Limitations
+
+- **Single payload per request** - Each HTTP request executes the transform
+  once. That execution may produce multiple rows. Sending multiple
+  independent payload documents in a single request is not supported.
+- **Per-request SQL compilation** - Transform SQL is compiled on every request.
+  This is acceptable for low-rate ingestion workloads. Compiled-plan caching is
+  a planned optimization.
+- **No table references** - The transform SELECT must not reference existing
+  tables. It can only use functions and expressions, including CTEs.
+- **SELECT only** - Only `SELECT` statements are allowed. `INSERT`, `UPDATE`,
+  and other statements are rejected at creation time.
+- **Schema drift** - Column names and types are validated against the target
+  table at creation time. Schema changes to the target table after creating a
+  transform may cause runtime errors.
+- **Concurrent DDL** - `CREATE`, `DROP`, and `OR REPLACE` for the same
+  transform name are not serialized. If two sessions operate on the same
+  transform name concurrently, the outcome is last-writer-wins.
+
+:::info Related documentation
+- [CREATE PAYLOAD TRANSFORM](/docs/query/sql/create-payload-transform/)
+- [DROP PAYLOAD TRANSFORM](/docs/query/sql/drop-payload-transform/)
+- [JSON functions](/docs/query/functions/json/)
+- [REST API](/docs/query/rest-api/)
+:::

documentation/query/rest-api.md

@@ -27,6 +27,7 @@ The [Web Console](/docs/getting-started/web-console/overview/) is the official W
 - [`/imp`](#imp---import-data) for importing data from `.CSV` files
 - [`/exec`](#exec---execute-queries) to execute a SQL statement
 - [`/exp`](#exp---export-data) to export data
+- [`/ingest`](#ingest---payload-transforms) to ingest data via payload transforms
 
 ## Examples
 
@@ -38,6 +39,7 @@ insert-capable entrypoints:
 | :----------------------------------------- | :---------- | :-------------------------------------- | :------------------------------------------------------------ |
 | [`/imp`](#imp-uploading-tabular-data)      | POST        | Import CSV data                         | [Reference](/docs/query/rest-api/#imp---import-data)      |
 | [`/exec?query=..`](#exec-sql-insert-query) | GET         | Run SQL Query returning JSON result set | [Reference](/docs/query/rest-api/#exec---execute-queries) |
+| [`/ingest?transform=..`](#ingest---payload-transforms) | POST | Execute a payload transform | [Reference](/docs/query/rest-api/#ingest---payload-transforms) |
 
 For details such as content type, query parameters and more, refer to the
 [REST API](/docs/query/rest-api/) docs.
@@ -688,6 +690,60 @@ curl -G \
   http://localhost:9000/exp > recent_trades.parquet
 ```
 
+## /ingest - Payload transforms
+
+`/ingest` executes a [payload transform](/docs/ingestion/payload-transforms/)
+against the raw HTTP request body and inserts the resulting rows into the
+transform's target table.
+
+### Overview
+
+`/ingest` expects an HTTP POST request with the raw payload as the request body.
+The `Content-Type` header is not enforced - the body is passed as-is to the
+transform's `payload()` function.
+
+#### Parameters
+
+| Parameter | Required | Description |
+| :--- | :--- | :--- |
+| `transform` | Yes | Name of the payload transform to execute |
+| Any other | No | Overrides a `DECLARE OVERRIDABLE` variable by name |
+
+#### Responses
+
+Success (HTTP 200):
+
+```json
+{"status": "ok", "rows_inserted": 1}
+```
+
+Error (HTTP 400 or 413):
+
+```json
+{"status": "error", "message": "..."}
+```
+
+Requests exceeding the configured `http.ingest.max.request.size` (default 5 MB)
+receive an HTTP 413 (Payload Too Large) response.
+
+### Example
+
+```shell title="Ingest a JSON payload"
+curl -X POST "http://localhost:9000/ingest?transform=binance_depth" \
+  -d '{"bids":[["65000.01","0.5"]],"asks":[["65000.02","0.3"]]}'
+```
+
+For full documentation on creating and managing transforms, see
+[Payload transforms](/docs/ingestion/payload-transforms/).
+
+:::note Enterprise
+
+In [QuestDB Enterprise](/enterprise/) deployments with
+[RBAC](/docs/security/rbac/) enabled, the caller must hold the `HTTP` endpoint
+grant and `INSERT` permission on the target table.
+
+:::
+
 ## Error responses
 
 ### Malformed queries