REST Section Migration

migration-context/link-placeholders/rest-link-placeholders.md

@@ -0,0 +1,39 @@
+# Link Placeholders for REST Section
+
+## reference_versioned_docs/version-v4/rest/overview.md
+
+- Line (See Also): `[Database / Schema](TODO:reference_versioned_docs/version-v4/database/schema.md 'Schema definition')`
+  - Context: Intro and See Also section — how to define and export resources
+  - Target should be: Database / Schema page
+  - **Status**: PENDING (Database section migration)
+
+## reference_versioned_docs/version-v4/rest/querying.md
+
+- Line (directURLMapping section): `[Database / Schema](TODO:reference_versioned_docs/version-v4/database/schema.md 'Schema and resource configuration')`
+  - Context: directURLMapping option reference
+  - Target should be: Database / Schema page
+  - **Status**: PENDING (Database section migration)
+
+- Line (See Also): `[Database / Schema](TODO:reference_versioned_docs/version-v4/database/schema.md 'Schema definition')`
+  - Context: See Also section
+  - Target should be: Database / Schema page
+  - **Status**: PENDING (Database section migration)
+
+## reference_versioned_docs/version-v4/rest/websockets.md
+
+- Line (Custom connect() Handler): `[Resource API](TODO:reference_versioned_docs/version-v4/resources/resource-api.md 'Resource API reference')`
+  - Context: Inline link for more on implementing custom resources
+  - Target should be: Resources / Resource API page
+  - **Status**: PENDING (Resources section migration)
+
+- Line (See Also): `[Resources](TODO:reference_versioned_docs/version-v4/resources/overview.md 'Resources overview')`
+  - Context: Link to custom resource API including `connect()` method
+  - Target should be: Resources section overview page
+  - **Status**: PENDING (Resources section migration)
+
+## reference_versioned_docs/version-v4/rest/server-sent-events.md
+
+- Line (See Also): `[Resources](TODO:reference_versioned_docs/version-v4/resources/overview.md 'Resources overview')`
+  - Context: Link to custom resource API including `connect()` method
+  - Target should be: Resources section overview page
+  - **Status**: PENDING (Resources section migration)

reference_versioned_docs/version-v4/rest/content-types.md

@@ -0,0 +1,100 @@
+---
+title: Content Types
+---
+
+<!-- Source: versioned_docs/version-4.7/reference/content-types.md (primary) -->
+<!-- Source: versioned_docs/version-4.7/developers/rest.md (content negotiation section) -->
+
+# Content Types
+
+Harper supports multiple content types (MIME types) for both HTTP request bodies and response bodies. Harper follows HTTP standards: use the `Content-Type` request header to specify the encoding of the request body, and use the `Accept` header to request a specific response format.
+
+```http
+Content-Type: application/cbor
+Accept: application/cbor
+```
+
+All content types work with any standard Harper operation.
+
+## Supported Formats
+
+### JSON — `application/json`
+
+JSON is the most widely used format, readable and easy to work with. It is well-supported across all HTTP tooling.
+
+**Limitations**: JSON does not natively support all Harper data types — binary data, `Date`, `Map`, and `Set` values require special handling. JSON also produces larger payloads than binary formats.
+
+**When to use**: Web development, debugging, interoperability with third-party clients, or when the standard JSON type set is sufficient. Pairing JSON with compression (`Accept-Encoding: gzip, br`) often yields compact network transfers due to favorable Huffman coding characteristics.
+
+### CBOR — `application/cbor`
+
+CBOR is the recommended format for most production use cases. It is a highly efficient binary format with native support for the full range of Harper data types, including binary data, typed dates, and explicit Maps/Sets.
+
+**Advantages**: Very compact encoding, fast serialization, native streaming support (indefinite-length arrays for optimal time-to-first-byte on query results). Well-standardized with growing ecosystem support.
+
+**When to use**: Production APIs, performance-sensitive applications, or any scenario requiring rich data types.
+
+### MessagePack — `application/x-msgpack`
+
+MessagePack is another efficient binary format similar to CBOR, with broader adoption in some ecosystems. It supports all Harper data types.
+
+**Limitations**: MessagePack does not natively support streaming arrays, so query results are returned as a concatenated sequence of MessagePack objects. Decoders must be prepared to handle a sequence of values rather than a single document.
+
+**When to use**: Systems with existing MessagePack support that don't have CBOR available, or when interoperability with MessagePack clients is required. CBOR is generally preferred when both are available.
+
+### CSV — `text/csv`
+
+Comma-separated values format, suitable for data export and spreadsheet import/export. CSV lacks hierarchical structure and explicit typing.
+
+**When to use**: Ad-hoc data export, spreadsheet workflows, batch data processing. Not recommended for frequent or production API use.
+
+## Content Type via URL Extension
+
+As an alternative to the `Accept` header, responses can be requested in a specific format using file-style URL extensions:
+
+```http
+GET /product/some-id.csv
+GET /product/.msgpack?category=software
+```
+
+Using the `Accept` header is the recommended approach for clean, standard HTTP interactions.
+
+## Custom Content Types
+
+Harper's content type system is extensible. Custom handlers for any serialization format (XML, YAML, proprietary formats, etc.) can be registered in the [`contentTypes`](../resources/global-apis.md) global Map.
+
+## Storing Arbitrary Content Types
+
+When a `PUT` or `POST` is made with a non-standard content type (e.g., `text/calendar`, `image/gif`), Harper stores the content as a record with `contentType` and `data` properties:
+
+```http
+PUT /my-resource/33
+Content-Type: text/calendar
+
+BEGIN:VCALENDAR
+VERSION:2.0
+...
+```
+
+This stores a record equivalent to:
+
+```json
+{ "contentType": "text/calendar", "data": "BEGIN:VCALENDAR\nVERSION:2.0\n..." }
+```
+
+Retrieving a record that has `contentType` and `data` properties returns the response with the specified `Content-Type` and body. If the content type is not from the `text` family, the data is treated as binary (a Node.js `Buffer`).
+
+Use `application/octet-stream` for binary data or for uploading to a specific property:
+
+```http
+PUT /my-resource/33/image
+Content-Type: image/gif
+
+...image data...
+```
+
+## See Also
+
+- [REST Overview](./overview.md) — HTTP methods and URL structure
+- [Headers](./headers.md) — Content negotiation headers
+- [Querying](./querying.md) — URL query syntax

reference_versioned_docs/version-v4/rest/headers.md

@@ -0,0 +1,97 @@
+---
+title: REST Headers
+---
+
+<!-- Source: versioned_docs/version-4.7/reference/headers.md (primary) -->
+<!-- Source: versioned_docs/version-4.7/developers/rest.md (ETag/conditional request headers) -->
+
+# REST Headers
+
+Harper's REST interface uses standard HTTP headers for content negotiation, caching, and performance instrumentation.
+
+## Response Headers
+
+These headers are included in all Harper REST API responses:
+
+| Header          | Example Value      | Description                                                                                                                                                                                               |
+| --------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `server-timing` | `db;dur=7.165`     | Duration of the operation in milliseconds. Follows the [Server-Timing](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing) standard and can be consumed by network monitoring tools. |
+| `content-type`  | `application/json` | MIME type of the returned content, negotiated based on the `Accept` request header.                                                                                                                       |
+| `etag`          | `"abc123"`         | Encoded version/last-modification time of the returned record. Used for conditional requests.                                                                                                             |
+| `location`      | `/MyTable/new-id`  | Returned on `POST` responses. Contains the path to the newly created record.                                                                                                                              |
+
+## Request Headers
+
+### Content-Type
+
+Specifies the format of the request body (for `PUT`, `PATCH`, `POST`):
+
+```http
+Content-Type: application/json
+Content-Type: application/cbor
+Content-Type: application/x-msgpack
+Content-Type: text/csv
+```
+
+See [Content Types](./content-types.md) for the full list of supported formats.
+
+### Accept
+
+Specifies the preferred response format:
+
+```http
+Accept: application/json
+Accept: application/cbor
+Accept: application/x-msgpack
+Accept: text/csv
+```
+
+### If-None-Match
+
+Used for conditional GET requests. Provide the `ETag` value from a previous response to avoid re-fetching unchanged data:
+
+```http
+GET /MyTable/123
+If-None-Match: "abc123"
+```
+
+If the record has not changed, Harper returns `304 Not Modified` with no body. This avoids serialization and network transfer overhead and works seamlessly with browser caches and external HTTP caches.
+
+### Accept-Encoding
+
+Harper supports standard HTTP compression. Including this header enables compressed responses:
+
+```http
+Accept-Encoding: gzip, br
+```
+
+Compression is particularly effective for JSON responses. For binary formats like CBOR, compression provides diminishing returns compared to the already-compact encoding.
+
+### Authorization
+
+Credentials for authenticating requests. See [Security Overview](../security/overview.md) for details on supported authentication mechanisms (Basic, JWT, mTLS).
+
+### Sec-WebSocket-Protocol
+
+When connecting via WebSocket for MQTT, the sub-protocol must be set to `mqtt` as required by the MQTT specification:
+
+```http
+Sec-WebSocket-Protocol: mqtt
+```
+
+## Content Type via URL Extension
+
+As an alternative to the `Accept` header, content types can be specified using file-style extensions in the URL path:
+
+```http
+GET /product/some-id.csv
+GET /product/.msgpack?category=software
+```
+
+This is not recommended for production use — prefer the `Accept` header for clean, standard HTTP interactions.
+
+## See Also
+
+- [REST Overview](./overview.md) — HTTP methods and URL structure
+- [Content Types](./content-types.md) — Supported encoding formats
+- [Security Overview](../security/overview.md) — Authentication headers and mechanisms

reference_versioned_docs/version-v4/rest/overview.md

@@ -0,0 +1,159 @@
+---
+title: REST Overview
+---
+
+<!-- Source: versioned_docs/version-4.7/developers/rest.md (primary) -->
+<!-- Source: versioned_docs/version-4.7/reference/components/built-in-extensions.md (rest plugin config options) -->
+<!-- Source: release-notes/v4-tucker/4.2.0.md (REST interface introduced) -->
+<!-- Source: release-notes/v4-tucker/4.5.0.md (URL path improvements, directURLMapping, record count removal) -->
+
+# REST Overview
+
+Added in: v4.2.0
+
+Harper provides a powerful, efficient, and standard-compliant HTTP REST interface for interacting with tables and other resources. The REST interface is the recommended interface for data access, querying, and manipulation over HTTP, providing the best performance and HTTP interoperability with different clients.
+
+## How the REST Interface Works
+
+Harper's REST interface exposes database tables and custom resources as RESTful endpoints. Tables are **not** exported by default; they must be explicitly exported in a schema definition. The name of the exported resource defines the base of the endpoint path, served on the application HTTP server port (default `9926`).
+
+For more on defining schemas and exporting resources, see [TODO:reference_versioned_docs/version-v4/database/schema.md 'Schema definition'].
+
+## Configuration
+
+Enable the REST interface by adding the `rest` plugin to your application's `config.yaml`:
+
+```yaml
+rest: true
+```
+
+**Options**:
+
+```yaml
+rest:
+  lastModified: true # enables Last-Modified response header support
+  webSocket: false # disables automatic WebSocket support (enabled by default)
+```
+
+## URL Structure
+
+The REST interface follows a consistent URL structure:
+
+| Path                                         | Description                                                                                       |
+| -------------------------------------------- | ------------------------------------------------------------------------------------------------- |
+| `/my-resource`                               | Root path — returns a description of the resource (e.g., table metadata)                          |
+| `/my-resource/`                              | Trailing slash indicates a collection — represents all records; append query parameters to search |
+| `/my-resource/record-id`                     | A specific record identified by its primary key                                                   |
+| `/my-resource/record-id/`                    | Trailing slash — the collection of records with the given id prefix                               |
+| `/my-resource/record-id/with/multiple/parts` | Record id with multiple path segments                                                             |
+
+Changed in: v4.5.0 — Resources can be defined with nested paths and accessed by exact path without a trailing slash. The `id.property` dot syntax for accessing properties via URL is only applied to properties declared in a schema.
+
+## HTTP Methods
+
+REST operations map to HTTP methods following uniform interface principles:
+
+### GET
+
+Retrieve a record or perform a search. Handled by the resource's `get()` method.
+
+```http
+GET /MyTable/123
+```
+
+Returns the record with primary key `123`.
+
+```http
+GET /MyTable/?name=Harper
+```
+
+Returns records matching `name=Harper`. See [Querying](./querying.md) for the full query syntax.
+
+```http
+GET /MyTable/123.propertyName
+```
+
+Returns a single property of a record. Only works for properties declared in the schema.
+
+#### Conditional Requests and Caching
+
+GET responses include an `ETag` header encoding the record's version/last-modification time. Clients with a cached copy can include `If-None-Match` on subsequent requests. If the record hasn't changed, Harper returns `304 Not Modified` with no body — avoiding serialization and network transfer overhead.
+
+### PUT
+
+Create or replace a record with a specified primary key (upsert semantics). Handled by the resource's `put(record)` method. The stored record will exactly match the submitted body — any properties not included in the body are removed from the previous record.
+
+```http
+PUT /MyTable/123
+Content-Type: application/json
+
+{ "name": "some data" }
+```
+
+Creates or replaces the record with primary key `123`.
+
+### POST
+
+Create a new record without specifying a primary key, or trigger a custom action. Handled by the resource's `post(data)` method. The auto-assigned primary key is returned in the `Location` response header.
+
+```http
+POST /MyTable/
+Content-Type: application/json
+
+{ "name": "some data" }
+```
+
+### PATCH
+
+Partially update a record, merging only the provided properties (CRDT-style update). Unspecified properties are preserved.
+
+Added in: v4.3.0
+
+```http
+PATCH /MyTable/123
+Content-Type: application/json
+
+{ "status": "active" }
+```
+
+### DELETE
+
+Delete a specific record or all records matching a query.
+
+```http
+DELETE /MyTable/123
+```
+
+Deletes the record with primary key `123`.
+
+```http
+DELETE /MyTable/?status=archived
+```
+
+Deletes all records matching `status=archived`.
+
+## Content Types
+
+Harper supports multiple content types for both request bodies and responses. Use the `Content-Type` header for request bodies and the `Accept` header to request a specific response format.
+
+See [Content Types](./content-types.md) for the full list of supported formats and encoding recommendations.
+
+## OpenAPI
+
+Added in: v4.3.0
+
+Harper automatically generates an OpenAPI specification for all resources exported via a schema. This endpoint is available at:
+
+```http
+GET /openapi
+```
+
+## See Also
+
+- [Querying](./querying.md) — Full URL query syntax, operators, and examples
+- [Headers](./headers.md) — HTTP headers used by the REST interface
+- [Content Types](./content-types.md) — Supported formats (JSON, CBOR, MessagePack, CSV)
+- [WebSockets](./websockets.md) — Real-time connections via WebSocket
+- [Server-Sent Events](./server-sent-events.md) — One-way streaming via SSE
+- [HTTP Server](../http/overview.md) — Underlying HTTP server configuration
+- [Database / Schema](TODO:reference_versioned_docs/version-v4/database/schema.md 'Schema definition') — How to define and export resources