Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/en/appendices/5-4-migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,13 @@ explicitly set `'strategy' => 'select'` when defining associations.
`FormProtectionComponent`.
See [Form Protection Component](../controllers/components/form-protection).

### Http

- Added `JsonStreamResponse` class for memory-efficient streaming of large JSON
datasets using generators. Supports standard JSON arrays and NDJSON formats,
envelope structures with metadata, transform callbacks, and graceful mid-stream
error handling. See [Streaming JSON Responses](../controllers/request-response#streaming-json-responses).

### Database

- Added `notBetween()` method for `NOT BETWEEN` expressions.
Expand Down
137 changes: 137 additions & 0 deletions docs/en/controllers/request-response.md
Original file line number Diff line number Diff line change
Expand Up @@ -812,6 +812,143 @@ public function sendIcs()
}
```

<a id="streaming-json-responses"></a>

### Streaming JSON Responses

`class` Cake\\Http\\Response\\**JsonStreamResponse**

When working with large datasets, loading everything into memory before encoding
to JSON can exhaust available memory. `JsonStreamResponse` provides memory-efficient
streaming of JSON data using generators, keeping only one item in memory at a time.

::: info Added in version 5.4.0
:::

#### Basic Usage

```php
use Cake\Http\Response\JsonStreamResponse;

public function index()
{
$query = $this->Articles->find();

// Simple array streaming
return new JsonStreamResponse($query);
// Output: [{"id":1,"title":"First"},{"id":2,"title":"Second"},...]
}
```

#### Constructor Options

The `JsonStreamResponse` constructor accepts an iterable and an options array:

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `root` | `string\|null` | `null` | Wrap data in `{"root": [...]}` |
| `envelope` | `array` | `[]` | Static metadata merged with streaming data |
| `dataKey` | `string` | `'data'` | Key for streaming data when envelope is used |
| `format` | `string` | `'json'` | Output format: `'json'` or `'ndjson'` |
| `transform` | `callable\|null` | `null` | Transform each item before encoding |
| `flags` | `int` | `DEFAULT_JSON_FLAGS` | JSON encode flags |

#### With Root Wrapper

Wrap the array in an object with a named key:

```php
return new JsonStreamResponse($query, ['root' => 'articles']);
// Output: {"articles":[{"id":1,"title":"First"},{"id":2,"title":"Second"}]}
```

#### With Envelope (Metadata)

Include static metadata alongside the streaming data:

```php
$total = $this->Articles->find()->count();

return new JsonStreamResponse($query, [
'envelope' => ['meta' => ['total' => $total, 'page' => 1]],
'dataKey' => 'articles',
]);
// Output: {"meta":{"total":100,"page":1},"articles":[{"id":1,"title":"First"},...]}
```

#### NDJSON Format

[NDJSON](http://ndjson.org/) (Newline Delimited JSON) outputs one JSON object per
line, useful for streaming to clients that process data incrementally:

```php
return new JsonStreamResponse($query, ['format' => 'ndjson']);
// Output:
// {"id":1,"title":"First"}
// {"id":2,"title":"Second"}
```

The content type is automatically set to `application/x-ndjson; charset=UTF-8`.

#### Transform Callback

Transform each item before JSON encoding. Useful for selecting specific fields
or formatting data:

```php
return new JsonStreamResponse($query, [
'transform' => fn($article) => [
'id' => $article->id,
'title' => $article->title,
'url' => Router::url(['action' => 'view', $article->id]),
],
]);
```

#### Immutability

`JsonStreamResponse` follows PSR-7 immutability patterns. Use `withStreamOptions()`
to create a modified copy:

```php
$response = new JsonStreamResponse($query);
$newResponse = $response->withStreamOptions(['root' => 'articles']);
```

#### Error Handling

`JsonStreamResponse` uses a three-layer error handling strategy:

1. **Pre-validation**: The first item is encoded before output starts. If encoding
fails, an exception is thrown and a proper error response can be returned.

2. **Mid-stream error marker**: If item N (where N > 1) fails to encode, an error
marker is output to maintain valid JSON structure:

```json
[{"id":1},{"__streamError":{"message":"Type is not supported","index":1}}]
```

3. **Server-side logging**: All encoding failures are logged via `Log::error()`.

#### ORM Integration

For true memory-efficient streaming, use unbuffered queries and avoid result
formatters:

```php
// Good - streams one row at a time
$query = $this->Articles->find()->bufferResults(false);
return new JsonStreamResponse($query);

// Avoid - formatters like map(), combine() buffer results internally
$query = $this->Articles->find()->map(fn($row) => $row); // Breaks streaming
```

> [!NOTE]
> Result formatters (`map()`, `combine()`, etc.) buffer results internally,
> which defeats the memory-efficient streaming purpose.

### Setting Headers

`method` Cake\\Http\\Response::**withHeader**(string $name, string|array $value): static
Expand Down
Loading