Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 33 additions & 5 deletions src/content/docs/ai-gateway/features/dlp/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,42 @@

The system works with all AI providers supported by AI Gateway, providing consistent protection regardless of which models or services you use.

### Inspection scope

DLP inspects the text content of request and response bodies as they pass through AI Gateway. The following details apply:

- **Non-streaming requests and responses**: DLP scans the full request and response body.
- **Streaming (SSE) responses**: DLP buffers the full streamed response before scanning. This means DLP-scanned streaming responses are not delivered incrementally to the client. Expect increased time-to-first-token latency when DLP response scanning is enabled on streaming requests, because the entire response must be received from the provider before DLP can evaluate it and release it to the client.
- **Tool call arguments and results**: DLP scans the text content present in the message body, which includes tool call arguments and results if they appear in the JSON request or response payload.
- **Base64-encoded images and file attachments**: DLP does not decode base64-encoded content or follow external URLs. Only the raw text of the request and response body is inspected.
- **Multipart form data**: DLP scans the text portions of the request body. Binary data within multipart payloads is not inspected.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part needs some auditing based on looking at underlying implementation, I don't trust my direct knowledge on DLP mechanics to be approver

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By my inspection, this all seems accurate. For the multi form data, we could consider adding a second sentence on responses (https://gitlab.cfdata.org/cloudflare/aig/ai-gateway-worker/-/blob/main/src/dlp.ts?ref_type=heads#L84):

Responses to multipart requests are not scanned.


### Streaming behavior

When DLP response scanning is enabled and a client sends a streaming request (`"stream": true`), AI Gateway buffers the complete provider response before running DLP inspection. This differs from requests without DLP, where streamed chunks are forwarded to the client as they arrive.

Because of this buffering:

- **Time-to-first-token latency increases** proportionally to the full response generation time.
- **Request-only DLP scanning** (where the **Check** setting is set to **Request**) does not buffer the response and has no impact on streaming latency.
- If you need low-latency streaming for certain requests while still using DLP on the same gateway, consider setting the DLP policy **Check** to **Request** only, or use separate gateways for latency-sensitive and DLP-scanned traffic.

### Per-request DLP controls

DLP policies are configured at the gateway level and apply uniformly to all requests passing through that gateway. There is no per-request header to select specific DLP profiles or to bypass DLP scanning for individual requests.

If you need different DLP policies for different use cases (for example, per-tenant policy variance in a multi-tenant application), the recommended approach is to create separate gateways with different DLP configurations and route requests to the appropriate gateway based on your application logic.

## Integration with Cloudflare DLP

AI Gateway DLP uses the same detection profiles and policies as Cloudflare's enterprise DLP solution. This means:
AI Gateway DLP uses the same [detection profiles](/cloudflare-one/data-loss-prevention/dlp-profiles/) as Cloudflare One's DLP solution. Profiles are shared account-level objects, so you can reuse existing predefined or custom profiles across both [Gateway HTTP policies](/cloudflare-one/data-loss-prevention/dlp-policies/) and AI Gateway DLP policies.

Key differences from Cloudflare One Gateway DLP:

- **Unified management** - Configure DLP policies once and apply them across web traffic, email, SaaS applications, and AI interactions
- **Consistent detection** - The same sensitive data patterns are detected across all channels
- **Centralized reporting** - All DLP events appear in the same dashboard and logs
- **Shared profiles** - Reuse existing DLP detection profiles for AI traffic
- **No Gateway proxy or TLS decryption required** - AI Gateway inspects traffic directly as an AI proxy, so you do not need to set up [Gateway HTTP filtering](/cloudflare-one/traffic-policies/get-started/http/) or [TLS decryption](/cloudflare-one/traffic-policies/http-policies/tls-decryption/).
- **Separate policy management** - DLP policies for AI Gateway are configured per gateway in the AI Gateway dashboard, not in Cloudflare One traffic policies.

Check warning on line 68 in src/content/docs/ai-gateway/features/dlp/index.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-month

Potential month found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
- **Separate logs** - DLP events for AI Gateway appear in [AI Gateway logs](/ai-gateway/observability/logging/), not in Cloudflare One HTTP request logs.

Check warning on line 69 in src/content/docs/ai-gateway/features/dlp/index.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-month

Potential month found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
- **Shared profiles** - DLP detection profiles (predefined and custom) are shared across both products. Changes to a profile apply everywhere it is used.

For more information about Cloudflare's DLP capabilities, refer to the [Data Loss Prevention documentation](/cloudflare-one/data-loss-prevention/).

Expand Down
21 changes: 13 additions & 8 deletions src/content/docs/ai-gateway/features/dlp/set-up-dlp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
order: 2
---

import { Render } from "~/components";

Add Data Loss Prevention (DLP) to any AI Gateway to start scanning AI prompts and responses for sensitive data.

## Prerequisites
Expand Down Expand Up @@ -69,21 +71,24 @@

### Viewing DLP logs in AI Gateway

DLP events are integrated into your AI Gateway logs:
DLP events are integrated into your AI Gateway logs. When a DLP policy matches, the log entry includes details about the match alongside standard log fields like provider, model, tokens, and cost.

1. Go to **AI** > **AI Gateway** > your gateway > **Logs**.
2. Click on any log entry to view detailed information. For requests where DLP policies were triggered, additional details are included:
- **DLP Action Taken**: Shows whether the action was "Flag" or "Block"
- **DLP Policies Matched**: Detailed information about each policy that matched, including:
- Which DLP profiles triggered within each policy
- Whether the match occurred in the request or response
- Specific entries that matched within each DLP profile
2. Select any log entry to view detailed information. For requests where DLP policies were triggered, the log entry includes additional DLP fields:

<Render file="dlp-log-fields" product="ai-gateway" />

### DLP fields in the Logs API

When you retrieve logs through the [Logs API](/api/resources/ai_gateway/subresources/logs/methods/list/), log entries for requests where DLP policies matched include DLP-specific fields in the response. These fields contain the same match data surfaced in the dashboard and in the `cf-aig-dlp` response header, including the action taken, matched policy IDs, matched profile IDs, and entry IDs.

For more information on log fields, refer to the [Logging documentation](/ai-gateway/observability/logging/).

### Filter DLP events

To view only DLP-related requests:

1. On the **Logs** tab, click **Add Filter**.
1. On the **Logs** tab, select **Add Filter**.
2. Select **DLP Action** from the filter options.
3. Choose to filter by:
- **FLAG** - Show only requests where sensitive data was flagged
Expand Down Expand Up @@ -145,11 +150,11 @@
When DLP blocks a request, your application will receive structured error responses:

- **Request blocked by DLP**
- `"code": 2029`

Check warning on line 153 in src/content/docs/ai-gateway/features/dlp/set-up-dlp.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-year

Potential year found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
- `"message": "Request content blocked due to DLP policy violations"`

- **Response blocked by DLP**
- `"code": 2030`

Check warning on line 157 in src/content/docs/ai-gateway/features/dlp/set-up-dlp.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-year

Potential year found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
- `"message": "Response content blocked due to DLP policy violations"`

Handle these errors in your application:
Expand All @@ -163,10 +168,10 @@
})
return Response.json(res)
} catch (e) {
if ((e as Error).message.includes('2029')) {

Check warning on line 171 in src/content/docs/ai-gateway/features/dlp/set-up-dlp.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-year

Potential year found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
return new Response('Request contains sensitive data and cannot be processed.')
}
if ((e as Error).message.includes('2030')) {

Check warning on line 174 in src/content/docs/ai-gateway/features/dlp/set-up-dlp.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-year

Potential year found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
return new Response('AI response was blocked due to sensitive content.')
}
return new Response('AI request failed')
Expand Down
12 changes: 11 additions & 1 deletion src/content/docs/ai-gateway/observability/logging/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { Render } from "~/components";

Logging is a fundamental building block for application development. Logs provide insights during the early stages of development and are often critical to understanding issues occurring in production.

Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status, token usage, cost, and duration. These logs persist, giving you the flexibility to store them for your preferred duration and do more with valuable request data.
Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status, token usage, cost, and duration. When [DLP](/ai-gateway/features/dlp/) policies are configured, logs for requests that trigger a DLP match also include the DLP action taken (Flag or Block), matched policy IDs, matched profile IDs, and the specific detection entries that were triggered. These logs persist, giving you the flexibility to store them for your preferred duration and do more with valuable request data.

By default, each gateway can store up to 10 million logs. You can customize this limit per gateway in your gateway settings to align with your specific requirements. If your storage limit is reached, new logs will stop being saved. To continue saving logs, you must delete older logs to free up space for new logs.
To learn more about your plan limits, refer to [Limits](/ai-gateway/reference/limits/).
Expand Down Expand Up @@ -47,9 +47,18 @@ curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/
'
```

## DLP fields in logs

When [Data Loss Prevention (DLP)](/ai-gateway/features/dlp/) policies are enabled on a gateway, log entries for requests that trigger a DLP policy match include additional fields:

<Render file="dlp-log-fields" product="ai-gateway" />

These fields are available both in the dashboard log viewer and through the [Logs API](/api/resources/ai_gateway/subresources/logs/methods/list/). You can filter logs by **DLP Action** in the dashboard to view only flagged or blocked requests. For more details on DLP monitoring, refer to [Monitor DLP events](/ai-gateway/features/dlp/set-up-dlp/#monitor-dlp-events).

## Managing log storage

To manage your log storage effectively, you can:

- Set Storage Limits: Configure a limit on the number of logs stored per gateway in your gateway settings to ensure you only pay for what you need.
- Enable Automatic Log Deletion: Activate the Automatic Log Deletion feature in your gateway settings to automatically delete the oldest logs once the log limit you've set or the default storage limit of 10 million logs is reached. This ensures new logs are always saved without manual intervention.

Expand Down Expand Up @@ -82,6 +91,7 @@ See full list of available filters and their descriptions below:
| Metadata Value | equals, does not equal | specific metadata values. |
| Log ID | equals, does not equal | a specific Log ID. |
| Event ID | equals, does not equal | a specific Event ID. |
| DLP Action | FLAG, BLOCK | the DLP action taken on the request. |

### API deletion

Expand Down
11 changes: 11 additions & 0 deletions src/content/docs/ai-gateway/reference/limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import { Render } from "~/components";

The following limits apply to gateway configurations, logs, and related features in Cloudflare's platform.

## Gateway and log limits

| Feature | Limit |
| --------------------------------------------------------------------- | ----------------------------------- |
| [Cacheable request size](/ai-gateway/features/caching/) | 25 MB per request |
Expand Down Expand Up @@ -39,4 +41,13 @@ on how to automatically delete logs.

<sup>3</sup> Logs larger than 10 MB will not be stored.

## DLP limits

[DLP](/ai-gateway/features/dlp/) for AI Gateway uses shared [Cloudflare One DLP profiles](/cloudflare-one/data-loss-prevention/dlp-profiles/). The following limits apply to DLP profiles and detection entries at the account level:

<Render file="data-loss-prevention/dlp-limits-table" product="cloudflare-one" />

DLP profiles are shared with Cloudflare One and are not coupled to individual gateways. You can apply the same DLP profiles across multiple gateways without additional profile limits. There is no separate limit on the number of DLP policies per gateway.


<Render file="limits-increase" product="ai-gateway" />
10 changes: 2 additions & 8 deletions src/content/docs/cloudflare-one/account-limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,7 @@ This page lists the default account limits for rules, applications, fields, and

## Data Loss Prevention (DLP)

| Feature | Limit |
| ---------------------------------------- | --------- |
| Custom entries | 25 |
| Exact Data Match cells per spreadsheet | 100,000 |
| Custom Wordlist keywords per spreadsheet | 200 |
| Custom Wordlist keywords per account | 1,000 |
| Dataset cells per account | 1,000,000 |
<Render file="data-loss-prevention/dlp-limits-table" product="cloudflare-one" />

## Cloudflare Tunnel

Expand Down Expand Up @@ -105,4 +99,4 @@ This page lists the default account limits for rules, applications, fields, and
| -------------------------------------------------------------------------- | ------ |
| Characters per device profile expression | 10,000 |
| Combined Split Tunnel and Local Domain Fallback entries per device profile | 1,000 |
| Device IP profiles per account | 30 |
| Device IP profiles per account | 30 |
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ sidebar:

You can scan HTTP traffic for sensitive data through Secure Web Gateway policies. To perform DLP filtering, first configure a DLP profile with the data patterns you want to detect, and then build a Gateway HTTP policy to allow or block the sensitive data from leaving your organization. Gateway will parse and scan your HTTP traffic for strings matching the keywords or regular expressions (regexes) specified in the DLP profile.

:::note
To scan AI prompts and responses without Gateway HTTP filtering, you can also enable DLP directly on an [AI Gateway](/ai-gateway/features/dlp/).
:::

## Prerequisites

- Set up [Gateway HTTP filtering](/cloudflare-one/traffic-policies/initial-setup/http/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ A DLP profile is a collection of regular expressions and [detection entries](/cl
product="cloudflare-one"
/>

You can now use this profile in a [DLP policy](/cloudflare-one/data-loss-prevention/dlp-policies/#2-create-a-dlp-policy) or [CASB integration](/cloudflare-one/cloud-and-saas-findings/casb-dlp/).
You can now use this profile in a [DLP policy](/cloudflare-one/data-loss-prevention/dlp-policies/#2-create-a-dlp-policy), [CASB integration](/cloudflare-one/cloud-and-saas-findings/casb-dlp/), or [AI Gateway DLP policy](/ai-gateway/features/dlp/set-up-dlp/).

## Build a custom profile

<Render file="data-loss-prevention/custom-profile" product="cloudflare-one" />

You can now use this profile in a [DLP policy](/cloudflare-one/data-loss-prevention/dlp-policies/#2-create-a-dlp-policy) or [CASB integration](/cloudflare-one/cloud-and-saas-findings/casb-dlp/).
You can now use this profile in a [DLP policy](/cloudflare-one/data-loss-prevention/dlp-policies/#2-create-a-dlp-policy), [CASB integration](/cloudflare-one/cloud-and-saas-findings/casb-dlp/), or [AI Gateway DLP policy](/ai-gateway/features/dlp/set-up-dlp/).
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@

To get started, refer to [Scan SaaS applications with DLP](/cloudflare-one/cloud-and-saas-findings/casb-dlp/).

## AI traffic

Data Loss Prevention integrates with [Cloudflare AI Gateway](/ai-gateway/) to scan AI prompts and responses for sensitive data. When DLP is enabled on an AI Gateway, it inspects the text content of requests sent to AI providers and responses returned from AI models, without requiring Gateway HTTP filtering or TLS decryption.

To get started, refer to [Set up DLP for AI Gateway](/ai-gateway/features/dlp/set-up-dlp/).

## Supported file types

### Formats
Expand All @@ -41,7 +47,7 @@
DLP supports reporting and scanning the following file types:

- Text and CSV
- Microsoft Office 2007 and later (`.docx`, `.xlsx,` `.pptx`), including Microsoft 365

Check warning on line 50 in src/content/docs/cloudflare-one/data-loss-prevention/index.mdx

View workflow job for this annotation

GitHub Actions / Semgrep

semgrep.style-guide-potential-date-year

Potential year found. Documentation should strive to represent universal truth, not something time-bound. (add [skip style guide checks] to commit message to skip)
- PDF
- ZIP files containing the above

Expand Down
4 changes: 4 additions & 0 deletions src/content/glossary/ai-gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ entries:
general_definition: |-
Header to customize the backoff type for [request retries](/ai-gateway/configuration/request-handling/#request-retries) of a request.

- term: cf-aig-dlp
general_definition: |-
A response header returned when a [DLP policy](/ai-gateway/features/dlp/set-up-dlp/#dlp-response-header) matches a request or response. Contains JSON with the action taken (Flag or Block), matched policy IDs, matched profile IDs, and detection entry IDs.

# Deprecated headers
- term: cf-cache-ttl
general_definition: |-
Expand Down
11 changes: 11 additions & 0 deletions src/content/partials/ai-gateway/dlp-log-fields.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
{}
---

| Field | Description |
| -------------------- | --------------------------------------------------------------------- |
| DLP Action | The action taken by the DLP policy: `FLAG` or `BLOCK` |
| DLP Policies Matched | The IDs of the DLP policies that matched |
| DLP Profiles Matched | The IDs of the DLP profiles that triggered within each matched policy |
| DLP Entries Matched | The specific detection entry IDs that matched within each profile |
| DLP Check | Whether the match occurred in the `REQUEST`, `RESPONSE`, or both |
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
{}
---

| Feature | Limit |
| ---------------------------------------- | --------- |
| Custom entries | 25 |
| Exact Data Match cells per spreadsheet | 100,000 |
| Custom Wordlist keywords per spreadsheet | 200 |
| Custom Wordlist keywords per account | 1,000 |
| Dataset cells per account | 1,000,000 |
Loading