Skip to content

chore: docs changes against feats #949

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
whysumedh wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: docs changes against feats #949

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api-reference/inference-api/config-object.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -650,7 +650,7 @@ You can find more examples of schemas [below](/api-reference/inference-api/confi
| `passthrough` | Defer provider resolution to the incoming request | boolean | No | - | When `true`, the target does not require `provider`. See [Passthrough targets](/product/ai-gateway/configs#passthrough-targets) |
| `override_params` | Pass model name and other hyper parameters | object | No | "model", "temperature", "frequency\_penalty", "logit\_bias", "logprobs", "top\_logprobs", "max\_tokens", "n", "presence\_penalty", "response\_format", "seed", "stop", "top_p", etc. | Pass everything that's typically part of the payload |
| `default_params` | Inject parameters into the request body only when not already present (respects client-sent values) | object | No | Any payload field | See [Default and drop params](/product/ai-gateway/configs#default-and-drop-params) |
| `drop_params` | Remove parameters from the final request body using dot-notation paths (supports nested keys and array indices) | array of strings | No | Any payload field path | See [Default and drop params](/product/ai-gateway/configs#default-and-drop-params) |
| `drop_params` | Remove parameters from the final request body using bracket-notation paths (nested keys, array indices, and `[*]` wildcards) | array of strings | No | Any payload field path | See [Default and drop params](/product/ai-gateway/configs#default-and-drop-params) |

### Strategy Object Details

Expand Down
52 changes: 9 additions & 43 deletions integrations/libraries/claude-code-bedrock.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,40 +72,11 @@ Replace:

That's it! All Claude Code requests now route through Bedrock via Portkey. Monitor usage in the [Portkey Dashboard](https://app.portkey.ai/logs).

## Forward Headers (Required for Some Features)

Some Claude Code features require the `anthropic-beta` header to reach Bedrock. Configure this in a [Portkey Config](/product/ai-gateway/configs):

<Steps>
<Step title="Create a Config">

Go to [Configs](https://app.portkey.ai/configs) → **Create Config**:

```json
{
"provider": "@bedrock-prod",
"forward_headers": ["anthropic-beta"]
}
```

Save and copy the Config ID.
</Step>

<Step title="Update Claude Code Settings">

Add the config to your settings:
<Info>
Portkey automatically handles `anthropic-beta` headers for Bedrock: unsupported beta flags are filtered, provider-specific values are remapped, and supported flags are forwarded upstream. You do not need `forward_headers` in your config for Claude Code to work.

```json
{
"env": {
"ANTHROPIC_BASE_URL": "https://api.portkey.ai",
"ANTHROPIC_AUTH_TOKEN": "YOUR_PORTKEY_API_KEY",
"ANTHROPIC_CUSTOM_HEADERS": "x-portkey-api-key: YOUR_PORTKEY_API_KEY\nx-portkey-config: YOUR_CONFIG_ID"
}
}
```
</Step>
</Steps>
To bypass this filtering and pass `anthropic-beta` through unchanged, add `"forward_headers": ["anthropic-beta"]` to your [Portkey Config](/product/ai-gateway/configs).
</Info>

## Trace Requests

Expand Down Expand Up @@ -139,8 +110,7 @@ Route to backup providers when Bedrock fails. Create a config with fallback targ
{ "provider": "@bedrock-prod" },
{ "provider": "@anthropic-prod" },
{ "provider": "@vertex-prod" }
],
"forward_headers": ["anthropic-beta"]
]
}
```

Expand All @@ -154,8 +124,7 @@ Distribute requests across multiple providers or regions:
"targets": [
{ "provider": "@bedrock-us-east", "weight": 0.5 },
{ "provider": "@bedrock-us-west", "weight": 0.5 }
],
"forward_headers": ["anthropic-beta"]
]
}
```

Expand All @@ -166,8 +135,7 @@ Reduce costs and latency for repeated queries:
```json
{
"provider": "@bedrock-prod",
"cache": { "mode": "simple" },
"forward_headers": ["anthropic-beta"]
"cache": { "mode": "simple" }
}
```

Expand All @@ -178,8 +146,7 @@ Automatically retry failed requests:
```json
{
"provider": "@bedrock-prod",
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] },
"forward_headers": ["anthropic-beta"]
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] }
}
```

Expand Down Expand Up @@ -284,8 +251,7 @@ With Portkey Config:
{ "provider": "@anthropic-prod" }
],
"cache": { "mode": "simple" },
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] },
"forward_headers": ["anthropic-beta"]
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] }
}
```

Expand Down
52 changes: 9 additions & 43 deletions integrations/libraries/claude-code-vertex.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,40 +68,11 @@ Replace:

That's it! All Claude Code requests now route through Vertex AI via Portkey. Monitor usage in the [Portkey Dashboard](https://app.portkey.ai/logs).

## Forward Headers (Required for Some Features)
<Info>
Portkey automatically handles `anthropic-beta` headers for Vertex AI: unsupported beta flags are filtered, provider-specific values are remapped, and supported flags are forwarded upstream. You do not need `forward_headers` in your config for Claude Code to work.

Some Claude Code features require the `anthropic-beta` header to reach Vertex AI. Configure this in a [Portkey Config](/product/ai-gateway/configs):

<Steps>
<Step title="Create a Config">

Go to [Configs](https://app.portkey.ai/configs) → **Create Config**:

```json
{
"provider": "@vertex-prod",
"forward_headers": ["anthropic-beta"]
}
```

Save and copy the Config ID.
</Step>

<Step title="Update Claude Code Settings">

Add the config to your settings:

```json
{
"env": {
"ANTHROPIC_BASE_URL": "https://api.portkey.ai",
"ANTHROPIC_AUTH_TOKEN": "YOUR_PORTKEY_API_KEY",
"ANTHROPIC_CUSTOM_HEADERS": "x-portkey-api-key: YOUR_PORTKEY_API_KEY\nx-portkey-config: YOUR_CONFIG_ID"
}
}
```
</Step>
</Steps>
To bypass this filtering and pass `anthropic-beta` through unchanged, add `"forward_headers": ["anthropic-beta"]` to your [Portkey Config](/product/ai-gateway/configs).
</Info>

## Trace Requests

Expand Down Expand Up @@ -135,8 +106,7 @@ Route to backup providers when Vertex AI fails. Create a config with fallback ta
{ "provider": "@vertex-prod" },
{ "provider": "@anthropic-prod" },
{ "provider": "@bedrock-prod" }
],
"forward_headers": ["anthropic-beta"]
]
}
```

Expand All @@ -150,8 +120,7 @@ Distribute requests across multiple regions:
"targets": [
{ "provider": "@vertex-us-central", "weight": 0.5 },
{ "provider": "@vertex-us-east", "weight": 0.5 }
],
"forward_headers": ["anthropic-beta"]
]
}
```

Expand All @@ -162,8 +131,7 @@ Reduce costs and latency for repeated queries:
```json
{
"provider": "@vertex-prod",
"cache": { "mode": "simple" },
"forward_headers": ["anthropic-beta"]
"cache": { "mode": "simple" }
}
```

Expand All @@ -174,8 +142,7 @@ Automatically retry failed requests:
```json
{
"provider": "@vertex-prod",
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] },
"forward_headers": ["anthropic-beta"]
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] }
}
```

Expand Down Expand Up @@ -280,8 +247,7 @@ With Portkey Config:
{ "provider": "@anthropic-prod" }
],
"cache": { "mode": "simple" },
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] },
"forward_headers": ["anthropic-beta"]
"retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] }
}
```

Expand Down
19 changes: 16 additions & 3 deletions product/ai-gateway/configs.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,10 +172,21 @@ Each config target can shape the request body before it reaches the upstream pro
|---|---|
| `default_params` | Injects parameters into the request body **only when not already present**. Unlike `override_params`, this respects values set by the client. |
| `override_params` | Always overwrites the matching field on the request body. |
| `drop_params` | Removes parameters from the final request body using dot-notation paths. Supports nested keys and array indices (e.g., `tools.0.function.name`). |
| `drop_params` | Removes fields from the final request body using bracket-notation paths (see below). |

Execution order: `default_params` → `override_params` → `drop_params`. All three fields inherit through nested target configs the same way `override_params` does — values defined on a parent target apply to its children unless overridden.

### `drop_params` paths

Each entry in `drop_params` is a path into the request body. The gateway deletes the matching property before forwarding the request. Paths that do not exist on the request are silently ignored.

| Path style | Example | What it removes |
| --- | --- | --- |
| Top-level key | `"logprobs"`, `"metadata"` | A field on the root object |
| Nested object | `"response_format.json_schema"` | A field at any depth (dot-separated segments) |
| Array index | `"tools[0].function.strict"`, `"messages[2]"` | One element in an array and anything nested under it |
| Wildcard array | `"tools[*].function.strict"`, `"messages[*].content[*].cache_control.scope"` | The same nested field on **every** array element (and nested arrays when using multiple `[*]`) |

### Example

```json
Expand All @@ -186,7 +197,7 @@ Execution order: `default_params` → `override_params` → `drop_params`. All t
"provider": "@openai-prod",
"default_params": { "temperature": 0.7, "max_tokens": 1024 },
"override_params": { "model": "gpt-4o" },
"drop_params": ["logprobs", "tools.0.function.strict"]
"drop_params": ["logprobs", "tools[0].function.strict", "tools[*].function.name"]
},
{
"provider": "@anthropic-backup",
Expand All @@ -199,7 +210,9 @@ Execution order: `default_params` → `override_params` → `drop_params`. All t
In this example:
- If the client does not send `temperature` or `max_tokens`, the gateway injects the defaults; if the client does send them, the client's values win.
- `model` is always rewritten to `gpt-4o` regardless of what the client sends.
- `logprobs` and the nested `tools[0].function.strict` field are stripped before the request is forwarded.
- `logprobs` is removed from the root body.
- `tools[0].function.strict` is removed only from the first tool entry.
- `function.name` is removed from **every** tool entry via the `tools[*]` wildcard.

## Passthrough targets

Expand Down