Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions charts/portkey-app/templates/networkpolicy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{{- /*
Egress NetworkPolicy to mitigate SSRF -> AWS IMDS credential theft (F-17).

When enabled, this policy blocks egress to the link-local IMDS address
(169.254.169.254) from every pod deployed by this chart (matched via the
chart-wide selectorLabels).

Requires a NetworkPolicy-enforcing CNI (Calico, Cilium, AWS VPC CNI with
NetworkPolicy support, etc.). Policies that only declare Egress are
additive — they do not deny unrelated traffic.
*/ -}}
{{- if .Values.networkPolicy.enabled }}
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: {{ include "portkey.fullname" . }}-block-imds
labels:
{{- include "portkey.labels" . | nindent 4 }}
spec:
podSelector:
matchLabels:
{{- include "portkey.selectorLabels" . | nindent 6 }}
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.169.254/32
{{- if .Values.networkPolicy.blockLinkLocal }}
- 169.254.0.0/16
{{- end }}
Comment on lines +30 to +33
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With blockLinkLocal enabled, this denies all 169.254.0.0/16 egress. That can break NodeLocal DNSCache (169.254.20.10) and other legitimate link-local traffic. Consider adding an explicit allow rule for NodeLocal DNSCache (ports 53) or documenting the limitation clearly.

Copilot uses AI. Check for mistakes.
{{- with .Values.networkPolicy.additionalBlockedCidrs }}
{{- toYaml . | nindent 14 }}
{{- end }}
Comment on lines +35 to +36
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the gateway chart: ipBlock.cidr: 0.0.0.0/0 is IPv4-only, so any IPv6 CIDRs placed into additionalBlockedCidrs would render an invalid except list and the NetworkPolicy will be rejected by the API server. Consider documenting IPv4-only, validating input, or adding a separate IPv6 egress rule (::/0) for dual-stack support.

Suggested change
{{- toYaml . | nindent 14 }}
{{- end }}
{{- range . }}
{{- if contains ":" . }}
{{- fail (printf "networkPolicy.additionalBlockedCidrs entry %q is IPv6, but charts/portkey-app/templates/networkpolicy.yaml currently renders an IPv4-only ipBlock.cidr (0.0.0.0/0). Use only IPv4 CIDRs here." .) }}
{{- end }}
- {{ . }}
{{- end }}
{{- end }}

Copilot uses AI. Check for mistakes.
{{- end }}
34 changes: 34 additions & 0 deletions charts/portkey-app/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,40 @@ commonLabels: {}
# -- Common environment variables that will be applied to all deployments/statefulsets created by the chart. Be careful not to override values already specified by the chart.
commonEnv: []

# -- NetworkPolicy to mitigate SSRF -> AWS IMDS credential theft.
# When enabled, blocks egress to 169.254.169.254 from all pods deployed by this chart.
# Requires a NetworkPolicy-enforcing CNI (Calico, Cilium, etc.).
networkPolicy:
enabled: false
# Also block the full link-local range 169.254.0.0/16.
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blockLinkLocal: true blocks all 169.254.0.0/16, which can break NodeLocal DNSCache (commonly 169.254.20.10) and other required link-local endpoints. Consider adding a warning here (and/or a way to allow specific link-local IPs) so enabling this doesn’t unexpectedly break DNS.

Suggested change
# Also block the full link-local range 169.254.0.0/16.
# Also block the full link-local range 169.254.0.0/16.
# Warning: enabling this can break NodeLocal DNSCache (commonly 169.254.20.10)
# and other required link-local endpoints. Only enable after verifying your
# cluster does not depend on link-local services for DNS or other networking.

Copilot uses AI. Check for mistakes.
blockLinkLocal: false
# Additional CIDRs to deny from pod egress (e.g., internal metadata services).
additionalBlockedCidrs: []
# - 169.254.170.2/32 # ECS task metadata endpoint

# -- Recommended hardened pod/container security context defaults.
# These are not applied by default; copy into individual component
# `deployment.podSecurityContext` / `deployment.securityContext` blocks
# (backend, gateway, dataservice, frontend, etc.) to enable.
#
# podSecurityContext:
# runAsNonRoot: true
# runAsUser: 1000
# fsGroup: 2000
# seccompProfile:
# type: RuntimeDefault
#
# securityContext:
# allowPrivilegeEscalation: false
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
# capabilities:
# drop:
# - ALL
# seccompProfile:
# type: RuntimeDefault

imageCredentials:
- name: portkeyenterpriseregistrycredentials
create: true
Expand Down
33 changes: 28 additions & 5 deletions charts/portkey-gateway/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,19 +176,34 @@ Trust Relationship:
}
```

**Method 2: EKS IRSA**
**Method 2: EKS IRSA (Recommended)**
```yaml
LOG_STORE: s3_assume
LOG_STORE_REGION: "<AWS Bucket Region>"
LOG_STORE_GENERATIONS_BUCKET: "<AWS Bucket Name>"
```

**Method 3: EC2 Instance Metadata (IMDS)**
Attach the role ARN to the pod's ServiceAccount:
```yaml
serviceAccount:
create: true
annotations:
eks.amazonaws.com/role-arn: "arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>"
```

**Method 3: EC2 Instance Metadata Service (IMDSv2 only)**

> ⚠️ **Security notice**
> - Only use IMDS when IRSA is not available. IRSA (Method 2) is strongly preferred.
> - Node EC2 instances **must** enforce IMDSv2 (`HttpTokens=required`) with `HttpPutResponseHopLimit=1`.
> This prevents an SSRF or RCE in a pod from stealing the node's IAM role credentials.
> - Do **not** set `AWS_IMDS_V1=true`. IMDSv1 fallback is deprecated and is an SSRF amplifier.
> - Also enable the chart's `networkPolicy` (see `values.yaml`) to block pod egress to `169.254.169.254`.

```yaml
LOG_STORE: s3_assume
LOG_STORE_REGION: "us-east-1"
LOG_STORE_GENERATIONS_BUCKET: "<AWS Bucket Name>"
AWS_IMDS_V1: true # Only if using IMDS v1
```
</details>

Expand Down Expand Up @@ -396,7 +411,10 @@ For AWS Bedrock integration, configure Assumed Role Access.

### Quick Setup

**Required IAM Policy:**
**Required IAM Policy (least-privilege):**

Scope `Resource` to the specific foundation models / inference profiles you actually invoke. Avoid `"Resource": "*"` in production.

```json
{
"Version": "2012-10-17",
Expand All @@ -407,12 +425,17 @@ For AWS Bedrock integration, configure Assumed Role Access.
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": "*"
"Resource": [
"arn:aws:bedrock:<REGION>::foundation-model/anthropic.claude-3-7-sonnet-*",
"arn:aws:bedrock:<REGION>:<ACCOUNT_ID>:inference-profile/*"
]
}
]
}
```

> `"Resource": "*"` is only acceptable for short-lived exploration; for production always list the specific model ARNs.

### Configuration Options

**With Long-term Credentials:**
Expand Down
18 changes: 14 additions & 4 deletions charts/portkey-gateway/docs/Bedrock.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Alternatively you can use an access token and secret key id, but using assumed r

## Step 1: Create Bedrock IAM Policy

Create an IAM policy with the necessary Bedrock permissions:
Create an IAM policy with the necessary Bedrock permissions. Scope `Resource` to the specific foundation models / inference profiles you actually invoke — avoid `"Resource": "*"` in production:

```json
{
Expand All @@ -22,13 +22,16 @@ Create an IAM policy with the necessary Bedrock permissions:
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": "*"
"Resource": [
"arn:aws:bedrock:<REGION>::foundation-model/anthropic.claude-3-7-sonnet-*",
"arn:aws:bedrock:<REGION>:<ACCOUNT_ID>:inference-profile/*"
]
}
]
}
```

**Note:** You can make the `Resource` field more granular by specifying specific model ARNs instead of using `"*"`.
> **Least-privilege note:** `"Resource": "*"` is only acceptable for short-lived exploration. For production, list the specific foundation-model ARNs you invoke. Overly broad Bedrock permissions combined with a pod compromise could be used to exfiltrate data through any model in the account.

## Step 2: Role Configuration Options

Expand Down Expand Up @@ -77,14 +80,21 @@ environment:
AWS_ASSUME_ROLE_REGION: <your-region>
```

### Method 2: IRSA (IAM Roles for Service Accounts) for EKS
### Method 2: IRSA (IAM Roles for Service Accounts) for EKS — Recommended
- Use the role attached to your EKS service account as the principal role
- No additional environment variables needed for authentication
- Eliminates SSRF-to-IMDS credential theft on EKS worker nodes

### Method 3: IMDS (Instance Metadata Service) for EC2
- Use the role attached to your EC2 instance as the principal role
- No additional environment variables needed for authentication

> ⚠️ **Security notice**
> - Use IRSA (Method 2) on EKS instead of IMDS whenever possible.
> - If you must use IMDS, the node **must** enforce IMDSv2 (`HttpTokens=required`, `HttpPutResponseHopLimit=1`) to prevent SSRF from a pod reaching the metadata endpoint and stealing node credentials.
> - Do **not** set `AWS_IMDS_V1=true`. IMDSv1 fallback is deprecated and is a direct SSRF amplifier.
> - Also enable the chart's `networkPolicy` to block pod egress to `169.254.169.254`.

## Step 4: Virtual Key Creation

When creating Virtual Keys in Portkey, provide:
Expand Down
65 changes: 65 additions & 0 deletions charts/portkey-gateway/templates/networkpolicy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
{{- /*
Egress NetworkPolicy to mitigate SSRF -> AWS IMDS credential theft (F-17).

When enabled, this policy blocks pod egress to the link-local IMDS address
(169.254.169.254) and, optionally, the full link-local range. It does NOT
restrict other egress; cluster DNS and normal traffic are allowed via an
explicit allow-all except-IMDS rule.

Requires a NetworkPolicy-enforcing CNI (Calico, Cilium, AWS VPC CNI with
NetworkPolicy support, etc.). Policies that only declare Egress are
additive — they do not deny unrelated traffic.
*/ -}}
{{- if .Values.networkPolicy.enabled }}
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: {{ include "portkeyenterprise.fullname" . }}-block-imds
labels:
{{- include "portkeyenterprise.labels" . | nindent 4 }}
spec:
podSelector:
matchLabels:
{{- include "portkeyenterprise.selectorLabels" . | nindent 6 }}
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.169.254/32
{{- if .Values.networkPolicy.blockLinkLocal }}
- 169.254.0.0/16
{{- end }}
Comment on lines +31 to +34
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When blockLinkLocal is enabled, this policy denies all 169.254.0.0/16 egress, which can break NodeLocal DNSCache (169.254.20.10) and other legitimate link-local endpoints. If you want to keep this option, consider adding an explicit allow rule for NodeLocal DNSCache (UDP/TCP 53) or documenting that users must disable NodeLocal DNSCache / provide their own egress exceptions.

Copilot uses AI. Check for mistakes.
{{- with .Values.networkPolicy.additionalBlockedCidrs }}
{{- toYaml . | nindent 14 }}
{{- end }}
Comment on lines +27 to +37
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ipBlock.cidr is IPv4-only (0.0.0.0/0), but additionalBlockedCidrs can contain IPv6 CIDRs; Kubernetes requires all except entries to be in the same IP family as the parent cidr, otherwise the NetworkPolicy is rejected. Consider validating additionalBlockedCidrs to IPv4-only (and fail the render on IPv6), or add a second IPv6 ipBlock rule (::/0) with its own except list for dual-stack clusters.

Copilot uses AI. Check for mistakes.
{{- if .Values.dataservice.enabled }}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: {{ include "portkeyenterprise.fullname" . }}-{{ .Values.dataservice.name }}-block-imds
labels:
{{- include "dataservice.labels" . | nindent 4 }}
spec:
podSelector:
matchLabels:
{{- include "dataservice.selectorLabels" . | nindent 6 }}
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.169.254/32
{{- if .Values.networkPolicy.blockLinkLocal }}
- 169.254.0.0/16
{{- end }}
{{- with .Values.networkPolicy.additionalBlockedCidrs }}
{{- toYaml . | nindent 14 }}
{{- end }}
{{- end }}
{{- end }}
28 changes: 25 additions & 3 deletions charts/portkey-gateway/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -165,15 +165,37 @@ podAnnotations: {}
podLabels: {}

podSecurityContext: {}
# Recommended hardened defaults (uncomment and adapt):
# runAsNonRoot: true
# runAsUser: 1000
# fsGroup: 2000
# seccompProfile:
# type: RuntimeDefault

securityContext: {}
# capabilities:
# drop:
# - ALL
# Recommended hardened defaults (uncomment and adapt):
# allowPrivilegeEscalation: false
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
# capabilities:
# drop:
# - ALL
# seccompProfile:
# type: RuntimeDefault

# NetworkPolicy to mitigate SSRF -> AWS IMDS credential theft.
# Blocks pod egress to 169.254.169.254 (EC2 Instance Metadata Service).
# Requires a NetworkPolicy-enforcing CNI (Calico, Cilium, etc.).
networkPolicy:
# Set to true to install egress NetworkPolicies for gateway and dataservice.
enabled: false
# Also block the full link-local range 169.254.0.0/16.
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blockLinkLocal: true blocks all 169.254.0.0/16, which can break clusters using NodeLocal DNSCache (commonly 169.254.20.10) and any other required link-local endpoints. Either add a prominent warning here (including the NodeLocal DNSCache caveat) or provide an explicit allow exception for required link-local IPs when blockLinkLocal is enabled.

Suggested change
# Also block the full link-local range 169.254.0.0/16.
# WARNING: When enabled, this blocks the full link-local range 169.254.0.0/16.
# This can break clusters that rely on link-local endpoints such as NodeLocal
# DNSCache (commonly 169.254.20.10) and other required local services.
# Only enable this if your cluster does not depend on such endpoints, or if
# required exceptions are handled elsewhere in your NetworkPolicy configuration.

Copilot uses AI. Check for mistakes.
blockLinkLocal: false
# Additional CIDRs to deny from pod egress (e.g., internal metadata services).
additionalBlockedCidrs: []
# - 169.254.170.2/32 # ECS task metadata endpoint
# - fd00:ec2::254/128 # IMDS IPv6
Comment on lines +195 to +198
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The values comment suggests adding an IPv6 CIDR (e.g., fd00:ec2::254/128) to networkPolicy.additionalBlockedCidrs, but the rendered policy uses an IPv4 ipBlock (cidr: 0.0.0.0/0). Kubernetes requires ipBlock.except entries to be the same IP family as cidr, so any IPv6 entry here will make the NetworkPolicy invalid and fail to apply. Consider either removing the IPv6 example / explicitly documenting IPv4-only, or adding dual-stack support (separate IPv6 egress rule and separate v6 blocked CIDRs).

Suggested change
# Additional CIDRs to deny from pod egress (e.g., internal metadata services).
additionalBlockedCidrs: []
# - 169.254.170.2/32 # ECS task metadata endpoint
# - fd00:ec2::254/128 # IMDS IPv6
# Additional IPv4 CIDRs to deny from pod egress (e.g., internal metadata services).
# Do not include IPv6 CIDRs here unless the templates are updated with separate IPv6 rules.
additionalBlockedCidrs: []
# - 169.254.170.2/32 # ECS task metadata endpoint

Copilot uses AI. Check for mistakes.

service:
type: NodePort
Expand Down