Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,13 @@
# --- Chutes ---
#CHUTES_API_KEY_1="YOUR_CHUTES_API_KEY"

# --- Hatz AI ---
# Hatz uses the Responses API internally. The proxy translates Chat Completions
# requests to Hatz's /v1/openai/responses endpoint automatically.
# Authentication uses X-API-Key header (not Bearer token).
#HATZ_API_KEY_1="YOUR_HATZ_API_KEY"
#HATZ_API_KEY_2="YOUR_HATZ_API_KEY_2"

# ------------------------------------------------------------------------------
# | [OAUTH] Provider OAuth 2.0 Credentials |
# ------------------------------------------------------------------------------
Expand Down
42 changes: 42 additions & 0 deletions DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,48 @@ The manager supports loading credentials from two sources, with a clear priority
- This is the key to "Stateless Deployment" for platforms like Railway, Render, Heroku
- Credentials are referenced internally using `env://` URIs (e.g., `env://gemini_cli/1`)

#### 2.6.4. Remote Host Authentication (SSH Port Forwarding)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Section numbering is out of order.

2.6.4 appears before 2.6.3, which breaks doc structure and cross-reference clarity. Please renumber/reorder this subsection sequence.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@DOCUMENTATION.md` at line 240, The subsection "2.6.4. Remote Host
Authentication (SSH Port Forwarding)" is placed before "2.6.3", breaking the
document order; locate the heading text "2.6.4. Remote Host Authentication (SSH
Port Forwarding)" and either renumber it to "2.6.3" (and adjust any subsequent
section numbers) or move that entire subsection below the existing "2.6.3" entry
so numbering is sequential, and update any in-doc cross-references that point to
"2.6.4" accordingly.


When the proxy is deployed on a remote host (VPS, cloud server, etc.), OAuth authentication requires special handling because OAuth callbacks are sent to `localhost`, which on the remote server refers to the server itself, not your local machine.

**The Problem:**

- Proxy runs on remote VPS at `your-vps-ip`
- You attempt to add OAuth credentials using the credential tool on the VPS
- OAuth provider redirects to `http://localhost:PORT/callback`
- On the VPS, `localhost` points to the VPS's localhost, not your local browser
- The callback fails because your browser cannot connect to the VPS's localhost

**The Solution: SSH Port Forwarding**

Create an SSH tunnel to forward OAuth callback ports from the VPS to your local machine:

```bash
# Single provider examples
ssh -L 8085:localhost:8085 user@your-vps-ip # Gemini CLI
ssh -L 51121:localhost:51121 user@your-vps-ip # Antigravity
ssh -L 11451:localhost:11451 user@your-vps-ip # iFlow

# Multiple providers simultaneously
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip
```

**Workflow:**

1. **Establish SSH tunnel** (keep this connection open)
2. **Run credential tool on VPS** (in separate SSH session)
3. **Complete browser-based OAuth** - callbacks are forwarded via tunnel
4. **Close SSH tunnel** after authentication completes

**Alternative Approach: Local Authentication + Export**

If SSH port forwarding is not feasible:
1. Complete OAuth flows locally on your machine
2. Export credentials to environment variables using credential tool's export feature
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be helpful to include the actual command for exporting credentials here, e.g., python -m rotator_library.credential_tool --export, so users don’t have to hunt for it.

3. Deploy `.env` file to remote server

This approach uses the credential tool's export functionality to generate environment variable representations of OAuth credentials, which can then be deployed to stateless environments without requiring SSH tunnels.

**Gemini CLI Environment Variables:**

Single credential (legacy format):
Expand Down
8 changes: 4 additions & 4 deletions Deployment guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,13 +523,13 @@ SSH tunnels forward ports from your local machine to the remote VPS, allowing yo
From your **local machine**, open a terminal and run:

```bash
# Forward all OAuth callback ports at once
ssh -L 51121:localhost:51121 -L 8085:localhost:8085 -L 11451:localhost:11451 user@your-vps-ip
# Forward all OAuth callback ports at once (recommended)
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip

# Alternative: Forward ports individually as needed
ssh -L 51121:localhost:51121 user@your-vps-ip # For Antigravity
ssh -L 8085:localhost:8085 user@your-vps-ip # For Gemini CLI
ssh -L 11451:localhost:11451 user@your-vps-ip # For iFlow
ssh -L 51121:localhost:51121 user@your-vps-ip # For Antigravity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a slight formatting inconsistency with the indentation of the comment here (triple space vs single space on other lines).

Suggested change
ssh -L 51121:localhost:51121 user@your-vps-ip # For Antigravity
ssh -L 51121:localhost:51121 user@your-vps-ip # For Antigravity

ssh -L 11451:localhost:11451 user@your-vps-ip # For iFlow
```

**Keep this SSH session open** during the entire authentication process.
Expand Down
78 changes: 76 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,76 @@ For platforms without file persistence (Railway, Render, Vercel):

</details>

<details>
<summary><b>Remote Host Deployment (SSH Port Forwarding)</b></summary>

When the proxy is running on a remote host (VPS, cloud server, etc.), OAuth token authentication requires SSH port forwarding. This is because the OAuth callback URL is sent to `localhost`, which on the remote server points to the server itself, not your local machine.

**The Problem:**

- You run the proxy on a remote VPS
- You try to add OAuth credentials using the credential tool
- The OAuth provider redirects to `http://localhost:PORT/callback`
- On the VPS, `localhost` refers to the VPS, not your local machine
- The callback fails because your browser can't reach the VPS's localhost

**The Solution: SSH Port Forwarding**

Use SSH to tunnel the OAuth callback ports from the VPS back to your local machine. You only need to do this when adding OAuth credentials.

**Single Provider Examples:**

```bash
# Gemini CLI (port 8085)
ssh -L 8085:localhost:8085 user@your-vps-ip

# Antigravity (port 51121)
ssh -L 51121:localhost:51121 user@your-vps-ip

# iFlow (port 11451)
ssh -L 11451:localhost:11451 user@your-vps-ip
```

**Multiple Providers at Once:**

```bash
# Forward all three OAuth ports simultaneously
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip
```

**Complete Workflow:**

1. **Establish SSH tunnel** (keep this connection open):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The iFlow port (11451) is missing from this example, while it is included in the examples just above (line 809). Adding it here would ensure consistency.

Suggested change
1. **Establish SSH tunnel** (keep this connection open):
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip

```bash
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 user@your-vps-ip
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Complete Workflow" step 1 SSH tunnel command is missing the iFlow port 11451. This is inconsistent with every other section in the PR (the "Multiple Providers at Once" block at line 811, the "Custom VPS / Systemd" section at line 1003, and DOCUMENTATION.md section 2.6.4).

A user following this workflow who also needs iFlow credentials won't have that port forwarded, causing their OAuth callback to silently fail.

Suggested change
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 user@your-vps-ip
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip

```

2. **Run the credential tool on the VPS** (in a separate terminal or SSH session):
```bash
ssh user@your-vps-ip
Comment on lines +816 to +823
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add blank lines before fenced code blocks in list items.

This section triggers MD031 (blanks-around-fences). Insert a blank line before each ```bash fence under the numbered steps.

🧰 Tools
🪛 markdownlint-cli2 (0.21.0)

[warning] 817-817: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)


[warning] 822-822: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 816 - 823, Add a blank line before each fenced code
block under the numbered list items "Establish SSH tunnel" and "Run the
credential tool on the VPS" so the Markdown has an empty line immediately above
each ```bash fence; update the two list entries that contain the code fences to
insert a blank line before their opening ```bash to satisfy MD031
(blanks-around-fences).

cd /path/to/LLM-API-Key-Proxy
python -m rotator_library.credential_tool
```

3. **Complete OAuth authentication**:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On many remote servers (especially headless VPS instances), the credential tool cannot automatically open a browser window. It usually prints a URL for the user to copy. Clarifying this (e.g., "The tool will provide a URL to open in your local browser") would be more accurate for VPS users.

- The credential tool will open a browser window
- Because of the SSH tunnel, the callback will be forwarded to your local machine
- Complete the authentication flow as normal

4. **Close SSH tunnel** after authentication is complete

**Alternative: Local Authentication + Deploy Credentials**

If you prefer not to use SSH port forwarding:

1. Complete OAuth flows locally on your machine
2. Export credentials to environment variables using the credential tool
3. Deploy the `.env` file to your remote server

See the "Stateless Deployment" section above for details on exporting credentials.

</details>

<details>
<summary><b>OAuth Callback Port Configuration</b></summary>

Expand Down Expand Up @@ -930,11 +1000,13 @@ For OAuth providers (Antigravity, Gemini CLI, etc.), you must authenticate local

```bash
# Forward callback ports through SSH
ssh -L 51121:localhost:51121 -L 8085:localhost:8085 user@your-vps
ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip

# Then run credential tool on the VPS
# Then run credential tool on the VPS in a separate terminal
```

This creates a tunnel that forwards OAuth callback ports from the VPS to your local machine, allowing the browser-based authentication to complete successfully.

**Systemd Service:**

```ini
Expand All @@ -953,6 +1025,7 @@ WantedBy=multi-user.target
```

See [VPS Deployment](Deployment%20guide.md#appendix-deploying-to-a-custom-vps) for complete guide.
See the [Remote Host Deployment (SSH Port Forwarding)](#remote-host-deployment-ssh-port-forwarding) section above for detailed OAuth setup instructions.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

The fragment link target is invalid (MD051).

#remote-host-deployment-ssh-port-forwarding does not resolve reliably because it points to <summary> text, not a markdown heading anchor.

🔗 Suggested fix
-See the [Remote Host Deployment (SSH Port Forwarding)](`#remote-host-deployment-ssh-port-forwarding`) section above for detailed OAuth setup instructions.
+See the **Remote Host Deployment (SSH Port Forwarding)** section above for detailed OAuth setup instructions.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
See the [Remote Host Deployment (SSH Port Forwarding)](#remote-host-deployment-ssh-port-forwarding) section above for detailed OAuth setup instructions.
See the **Remote Host Deployment (SSH Port Forwarding)** section above for detailed OAuth setup instructions.
🧰 Tools
🪛 markdownlint-cli2 (0.21.0)

[warning] 1028-1028: Link fragments should be valid

(MD051, link-fragments)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` at line 1028, The link fragment
"#remote-host-deployment-ssh-port-forwarding" points to a <summary> text (not a
true heading) so it doesn't resolve; fix by making the target a real anchor:
either convert the existing <summary> block "Remote Host Deployment (SSH Port
Forwarding)" into a proper Markdown heading (e.g., prefix with "##") or add an
explicit HTML anchor immediately before that section (e.g., <a
name="remote-host-deployment-ssh-port-forwarding"></a>), then update/keep the
link text "See the [Remote Host Deployment (SSH Port
Forwarding)](`#remote-host-deployment-ssh-port-forwarding`)..." to point to that
valid anchor.


</details>

Expand All @@ -967,6 +1040,7 @@ See [VPS Deployment](Deployment%20guide.md#appendix-deploying-to-a-custom-vps) f
| All keys on cooldown | All keys failed recently; check `logs/detailed_logs/` for upstream errors |
| Model not found | Verify format is `provider/model_name` (e.g., `gemini/gemini-2.5-flash`) |
| OAuth callback failed | Ensure callback port (8085, 51121, 11451) isn't blocked by firewall |
| OAuth callback failed on remote VPS | Use SSH port forwarding: `ssh -L 8085:localhost:8085 -L 51121:localhost:51121 user@your-vps-ip` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The iFlow port (11451) is also missing from this troubleshooting entry. Including it would ensure all OAuth providers are covered.

Suggested change
| OAuth callback failed on remote VPS | Use SSH port forwarding: `ssh -L 8085:localhost:8085 -L 51121:localhost:51121 user@your-vps-ip` |
| OAuth callback failed on remote VPS | Use SSH port forwarding: `ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip` |

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The troubleshooting table entry for "OAuth callback failed on remote VPS" is also missing iFlow port 11451. This is inconsistent with the "Multiple Providers at Once" command block immediately above (line 811), which correctly lists all three ports.

Suggested change
| OAuth callback failed on remote VPS | Use SSH port forwarding: `ssh -L 8085:localhost:8085 -L 51121:localhost:51121 user@your-vps-ip` |
| OAuth callback failed on remote VPS | Use SSH port forwarding: `ssh -L 8085:localhost:8085 -L 51121:localhost:51121 -L 11451:localhost:11451 user@your-vps-ip` |

| Streaming hangs | Increase `TIMEOUT_READ_STREAMING`; check provider status |

**Detailed Logs:**
Expand Down
3 changes: 3 additions & 0 deletions src/proxy_app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -703,6 +703,9 @@ async def verify_anthropic_api_key(
Dependency to verify API key for Anthropic endpoints.
Accepts either x-api-key header (Anthropic style) or Authorization Bearer (OpenAI style).
"""
# If PROXY_API_KEY is not set or empty, skip verification (open access)
if not PROXY_API_KEY:
return x_api_key or auth
Comment on lines +706 to +708
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail-open auth here should be explicit opt-in, not implicit.

Line 706 silently disables endpoint authentication when PROXY_API_KEY is unset/empty. A missing env var then exposes Anthropic endpoints without auth.

🔐 Suggested fix (explicit unauthenticated mode)
 async def verify_anthropic_api_key(
     x_api_key: str = Depends(anthropic_api_key_header),
     auth: str = Depends(api_key_header),
 ):
@@
-    # If PROXY_API_KEY is not set or empty, skip verification (open access)
-    if not PROXY_API_KEY:
-        return x_api_key or auth
+    allow_unauthenticated = (
+        os.getenv("ALLOW_UNAUTHENTICATED", "false").lower() == "true"
+    )
+    if not PROXY_API_KEY:
+        if allow_unauthenticated:
+            return x_api_key or auth
+        raise HTTPException(
+            status_code=500,
+            detail="Server misconfigured: PROXY_API_KEY is not set",
+        )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# If PROXY_API_KEY is not set or empty, skip verification (open access)
if not PROXY_API_KEY:
return x_api_key or auth
async def verify_anthropic_api_key(
x_api_key: str = Depends(anthropic_api_key_header),
auth: str = Depends(api_key_header),
):
allow_unauthenticated = (
os.getenv("ALLOW_UNAUTHENTICATED", "false").lower() == "true"
)
if not PROXY_API_KEY:
if allow_unauthenticated:
return x_api_key or auth
raise HTTPException(
status_code=500,
detail="Server misconfigured: PROXY_API_KEY is not set",
)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/proxy_app/main.py` around lines 706 - 708, The current check silently
allows open access when PROXY_API_KEY is empty (the conditional using
PROXY_API_KEY with "if not PROXY_API_KEY: return x_api_key or auth"), which must
be changed to require an explicit opt-in flag; update this branch to only bypass
authentication when a dedicated environment flag (e.g.,
PROXY_ALLOW_UNAUTHENTICATED or PROXY_AUTH_DISABLED) is set to a truthy value,
otherwise treat a missing PROXY_API_KEY as a configuration error and
deny/require authentication (return a failure rather than implicitly returning
x_api_key or auth); locate and modify the conditional that references
PROXY_API_KEY, x_api_key, and auth to enforce the explicit opt-in flag check and
add a clear error/deny path when PROXY_API_KEY is unset and the opt-in is not
enabled.

# Check x-api-key first (Anthropic style)
if x_api_key and x_api_key == PROXY_API_KEY:
return x_api_key
Expand Down
55 changes: 45 additions & 10 deletions src/rotator_library/providers/antigravity_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -747,16 +747,19 @@ def _clean_claude_schema(schema: Any, for_gemini: bool = False) -> Any:
"$id",
"$ref",
"$defs",
"$schema", # Rejected by 'parameters' key
"$schema",
"$comment",
"$vocabulary",
"$dynamicRef",
"$dynamicAnchor",
"definitions",
"default", # Rejected by 'parameters' key
"examples", # Rejected by 'parameters' key
"default", # Rejected by 'parameters' key, sometimes
"examples", # Rejected by 'parameters' key, sometimes
"title", # May cause issues in nested objects
}

# Validation keywords - only remove at schema-definition level,
# NOT when they appear as property names under "properties"
# Note: These are common property names that could be used by tools:
# Validation keywords to strip ONLY for Claude (Gemini accepts these)
# These are common property names that could be used by tools:
# - "pattern" (glob, grep, regex tools)
# - "format" (export, date/time tools)
# - "minimum"/"maximum" (range tools)
Expand All @@ -765,27 +768,55 @@ def _clean_claude_schema(schema: Any, for_gemini: bool = False) -> Any:
# but we now use 'parameters' key which may silently ignore some):
# Note: $schema, default, examples, title moved to meta_keywords (always stripped)
validation_keywords_claude_only = {
# Array validation - Gemini accepts
"minItems",
"maxItems",
"uniqueItems",
# String validation - Gemini accepts
"pattern",
"minLength",
"maxLength",
"format",
# Number validation - Gemini accepts
"minimum",
"maximum",
# Object validation - Gemini accepts
"minProperties",
"maxProperties",
# Composition - Gemini accepts
"not",
"prefixItems",
}

# Validation keywords to strip for ALL models (Gemini and Claude)
validation_keywords_all_models = {
# Number validation - Gemini rejects
"exclusiveMinimum",
"exclusiveMaximum",
"multipleOf",
"format",
"minProperties",
"maxProperties",
# Array validation - Gemini rejects
"uniqueItems",
"contains",
"minContains",
"maxContains",
"unevaluatedItems",
# Object validation - Gemini rejects
"propertyNames",
"unevaluatedProperties",
"dependentRequired",
"dependentSchemas",
# Content validation - Gemini rejects
"contentEncoding",
"contentMediaType",
"contentSchema",
# Meta annotations - Gemini rejects
"examples",
"deprecated",
"readOnly",
"writeOnly",
# Conditional - Gemini rejects
"if",
"then",
"else",
}

# Handle 'anyOf', 'oneOf', and 'allOf' for Claude
Expand Down Expand Up @@ -891,6 +922,10 @@ def _clean_claude_schema(schema: Any, for_gemini: bool = False) -> Any:
# For Claude: skip - not supported
continue

# Strip keywords unsupported by ALL models (both Gemini and Claude)
if key in validation_keywords_all_models:
continue

# Special handling for additionalProperties:
# For Gemini: pass through as-is (Gemini accepts {}, true, false, typed schemas)
# For Claude: normalize permissive values ({} or true) to true
Expand Down
Loading