Skip to content
126 changes: 61 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ This project is a swiss-army knife for anyone working with language models and a

## Contents

- [Use Cases](#use-cases)
- [Killer Features](#killer-features)
- [Supported APIs (Front-Ends) and Providers (Back-Ends)](#supported-apis-front-ends-and-providers-back-ends)
- [Gemini Backends Overview](#gemini-backends-overview)
Expand All @@ -16,8 +17,8 @@ This project is a swiss-army knife for anyone working with language models and a
- [Popular Scenarios](#popular-scenarios)
- [Errors and Troubleshooting](#errors-and-troubleshooting)
- [Support](#support)
- [License](#license)
- [Changelog](#changelog)
- [License](#license)

## Use Cases

Expand All @@ -39,7 +40,7 @@ This project is a swiss-army knife for anyone working with language models and a
- Multiple front-ends, many providers: exposes OpenAI, Anthropic, and Gemini APIs while routing to OpenAI, Anthropic, Gemini, OpenRouter, ZAI, Qwen, and more
- OpenAI compatibility: drop-in `/v1/chat/completions` for most clients and coding agents
- Streaming everywhere: consistent streaming and non‑streaming support across providers
- Gemini OAuth personal gateway: use Googles free personal OAuth (CLI-style) through an OpenAI-compatible endpoint
- Gemini OAuth personal gateway: use Google's free personal OAuth (CLI-style) through an OpenAI-compatible endpoint

### Reliability

Expand All @@ -63,7 +64,7 @@ This project is a swiss-army knife for anyone working with language models and a

- Wire capture and audit: optional request/response capture file plus usage tracking

## Supported LLM APIs
## Supported APIs (Front-Ends) and Providers (Back-Ends)

These are ready out of the box. Front-ends are the client-facing APIs the proxy exposes; back-ends are the providers the proxy calls.

Expand Down Expand Up @@ -91,6 +92,55 @@ These are ready out of the box. Front-ends are the client-facing APIs the proxy
| `zai-coding-plan` | ZAI Coding Plan | `ZAI_API_KEY` | Works with any supported front-end and coding agent |
| `qwen-oauth` | Alibaba Qwen | Local `oauth_creds.json` | Qwen CLI OAuth; OpenAI-compatible endpoint |

## Gemini Backends Overview

Choose the Gemini integration that fits your environment.

| Backend | Authentication | Cost | Best for |
| - | - | - | - |
| `gemini` | API key (`GEMINI_API_KEY`) | Metered (pay-per-use) | Production apps, high-volume usage |
| `gemini-cli-oauth-personal` | OAuth (no API key) | Free tier with limits | Local development, testing, personal use |
| `gemini-cli-cloud-project` | OAuth + `GOOGLE_CLOUD_PROJECT` (ADC/service account) | Billed to your GCP project | Enterprise, team workflows, central billing |

Notes

- Personal OAuth uses credentials from the local Google CLI/Code Assist-style flow and does not require a `GEMINI_API_KEY`.
- Cloud Project requires `GOOGLE_CLOUD_PROJECT` and Application Default Credentials (or a service account file).

Quick setup

For `gemini` (API key)

```bash
export GEMINI_API_KEY="AIza..."
python -m src.core.cli --default-backend gemini
```

For `gemini-cli-oauth-personal` (free personal OAuth)

```bash
# Install and authenticate with the Google Gemini CLI (one-time):
gemini auth

# Then start the proxy using the personal OAuth backend
python -m src.core.cli --default-backend gemini-cli-oauth-personal
```

For `gemini-cli-cloud-project` (GCP-billed)

```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"

# Provide Application Default Credentials via one of the following:
# Option A: User credentials (interactive)
gcloud auth application-default login

# Option B: Service account file
export GOOGLE_APPLICATION_CREDENTIALS="/absolute/path/to/service-account.json"

python -m src.core.cli --default-backend gemini-cli-cloud-project
```

## Quick Start

1) Export provider keys (only for the back-ends you plan to use)
Expand All @@ -114,12 +164,7 @@ python -m src.core.cli --default-backend openai
Useful flags

- `--host 0.0.0.0` and `--port 8000` to change bind address
- `--config config/config.example.yaml` to load a saved config

### Configuration Format
- Configuration files are YAML-only. JSON configs are deprecated and not supported by the loader.
- Keep `pyproject.toml` for packaging/tooling, and JSON for wire-capture logs only.
- On startup, the server validates YAML files and refuses to start on syntax/schema errors.
- `--config config/config.example.yaml` to load a saved config
- `--capture-file wire.log` to record requests/replies (see Debugging)
- `--disable-auth` for local only (forces host=127.0.0.1)

Expand Down Expand Up @@ -168,66 +213,13 @@ Write outbound requests and inbound replies/streams to a rotating file for troub
- Content rewriting: REPLACE/PREPEND/APPEND rules on inbound/outbound content
- Context window enforcement: per-model token limits with friendly errors

## Gemini Backends Overview

Choose the Gemini integration that fits your environment.

| Backend | Authentication | Cost | Best for |
| - | - | - | - |
| `gemini` | API key (`GEMINI_API_KEY`) | Metered (pay-per-use) | Production apps, high-volume usage |
| `gemini-cli-oauth-personal` | OAuth (no API key) | Free tier with limits | Local development, testing, personal use |
| `gemini-cli-cloud-project` | OAuth + `GOOGLE_CLOUD_PROJECT` (ADC/service account) | Billed to your GCP project | Enterprise, team workflows, central billing |

Notes

- Personal OAuth uses credentials from the local Google CLI/Code Assist-style flow and does not require a `GEMINI_API_KEY`.
- Cloud Project requires `GOOGLE_CLOUD_PROJECT` and Application Default Credentials (or a service account file).

Quick setup

For `gemini` (API key)

```bash
export GEMINI_API_KEY="AIza..."
python -m src.core.cli --default-backend gemini
```

For `gemini-cli-oauth-personal` (free personal OAuth)

```bash
# Install and authenticate with the Google Gemini CLI (one-time):
gemini auth

# Then start the proxy using the personal OAuth backend
python -m src.core.cli --default-backend gemini-cli-oauth-personal
```

For `gemini-cli-cloud-project` (GCP-billed)

```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"

# Provide Application Default Credentials via one of the following:
# Option A: User credentials (interactive)
gcloud auth application-default login

# Option B: Service account file
export GOOGLE_APPLICATION_CREDENTIALS="/absolute/path/to/service-account.json"

python -m src.core.cli --default-backend gemini-cli-cloud-project
```

## Example Config (minimal)

```yaml
# config.yaml
backends:
openai:
type: openai

## License

This project is licensed under the GNU Affero General Public License v3.0 (or later). See the `LICENSE` file for the full text.
default_backend: openai
proxy:
host: 0.0.0.0
Expand All @@ -239,7 +231,7 @@ auth:

Run: `python -m src.core.cli --config config.yaml`

## Popular Usage Scenarios
## Popular Scenarios

### Claude Code with any model/provider

Expand Down Expand Up @@ -284,8 +276,12 @@ Then launch `claude`. You can switch models during a session:

## Support

- Issues: open a ticket in the repository’s issue tracker
- Issues: open a ticket in the repository's issue tracker

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Changelog

- See the full change history in [CHANGELOG.md](CHANGELOG.md)
See the full change history in [CHANGELOG.md](CHANGELOG.md)