Skip to content

Commit 4215f63

Browse files
committed
feat: native bidings
1 parent bcc708e commit 4215f63

19 files changed

Lines changed: 2616 additions & 136 deletions

File tree

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: Build Native Wheels (codex-native)
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v*'
7+
workflow_dispatch: {}
8+
9+
permissions:
10+
contents: read
11+
id-token: write
12+
13+
jobs:
14+
build:
15+
continue-on-error: ${{ matrix.allow-failure == true }}
16+
strategy:
17+
fail-fast: false
18+
matrix:
19+
os: [ubuntu-latest, macos-14, windows-latest]
20+
python-version: ['3.12', '3.13']
21+
include:
22+
- os: ubuntu-latest
23+
python-version: '3.14'
24+
allow-failure: true
25+
runs-on: ${{ matrix.os }}
26+
steps:
27+
- name: Checkout repository
28+
uses: actions/checkout@v4
29+
30+
- name: Set up Rust
31+
uses: dtolnay/rust-toolchain@stable
32+
33+
- name: Set up Python
34+
uses: actions/setup-python@v5
35+
with:
36+
python-version: ${{ matrix.python-version }}
37+
38+
- name: Install maturin
39+
run: |
40+
python -m pip install --upgrade pip
41+
pip install maturin
42+
43+
- name: Build wheels with maturin
44+
env:
45+
PYO3_USE_ABI3_FORWARD_COMPATIBILITY: '1'
46+
run: |
47+
maturin build -m crates/codex_native/Cargo.toml --release --interpreter python
48+
49+
- name: Upload wheels
50+
uses: actions/upload-artifact@v4
51+
with:
52+
name: wheels-${{ matrix.os }}-${{ matrix.python-version }}
53+
path: crates/codex_native/target/wheels/*.whl
54+
55+
publish:
56+
needs: build
57+
runs-on: ubuntu-latest
58+
steps:
59+
- name: Download all wheels
60+
uses: actions/download-artifact@v4
61+
with:
62+
path: dist
63+
64+
- name: Publish to PyPI (Trusted Publishing)
65+
uses: pypa/gh-action-pypi-publish@release/v1
66+
with:
67+
packages-dir: dist
68+
skip-existing: true

.github/workflows/publish.yml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ on:
77

88
permissions:
99
contents: read
10+
id-token: write
1011

1112
jobs:
1213
build-and-publish:
@@ -28,7 +29,5 @@ jobs:
2829
- name: Build distribution
2930
run: uv build
3031

31-
- name: Publish to PyPI
32-
env:
33-
PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
34-
run: uv publish --token "$PYPI_API_TOKEN"
32+
- name: Publish to PyPI (Trusted Publishing)
33+
run: uv publish --trusted-publishing=always

.gitignore

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,24 @@ coverage.xml
4242

4343
# uv
4444
.uv/
45+
uv.lock
4546

47+
# Generated artifacts
48+
# Generated artifacts
49+
.generated/
50+
codex-proj/
51+
.githooks/
52+
test/
53+
tests/
54+
55+
# Ignore new markdown files by default
56+
*.md
57+
58+
# Rust build outputs
59+
target/
60+
crates/**/target/
61+
crates/**/Cargo.lock
62+
63+
# Local environment files
64+
.env
65+
.env.*

CHANGELOG.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,15 @@ All notable changes to this project will be documented in this file.
44

55
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
66

7+
## [0.1.1] - 2025-09-10
8+
### Added
9+
- CodexClient synchronous wrapper with defaults
10+
- Python API `run_exec` with robust error handling
11+
12+
### Changed
13+
- Switch publish workflow to PyPI Trusted Publishing (OIDC)
14+
- Docs and Makefile updates
15+
716
## [0.1.0] - 2025-09-10
817
### Added
918
- Initial project scaffold with Python 3.13+
@@ -15,3 +24,4 @@ The format is based on Keep a Changelog and this project adheres to Semantic Ver
1524
- MIT License
1625

1726
[0.1.0]: https://github.com/gersmann/codex-python/releases/tag/v0.1.0
27+
[0.1.1]: https://github.com/gersmann/codex-python/releases/tag/v0.1.1

CONTRIBUTING.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ This project is typed and ships a `py.typed` marker. Please keep public APIs typ
3636
2. Update `CHANGELOG.md`.
3737
3. Merge to `main`.
3838
4. Tag the release: `git tag -a vX.Y.Z -m "vX.Y.Z" && git push --tags`.
39-
5. GitHub Actions (publish workflow) will build and publish to PyPI on `v*` tags.
39+
5. GitHub Actions (publish workflow) will build and publish to PyPI on `v*` tags using Trusted Publishing (OIDC). No token is required.
4040

4141
## Pre-commit (optional but recommended)
4242
```
@@ -51,4 +51,3 @@ Please open an issue with reproduction steps, expected vs actual behavior, and e
5151

5252
## Code of Conduct
5353
By participating, you agree to abide by our [Code of Conduct](CODE_OF_CONDUCT.md).
54-

Makefile

Lines changed: 45 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ help:
77
@echo " make build - Build sdist and wheel with uv"
88
@echo " make publish - Publish to PyPI via uv (uses PYPI_API_TOKEN)"
99
@echo " make clean - Remove build artifacts"
10+
@echo " make gen-protocol - Generate Python protocol bindings from codex-rs"
1011

1112
venv:
1213
uv venv --python 3.13
@@ -16,22 +17,60 @@ fmt:
1617
uv run --group dev ruff format .
1718

1819
lint:
19-
uv run --group dev ruff format --check .
20-
uv run --group dev ruff check .
20+
uv run --group dev ruff format .
21+
uv run --group dev ruff check --fix --unsafe-fixes .
2122
uv run --group dev mypy codex
2223

2324
test:
24-
uv run --group dev pytest
25+
@bash -lc 'uv run --group dev pytest -q; ec=$$?; if [ $$ec -eq 5 ]; then echo "No tests collected"; exit 0; else exit $$ec; fi'
2526

2627
build:
2728
uv build
2829

2930
publish: build
30-
@if [ -z "$${PYPI_API_TOKEN}" ]; then \
31-
echo "PYPI_API_TOKEN is not set"; \
31+
@# Load local environment if present
32+
@set -e; \
33+
if [ -f .env ]; then set -a; . ./.env; set +a; fi; \
34+
if [ -n "$${UV_PUBLISH_TOKEN:-}" ]; then \
35+
echo "Publishing with token (UV_PUBLISH_TOKEN)"; \
36+
uv publish --token "$${UV_PUBLISH_TOKEN}"; \
37+
elif [ -n "$${PYPI_API_TOKEN:-}" ]; then \
38+
echo "Publishing with token (PYPI_API_TOKEN)"; \
39+
uv publish --token "$${PYPI_API_TOKEN}"; \
40+
elif [ -n "$${UV_PUBLISH_USERNAME:-}" ] && [ -n "$${UV_PUBLISH_PASSWORD:-}" ]; then \
41+
echo "Publishing with username/password (UV_PUBLISH_USERNAME)"; \
42+
uv publish --username "$${UV_PUBLISH_USERNAME}" --password "$${UV_PUBLISH_PASSWORD}"; \
43+
elif [ -n "$${PYPI_USERNAME:-}" ] && [ -n "$${PYPI_PASSWORD:-}" ]; then \
44+
echo "Publishing with username/password (PYPI_USERNAME)"; \
45+
uv publish --username "$${PYPI_USERNAME}" --password "$${PYPI_PASSWORD}"; \
46+
else \
47+
echo "No credentials found. Set UV_PUBLISH_TOKEN or PYPI_API_TOKEN, or UV_PUBLISH_USERNAME/UV_PUBLISH_PASSWORD (or PYPI_USERNAME/PYPI_PASSWORD)."; \
3248
exit 1; \
3349
fi
34-
uv publish --token "$$PYPI_API_TOKEN"
3550

3651
clean:
3752
rm -rf build dist *.egg-info .pytest_cache .mypy_cache .ruff_cache
53+
54+
gen-protocol:
55+
@echo "Generating TypeScript protocol types via codex-proj/codex-rs ..."
56+
@mkdir -p .generated/ts
57+
@if command -v codex >/dev/null 2>&1; then \
58+
echo "Using 'codex generate-types'"; \
59+
codex generate-types --out .generated/ts; \
60+
else \
61+
echo "Using cargo run -p codex-protocol-ts"; \
62+
cd codex-proj/codex-rs && cargo run -p codex-protocol-ts -- --out ../../.generated/ts; \
63+
fi
64+
@echo "Generating Python bindings ..."
65+
@python3 scripts/generate_protocol_py.py .generated/ts
66+
@$(MAKE) fmt
67+
68+
.PHONY: build-native dev-native
69+
70+
build-native:
71+
@echo "Building native extension with maturin..."
72+
@maturin build -m crates/codex_native/Cargo.toml --release
73+
74+
dev-native:
75+
@echo "Installing native extension in dev mode..."
76+
@maturin develop -m crates/codex_native/Cargo.toml

README.md

Lines changed: 76 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# codex-python
22

3-
A minimal Python library scaffold using `uv` with Python 3.13+.
3+
Python interface to Codex, packaged as a single distribution (`codex-python`). Platform wheels include a native extension for in‑process execution; pure‑Python installs fall back to the CLI JSON mode.
44

55
## Quickstart
66

7-
- Requires Python 3.13+.
7+
- Requires Python 3.12+.
88
- Package import name: `codex`.
99
- Distribution name (PyPI): `codex-python`.
1010

@@ -30,6 +30,27 @@ Options:
3030
- Full auto: `run_exec("scaffold a cli", full_auto=True)`
3131
- Run in another dir: `run_exec("...", cd="/path/to/project")`
3232

33+
Streaming JSON events (CLI; no native required):
34+
35+
```
36+
from codex.protocol.runtime import stream_exec_events
37+
38+
for event in stream_exec_events("explain this repo", full_auto=True):
39+
# event is a dict with shape {"id": str, "msg": {...}}
40+
print(event)
41+
```
42+
43+
The event payload matches the Pydantic models in `codex.protocol.types` (e.g., `EventMsg`).
44+
45+
Using a client with defaults:
46+
47+
```
48+
from codex import CodexClient
49+
50+
client = CodexClient(model="gpt-4.1", full_auto=True)
51+
print(client.run("explain this repo"))
52+
```
53+
3354
### Install uv
3455

3556
- macOS (Homebrew): `brew install uv`
@@ -64,8 +85,10 @@ export PYPI_API_TOKEN="pypi-XXXX" # create at https://pypi.org/manage/account/t
6485
uv publish --token "$PYPI_API_TOKEN"
6586
```
6687

67-
- GitHub Actions: add a repository secret `PYPI_API_TOKEN` and push a tag like `v0.1.0`.
68-
The workflow at `.github/workflows/publish.yml` builds and publishes with `uv` on `v*` tags.
88+
- GitHub Actions (Trusted Publishing): enable PyPI Trusted Publishing for
89+
`gersmann/codex-python` and push a tag like `v0.1.0`. No token is needed.
90+
The workflow at `.github/workflows/publish.yml` requests an OIDC token and
91+
runs `uv publish --trusted-publishing=always` on `v*` tags.
6992

7093
### Dev tooling
7194

@@ -74,6 +97,54 @@ uv publish --token "$PYPI_API_TOKEN"
7497
- Format: `make fmt` (ruff formatter)
7598
- Pre-commit: `uvx pre-commit install && uvx pre-commit run --all-files`
7699

100+
### Protocol bindings (from codex-rs)
101+
102+
- Prereq: Rust toolchain (`cargo`) installed.
103+
- Generate Python types from the upstream protocol with:
104+
105+
```
106+
make gen-protocol
107+
```
108+
109+
This will:
110+
- emit TypeScript types under `.generated/ts/`
111+
- convert them to Python Pydantic models + Literal unions at `codex/protocol/types.py`
112+
113+
### Native bindings (PyO3)
114+
115+
- Install path
116+
117+
`pip install codex-python` installs a platform wheel that includes the native extension on supported platforms (Python 3.12/3.13; CI also attempts 3.14). If a platform wheel isn’t available, pip installs a pure‑Python build and CLI JSON streaming still works.
118+
119+
- Build locally (requires Rust + maturin):
120+
121+
```
122+
make dev-native
123+
```
124+
125+
- Use (collect all events):
126+
127+
```
128+
from codex.native import run_exec_collect
129+
130+
events = run_exec_collect("say hello", full_auto=True)
131+
for e in events:
132+
print(e)
133+
```
134+
135+
- Use (streaming iterator):
136+
137+
```
138+
from codex.native import start_exec_stream
139+
140+
for e in start_exec_stream("say hello", full_auto=True):
141+
print(e)
142+
```
143+
144+
- Notes:
145+
- The native path embeds Codex directly; no subprocess.
146+
- Native is optional; when not present, use the CLI JSON streaming helpers.
147+
77148
## Project Layout
78149

79150
```
@@ -91,4 +162,4 @@ Version is managed via `codex/__init__.py` and exposed as `__version__`. The bui
91162

92163
## Python Compatibility
93164

94-
- Requires Python `>=3.13`.
165+
- Supported: Python 3.12, 3.13. CI attempts 3.14 wheels when available.

codex/__init__.py

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,19 +4,29 @@
44
55
Usage:
66
from codex import run_exec
7-
output = run_exec("explain this codebase to me")
7+
events = run_exec("explain this codebase to me")
88
"""
99

10-
from .api import CodexError, CodexNotFoundError, CodexProcessError, find_binary, run_exec
10+
from .api import (
11+
CodexClient,
12+
CodexError,
13+
CodexNativeError,
14+
Conversation,
15+
run_exec,
16+
)
17+
from .config import CodexConfig
18+
from .event import Event
1119

1220
__all__ = [
1321
"__version__",
1422
"CodexError",
15-
"CodexNotFoundError",
16-
"CodexProcessError",
17-
"find_binary",
23+
"CodexNativeError",
24+
"CodexClient",
25+
"Conversation",
1826
"run_exec",
27+
"Event",
28+
"CodexConfig",
1929
]
2030

2131
# Managed by Hatch via pyproject.toml [tool.hatch.version]
22-
__version__ = "0.1.0"
32+
__version__ = "0.1.2"

0 commit comments

Comments
 (0)