Skip to content

Commit 9e9875f

Browse files
committed
refactor: finish renaming stuff for ai-provider now that ai-provider is in ai
1 parent 17d9814 commit 9e9875f

411 files changed

Lines changed: 1117 additions & 2868 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/CLAUDE.md

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@ task-graph (core DAG pipeline engine)
4141
4242
dataset, tasks (KnowledgeBase, documents, chunks; utility tasks)
4343
44-
ai (AI task base classes, model registry)
44+
ai (AI task base classes, model registry, provider helpers)
4545
46-
ai-provider (concrete provider implementations)
46+
providers/* (concrete provider implementations: anthropic, openai, gemini, ollama, ...)
4747
4848
test (integration tests across all packages)
4949
workglow (meta-package re-exporting everything)
@@ -61,7 +61,7 @@ Each package builds three runtime targets via `bun build --target=X`:
6161

6262
Types built with `tsc` (composite + incremental). Conditional exports in `package.json` resolve automatically per runtime.
6363

64-
Exception: `ai-provider` builds per-provider sub-paths (`./anthropic`, `./openai`, `./google-gemini`, etc.) instead of browser/node/bun.
64+
Exception: vendor packages under `providers/*` (e.g. `@workglow/anthropic`, `@workglow/openai`, `@workglow/google-gemini`) ship `./ai` and `./ai-runtime` sub-paths instead of browser/node/bun.
6565

6666
Exception: `util` has multiple named exports beyond `"."`:
6767

@@ -178,18 +178,21 @@ Task categories: text generation/embedding/summary/translation/rewriting/classif
178178

179179
RAG tasks: `ChunkVectorUpsertTask` (input: `knowledgeBase` + `chunks` + `vector`, optional `doc_title`), `ChunkRetrievalTask` (input: `knowledgeBase` + `query` + `model`, with `method: "similarity" | "hybrid"`), `HierarchyJoinTask`, `RerankerTask`, `QueryExpanderTask`, `TextChunkerTask`, `HierarchicalChunkerTask`.
180180

181-
### `@workglow/ai-provider` — provider implementations
181+
### `providers/*` — provider implementations
182182

183-
Each provider is a separate sub-export with optional peer dependencies:
183+
Each provider is a standalone package with optional third-party peer dependencies. They each expose `./ai` (main-thread shell) and `./ai-runtime` (worker / inline runtime):
184184

185-
- `@workglow/ai-provider/anthropic` — Claude
186-
- `@workglow/ai-provider/openai` — OpenAI
187-
- `@workglow/ai-provider/gemini` — Gemini
188-
- `@workglow/ai-provider/ollama` — Ollama (browser + node)
189-
- `@workglow/ai-provider/hf-transformers` — HuggingFace Transformers.js
190-
- `@workglow/ai-provider/hf-inference` — HuggingFace Inference API
191-
- `@workglow/ai-provider/llamacpp` — node-llama-cpp
192-
- `@workglow/ai-provider/tf-mediapipe` — TensorFlow MediaPipe (browser)
185+
- `@workglow/anthropic` — Claude
186+
- `@workglow/openai` — OpenAI
187+
- `@workglow/google-gemini` — Gemini
188+
- `@workglow/ollama` — Ollama (browser + node)
189+
- `@workglow/huggingface-transformers` — HuggingFace Transformers.js
190+
- `@workglow/huggingface-inference` — HuggingFace Inference API
191+
- `@workglow/node-llama-cpp` — node-llama-cpp
192+
- `@workglow/tf-mediapipe` — TensorFlow MediaPipe (browser)
193+
- `@workglow/chrome-ai` — built-in Chrome / WebBrowser AI
194+
195+
Shared cloud-provider helpers (base classes, registration, model search, OpenAI-shape chat, image-output conversion, tool-call parsing) live in `@workglow/ai/provider-utils` and are imported by every vendor package.
193196

194197
**Important: `*_JobRunFns.ts` files execute inside workers.** Workers have an isolated runtime with a separate `globalServiceRegistry`. Do not access main-thread-only state (e.g., credential stores, service registries) from run functions. Instead, resolve such state in the task class on the main thread (e.g., `AiTask.getJobInput()`) and pass the resolved values through the serialized job input.
195198

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ dist
66
*.d.ts
77
*.d.ts.map
88
!examples/web/src/vite-env.d.ts
9-
!packages/ai-provider/src/provider-chrome/common/WebBrowser_ChromeAI.d.ts
9+
!providers/chrome-ai/src/ai/common/WebBrowser_ChromeAI.d.ts
1010
!packages/tasks/src/util/interpreter.d.ts
1111
storybook-static/
1212
packages/cli-web/

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ Packages:
6464
- **[packages/job-queue](packages/job-queue/README.md)**
6565
- **[packages/tasks](packages/tasks/README.md)**
6666
- **[packages/ai](packages/ai/README.md)**
67-
- **[packages/ai-provider](packages/ai-provider/README.md)**
67+
- **[providers/*](providers/) — vendor provider packages (Anthropic, OpenAI, Gemini, Ollama, HuggingFace, llama.cpp, MediaPipe, Chrome AI, Playwright, Electron, SQLite, Postgres, Supabase, bun-webview)**
6868
- **[packages/util](packages/util/README.md)**
6969
- **[packages/test](packages/test/README.md)**
7070

bun.lock

Lines changed: 25 additions & 98 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/developers/01_getting_started.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
- [`packages/job-queue`](#packagesjob-queue)
1818
- [`packages/task-graph`](#packagestask-graph)
1919
- [`packages/ai`](#packagesai)
20-
- [`packages/ai-provider`](#packagesai-provider)
20+
- [`providers/*`](#providers)
2121
- [`packages/util`](#packagesutil)
2222
- [`examples/cli`](#examplescli)
2323
- [`examples/web`](#examplesweb)
@@ -559,9 +559,9 @@ This is the main task handling library, with tasks, compound tasks, data flows,
559559

560560
These are the LLM tasks, models, etc. These tasks are agnostic to the provider and thus are like abstract versions. AI providers contribute the concrete implementations. Which implementation is used is determined by the model repository.
561561

562-
### `packages/ai-provider`
562+
### `providers/*`
563563

564-
This is the Huggingface Transformers JS (using ONNX) and TensorFlow MediaPipe providers.
564+
Each AI provider lives in its own package under `providers/`: `@workglow/anthropic`, `@workglow/openai`, `@workglow/google-gemini`, `@workglow/ollama`, `@workglow/huggingface-transformers`, `@workglow/huggingface-inference`, `@workglow/node-llama-cpp`, `@workglow/tf-mediapipe`, and `@workglow/chrome-ai`. Each ships an `./ai` and `./ai-runtime` subpath. Shared helpers (cloud client base classes, registration utilities, OpenAI-shape chat plumbing) live in `@workglow/ai/provider-utils`. Non-AI vendor packages (Playwright, Electron, SQLite, Postgres, Supabase, bun-webview) also live under `providers/`.
565565

566566
### `packages/util`
567567

docs/technical/11-ai-provider-system.md

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -382,27 +382,29 @@ yields a single `finish` event so GPU serialization is still respected.
382382

383383
---
384384

385-
## Sub-Path Exports
385+
## Vendor Provider Packages
386386

387-
Unlike other packages that build per-runtime targets (`browser.ts`, `node.ts`, `bun.ts`), the
388-
`@workglow/ai-provider` package builds per-provider sub-paths. Each provider is a separate import
389-
with optional peer dependencies:
387+
Unlike core packages that build per-runtime targets (`browser.ts`, `node.ts`, `bun.ts`), each
388+
provider lives in its own package under `providers/*` with optional peer dependencies on the
389+
underlying vendor SDK. Providers import shared base classes and helpers from
390+
`@workglow/ai/provider-utils`.
390391

391392
```typescript
392-
import "@workglow/ai-provider/anthropic"; // Claude (requires @anthropic-ai/sdk)
393-
import "@workglow/ai-provider/openai"; // OpenAI (requires openai)
394-
import "@workglow/ai-provider/gemini"; // Google Gemini (requires @google/generative-ai)
395-
import "@workglow/ai-provider/ollama"; // Ollama (requires ollama)
396-
import "@workglow/ai-provider/hf-transformers"; // HuggingFace Transformers.js
397-
import "@workglow/ai-provider/hf-inference"; // HuggingFace Inference API
398-
import "@workglow/ai-provider/llamacpp"; // node-llama-cpp
399-
import "@workglow/ai-provider/tf-mediapipe"; // TensorFlow MediaPipe (browser)
400-
import "@workglow/ai-provider/chrome"; // Chrome Built-in AI
393+
import "@workglow/anthropic/ai"; // Claude (requires @anthropic-ai/sdk)
394+
import "@workglow/openai/ai"; // OpenAI (requires openai)
395+
import "@workglow/google-gemini/ai"; // Google Gemini (requires @google/generative-ai)
396+
import "@workglow/ollama/ai"; // Ollama (requires ollama)
397+
import "@workglow/huggingface-transformers/ai"; // HuggingFace Transformers.js
398+
import "@workglow/huggingface-inference/ai"; // HuggingFace Inference API
399+
import "@workglow/node-llama-cpp/ai"; // node-llama-cpp
400+
import "@workglow/tf-mediapipe/ai"; // TensorFlow MediaPipe (browser)
401+
import "@workglow/chrome-ai/ai"; // Chrome Built-in AI
401402
```
402403

403-
Each sub-path also has a `/runtime` variant (e.g., `@workglow/ai-provider/anthropic/runtime`) that
404-
exports the heavy run function implementations and worker registration helpers. The main sub-path
405-
exports only the lightweight provider class, constants, and the worker-backed registration function.
404+
Each vendor package also exposes an `/ai-runtime` entry (e.g.
405+
`@workglow/anthropic/ai-runtime`) that exports the heavy run function implementations
406+
and worker registration helpers. The `/ai` entry exports only the lightweight provider
407+
class, constants, and the worker-backed registration function.
406408

407409
Some providers (Ollama, OpenAI) also have browser-specific conditional exports in `package.json`.
408410

docs/technical/13-worker-system.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -383,8 +383,9 @@ important implications:
383383
keeping the worker bundle small.
384384

385385
4. **Provider run functions execute in workers.** Files named `*_JobRunFns.ts`
386-
in the `ai-provider` package contain the actual inference logic that runs
387-
inside workers. These functions must not import main-thread-only modules.
386+
in each vendor provider package (under `providers/*`) contain the actual
387+
inference logic that runs inside workers. These functions must not import
388+
main-thread-only modules.
388389

389390
## Provider Integration
390391

docs/technical/15-credential-management.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -376,8 +376,8 @@ the `onExpiry` callback is invoked -- typically used to lock a
376376
- The Web Crypto API (`crypto.subtle`) is available natively -- no polyfills needed.
377377
- Provider clients (Anthropic, OpenAI) set `dangerouslyAllowBrowser: true`
378378
when they detect a browser or web worker context (see
379-
`packages/ai-provider/src/provider-anthropic/common/Anthropic_Client.ts:48`
380-
and `packages/ai-provider/src/provider-openai/common/OpenAI_Client.ts:49`).
379+
`providers/anthropic/src/ai/common/Anthropic_Client.ts`
380+
and `providers/openai/src/ai/common/OpenAI_Client.ts`).
381381
This is **not** automatically safe: any API key loaded into a browser
382382
context — even one decrypted from `EncryptedKvCredentialStore` at runtime —
383383
is reachable from the page and should be treated as exposed to the end user.

docs/technical/19-build-system.md

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -171,18 +171,18 @@ Here, `build-js` itself expands into many concurrent builds (browser, node, bun,
171171
graph, media, compress), and `build-types` runs after all JS builds complete. The sequential
172172
ordering (`&&`) ensures that type generation can reference the built artifacts.
173173

174-
**Provider package (`@workglow/ai-provider`):**
174+
**Vendor provider packages (`providers/*`):**
175175

176176
```json
177177
{
178-
"build-package": "concurrently -c 'auto' -n 'code,browser,types' 'bun run build-code' 'bun run build-browser' 'bun run build-types'"
178+
"build-package": "concurrently -c 'auto' -n 'code,types' 'bun run build-code' 'bun run build-types'"
179179
}
180180
```
181181

182-
The ai-provider package does not follow the standard browser/node/bun entry point pattern.
183-
Instead, it builds per-provider sub-paths (anthropic, openai, gemini, etc.) with `--root ./src`
184-
to preserve directory structure, plus a separate browser-specific build for providers that support
185-
browser environments (ollama, openai, tf-mediapipe).
182+
Each vendor package (`@workglow/anthropic`, `@workglow/openai`, `@workglow/google-gemini`, ...)
183+
ships two entry points — `./ai` (main thread) and `./ai-runtime` (worker /
184+
inline runtime) — built with `--root ./src` to preserve directory structure. Browser-capable
185+
providers (ollama, openai) add `*.browser.ts` variants with `--target=browser`.
186186

187187
### `build-js` — JavaScript Only
188188

@@ -268,18 +268,16 @@ Platform-agnostic sub-paths (schema, graph) are built with `--target=browser` si
268268
code runs everywhere. Platform-specific sub-paths (media, compress, worker) are built with the
269269
matching target for each variant.
270270

271-
### Provider Pattern (ai-provider)
271+
### Provider Pattern (vendor packages under `providers/*`)
272272

273-
`@workglow/ai-provider` uses `--root ./src` to build multiple entry points in a single invocation
274-
while preserving their subdirectory structure in the output:
273+
Each vendor provider package (e.g. `@workglow/anthropic`, `@workglow/openai`) uses `--root ./src`
274+
to build the `ai` and `ai-runtime` entry points in a single invocation while
275+
preserving their subdirectory structure in the output:
275276

276277
```bash
277278
bun build --sourcemap=external --packages=external --root ./src --outdir ./dist \
278-
./src/provider-anthropic/index.ts \
279-
./src/provider-anthropic/runtime.ts \
280-
./src/provider-gemini/index.ts \
281-
./src/provider-gemini/runtime.ts \
282-
# ... more providers
279+
./src/ai.ts \
280+
./src/ai-runtime.ts
283281
```
284282

285283
This produces output like:

eslint.config.js

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,25 @@
11
import tsPlugin from "@typescript-eslint/eslint-plugin";
22
import tsParser from "@typescript-eslint/parser";
3-
import globals from "globals";
3+
import jsxA11yPlugin from "eslint-plugin-jsx-a11y";
44
import reactPlugin from "eslint-plugin-react";
55
import reactHooksPlugin from "eslint-plugin-react-hooks";
6-
import jsxA11yPlugin from "eslint-plugin-jsx-a11y";
76
import regexpPlugin from "eslint-plugin-regexp";
7+
import globals from "globals";
88

99
export default [
1010
{
1111
ignores: ["dist/**", "**/storybook-static/**"],
1212
},
1313
{
14-
files: ["{packages|examples}/**/*.{js,ts,tsx,mts,cts}"],
14+
files: ["{packages|providers|examples}/**/*.{js,ts,tsx,mts,cts}"],
1515
languageOptions: {
1616
parser: tsParser,
1717
parserOptions: {
1818
ecmaVersion: "latest",
1919
sourceType: "module",
2020
ecmaFeatures: {
21-
jsx: true
22-
}
21+
jsx: true,
22+
},
2323
},
2424
globals: {
2525
...globals.browser,
@@ -28,10 +28,10 @@ export default [
2828
},
2929
plugins: {
3030
"@typescript-eslint": tsPlugin,
31-
"react": reactPlugin,
31+
react: reactPlugin,
3232
"react-hooks": reactHooksPlugin,
3333
"jsx-a11y": jsxA11yPlugin,
34-
"regexp": regexpPlugin,
34+
regexp: regexpPlugin,
3535
},
3636
rules: {
3737
...tsPlugin.configs["recommended"].rules,

0 commit comments

Comments
 (0)