Skip to content

feat(provider): add LocalAI and Ollama providers#470

Open
devin-ai-integration[bot] wants to merge 2 commits into
feature/2.0.0-text2jsonfrom
devin/1778709313-providers-localai-ollama
Open

feat(provider): add LocalAI and Ollama providers#470
devin-ai-integration[bot] wants to merge 2 commits into
feature/2.0.0-text2jsonfrom
devin/1778709313-providers-localai-ollama

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 13, 2026

Description

Adds two new built-in providers for self-hosted / local AI servers:

  • SmartHopper.Providers.LocalAI — talks to a LocalAI instance via its standard OpenAI-compatible POST /v1/chat/completions endpoint.
  • SmartHopper.Providers.Ollama — talks to an Ollama instance via its OpenAI-compatible endpoint (POST /v1/chat/completions).

Both providers are near drop-in mirrors of the existing DeepSeek / OpenAI providers, adapted for local servers. Because LocalAI and Ollama both speak OpenAI's chat-completions wire format, almost all of the per-request encoding, decoding, tool-call handling, and SSE streaming logic is shared with the other OpenAI-compatible providers.

What each provider supports

  • OpenAI-compatible messages array (system / user / assistant / tool roles, including multi-turn tool-call results)
  • Tools / function calling (tools / tool_choice, parallel tool calls, tool-call results)
  • JSON Schema structured outputs via response_format (json_schema) with a wrapping *JsonSchemaAdapter for non-object root schemas, mirroring the OpenAI adapter pattern
  • Server-Sent Events streaming with the nested *StreamingAdapter partial class
  • Optional Authorization: Bearer <token> header — useful when the local server is fronted by a reverse-proxy that requires auth, but not required for the default loopback setups

Settings (per-provider — Base URL is the key new addition)

Setting Type LocalAI default Ollama default Required
Base URL URL http://localhost:8080/v1 http://localhost:11434/v1
API Key string (empty) (empty)
Model string (empty — user-managed) (empty — user-managed)
Enable Streaming bool true true
Max Tokens int 4096 4096
Temperature double [0.0, 2.0] 0.7 0.7
seed (Ollama only) int (unset)

The Base URL is validated as an absolute http(s) URI; Max Tokens must be positive; Temperature is clamped to [0.0, 2.0]. Invalid values surface a styled error dialog when persisting settings, matching the pattern used by DeepSeekProviderSettings.

Model registries

Both *ProviderModels.cs return an empty list. LocalAI and Ollama users install their own models locally (e.g. ollama pull llama3.1, or any GGUF / Transformers model in a LocalAI model directory), so the provider does not attempt to enumerate them. The UI lets the user type whatever model name they have configured on the server (e.g. llama3.1, qwen2.5:14b, mistral:7b-instruct, phi3, …).

Wiring

  • SmartHopper.sln — new project entries + 24 build-configuration rows
  • SmartHopper.Infrastructure.csproj — two new InternalsVisibleTo entries (SmartHopper.Providers.LocalAI, SmartHopper.Providers.Ollama) with PublicKey=$(SmartHopperPublicKey), matching the existing built-in providers
  • SmartHopper.Components.Test.csproj — two new ProjectReference entries so test components can resolve the providers
  • yak-package/manifest.yml — adds localai / ollama keywords
  • DEV.md — promotes both providers from "Planned" to "🟠 Testing" in the capabilities table and adds their *ProviderModels.cs paths to the model-registry source-file list
  • README.md — adds the two providers to the supported-providers list
  • CHANGELOG.md — entries under [Unreleased] → Added

Notes on tooling

  • The provider model auto-update workflow (.github/workflows/chore-update-provider-models.yml) is intentionally not extended to these providers: it pulls metadata from OpenRouter as the single source of truth, and OpenRouter does not expose local LocalAI / Ollama models.
  • Icon assets (localai_icon.png, ollama_icon.png) are minimal 16×16 placeholders to match the existing convention; they can be replaced with branded artwork in a follow-up.

Breaking Changes

None — both providers are purely additive.

Testing Done

  • Followed the existing OpenAI / DeepSeek provider patterns exactly (same AIProvider<T> singleton + AIProviderSettings + IAIProviderFactory + *ProviderModels + *JsonSchemaAdapter + nested *StreamingAdapter partial class).
  • Compiled both projects locally on Linux against net7.0 (SmartHopper.Providers.LocalAI, SmartHopper.Providers.Ollama) — both build with 0 errors after wiring up the Infrastructure InternalsVisibleTo entries. The remaining build warnings are pre-existing analyzer warnings from SmartHopper.Infrastructure and are not introduced by this change.
  • End-to-end runtime verification against live LocalAI / Ollama servers is left for follow-up since it requires those servers to be installed on the developer machine.

Checklist

  • This PR is focused on a single feature or bug fix
  • Version in Solution.props was updated, if necessary, and follows semantic versioning (not needed — additive providers, no public-API or behavior change)
  • CHANGELOG.md has been updated
  • PR title follows Conventional Commits format
  • PR description follows Pull Request Description Template

Link to Devin session: https://app.devin.ai/sessions/9ce31e9f6c7140d5adfcdc9c557b4a3f
Requested by: @marc-romu

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

devin-ai-integration Bot and others added 2 commits May 17, 2026 12:38
- New SmartHopper.Providers.LocalAI for self-hosted LocalAI servers
  (OpenAI-compatible /v1/chat/completions). Defaults to
  http://localhost:8080/v1; users configure model and optional bearer token.
- New SmartHopper.Providers.Ollama for local Ollama instances via the
  OpenAI-compatible endpoint. Defaults to http://localhost:11434/v1;
  exposes an extra 'seed' parameter for reproducible sampling.
- Both providers support tools/function calling, SSE streaming, and JSON
  Schema structured outputs.
- Wire new projects into SmartHopper.sln, Infrastructure InternalsVisibleTo
  list, Components.Test project references, manifest keywords, DEV.md
  provider/model tables, README provider list, and CHANGELOG.
@marc-romu marc-romu force-pushed the devin/1778709313-providers-localai-ollama branch from 74b8ed7 to c14f0ae Compare May 17, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant