feat(provider): add LocalAI and Ollama providers#470
Open
devin-ai-integration[bot] wants to merge 2 commits into
Open
feat(provider): add LocalAI and Ollama providers#470devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
- New SmartHopper.Providers.LocalAI for self-hosted LocalAI servers (OpenAI-compatible /v1/chat/completions). Defaults to http://localhost:8080/v1; users configure model and optional bearer token. - New SmartHopper.Providers.Ollama for local Ollama instances via the OpenAI-compatible endpoint. Defaults to http://localhost:11434/v1; exposes an extra 'seed' parameter for reproducible sampling. - Both providers support tools/function calling, SSE streaming, and JSON Schema structured outputs. - Wire new projects into SmartHopper.sln, Infrastructure InternalsVisibleTo list, Components.Test project references, manifest keywords, DEV.md provider/model tables, README provider list, and CHANGELOG.
74b8ed7 to
c14f0ae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds two new built-in providers for self-hosted / local AI servers:
SmartHopper.Providers.LocalAI— talks to a LocalAI instance via its standard OpenAI-compatiblePOST /v1/chat/completionsendpoint.SmartHopper.Providers.Ollama— talks to an Ollama instance via its OpenAI-compatible endpoint (POST /v1/chat/completions).Both providers are near drop-in mirrors of the existing
DeepSeek/OpenAIproviders, adapted for local servers. Because LocalAI and Ollama both speak OpenAI's chat-completions wire format, almost all of the per-request encoding, decoding, tool-call handling, and SSE streaming logic is shared with the other OpenAI-compatible providers.What each provider supports
messagesarray (system/user/assistant/toolroles, including multi-turn tool-call results)tools/tool_choice, parallel tool calls, tool-call results)response_format(json_schema) with a wrapping*JsonSchemaAdapterfor non-object root schemas, mirroring the OpenAI adapter pattern*StreamingAdapterpartial classAuthorization: Bearer <token>header — useful when the local server is fronted by a reverse-proxy that requires auth, but not required for the default loopback setupsSettings (per-provider — Base URL is the key new addition)
http://localhost:8080/v1http://localhost:11434/v1truetrue40964096[0.0, 2.0]0.70.7seed(Ollama only)The Base URL is validated as an absolute
http(s)URI; Max Tokens must be positive; Temperature is clamped to[0.0, 2.0]. Invalid values surface a styled error dialog when persisting settings, matching the pattern used byDeepSeekProviderSettings.Model registries
Both
*ProviderModels.csreturn an empty list. LocalAI and Ollama users install their own models locally (e.g.ollama pull llama3.1, or any GGUF / Transformers model in a LocalAI model directory), so the provider does not attempt to enumerate them. The UI lets the user type whatever model name they have configured on the server (e.g.llama3.1,qwen2.5:14b,mistral:7b-instruct,phi3, …).Wiring
SmartHopper.sln— new project entries + 24 build-configuration rowsSmartHopper.Infrastructure.csproj— two newInternalsVisibleToentries (SmartHopper.Providers.LocalAI,SmartHopper.Providers.Ollama) withPublicKey=$(SmartHopperPublicKey), matching the existing built-in providersSmartHopper.Components.Test.csproj— two newProjectReferenceentries so test components can resolve the providersyak-package/manifest.yml— addslocalai/ollamakeywordsDEV.md— promotes both providers from "Planned" to "🟠 Testing" in the capabilities table and adds their*ProviderModels.cspaths to the model-registry source-file listREADME.md— adds the two providers to the supported-providers listCHANGELOG.md— entries under[Unreleased] → AddedNotes on tooling
.github/workflows/chore-update-provider-models.yml) is intentionally not extended to these providers: it pulls metadata from OpenRouter as the single source of truth, and OpenRouter does not expose local LocalAI / Ollama models.localai_icon.png,ollama_icon.png) are minimal 16×16 placeholders to match the existing convention; they can be replaced with branded artwork in a follow-up.Breaking Changes
None — both providers are purely additive.
Testing Done
AIProvider<T>singleton +AIProviderSettings+IAIProviderFactory+*ProviderModels+*JsonSchemaAdapter+ nested*StreamingAdapterpartial class).net7.0(SmartHopper.Providers.LocalAI,SmartHopper.Providers.Ollama) — both build with 0 errors after wiring up the InfrastructureInternalsVisibleToentries. The remaining build warnings are pre-existing analyzer warnings fromSmartHopper.Infrastructureand are not introduced by this change.Checklist
Link to Devin session: https://app.devin.ai/sessions/9ce31e9f6c7140d5adfcdc9c557b4a3f
Requested by: @marc-romu