Add dashboard-managed LLM provider system with OpenAI-compatible API support by devarshishimpi · Pull Request #13 · devarshishimpi/codra

devarshishimpi · 2026-05-26T14:17:35Z

Description

This PR implements a comprehensive dashboard-managed LLM provider system that allows users to configure and manage their own AI model providers directly through the Codra dashboard. Users can now securely store and use custom API keys for various LLM providers, including OpenAI, Anthropic, Google, and any OpenAI-compatible APIs.

Key Features:

Dashboard Provider Management: Users can add, edit, and manage LLM provider configurations (API keys, model selections) through the UI
Secure Configuration Storage: Provider configurations are encrypted using a new LLM_CONFIG_ENCRYPTION_KEY
OpenAI-Compatible API Support: Support for any OpenAI-compatible API endpoint
Provider Validation: Added validation for provider configurations to ensure correctness
Database Schema Updates: New updated global migration (001_initial.sql) to store provider configurations
Enhanced Model Catalog: Updated model catalog with support for multiple providers

Technical Changes:

New encryption utilities for storing sensitive provider configurations (llm-crypto.ts)
Expanded API endpoints for model management (/api/models)
Settings page UI redesign for provider management
Provider-agnostic model service that routes to configured providers
Support for Anthropic, OpenAI, Google, and Cloudflare models

Fixes #4

Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Unit Tests
Integration Tests
Manual Dashboard Verification
Manual GitHub Webhook Verification

Checklist:

I have starred Codra on GitHub
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing unit tests pass locally with my changes
I have signed the CLA

codra-app-personal

Codra Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b49a816918

ℹ️ About Codra in GitHub

Your team has set up Codra to review pull requests in this repo. Reviews are triggered when you:

Open a pull request for review
Mark a draft as ready
Comment "@codra-app review"

If Codra has suggestions, it will comment; otherwise it will react with 👍.

Codra can also answer questions or update the PR. Try commenting "@codra-app address that feedback".

codra-app-personal · 2026-06-02T06:16:01Z

 - A GitHub App (for webhooks/checks).
 - A GitHub OAuth App (for dashboard authentication).
- A Gemini API Key.
+- `LLM_CONFIG_ENCRYPTION_KEY` for encrypting dashboard-managed provider API keys.


Inconsistent list formatting

The newly added lines lack the bullet point prefix ('- ') used by the other items in the setup list, which breaks the visual consistency of the documentation.

Suggested change

- `LLM_CONFIG_ENCRYPTION_KEY` for encrypting dashboard-managed provider API keys.

- `LLM_CONFIG_ENCRYPTION_KEY` for encrypting dashboard-managed provider API keys.

- LLM providers and model credentials from the Settings dashboard.

codra-app-personal · 2026-06-02T06:16:01Z

+  tpm        INTEGER,
+  rpd        INTEGER,
+
+  model_id   TEXT PRIMARY KEY,


Missing seed data for model_configs

The migration removes the 'INSERT INTO model_configs' block (previously lines 162-173). For fresh database installations, this results in an empty 'model_configs' table. Since this table contains critical rate-limiting parameters (rpm, tpm, rpd) for LLM providers, the application may fail to function or behave unpredictably when attempting to look up provider configurations.

Suggested change

model_id TEXT PRIMARY KEY,

Restore the INSERT INTO model_configs statement with the required default model configurations before the DELETE and UPDATE operations.

codra-app-personal · 2026-06-02T06:16:01Z

+  created_at TIMESTAMPTZ DEFAULT now(),
+  updated_at TIMESTAMPTZ DEFAULT now(),
+
+  rpm        INTEGER,


Removal of NOT NULL constraints on rate limit columns

The columns 'rpm', 'tpm', and 'rpd' in the 'model_configs' table have been changed from 'INTEGER NOT NULL' to 'INTEGER'. Allowing NULLs for these fields can lead to runtime errors or the bypassing of rate limits in the application logic if the code expects these values to always be present.

Suggested change

rpm INTEGER,

rpm INTEGER NOT NULL,

tpm INTEGER NOT NULL,

rpd INTEGER NOT NULL,

codra-app-personal · 2026-06-02T06:16:01Z

+
+  await query(
+    `
+      UPDATE model_configs mc


Potential data corruption in provider migration

The update block at lines 304-313 assigns any remaining NULL provider_id in model_configs to the 'Cloudflare' provider. If the model_configs table contains providers not explicitly handled in the mapping (lines 292-297), those records will be incorrectly associated with Cloudflare. This causes silent data corruption for any existing providers not listed in the migration script.

Suggested change

UPDATE model_configs mc

// Remove the catch-all assignment or add a check to ensure mc.provider is actually 'cloudflare'

UPDATE model_configs mc

SET provider_id = provider_record.id

FROM llm_providers provider_record

WHERE mc.provider_id IS NULL

AND provider_record.name = 'Cloudflare'

AND mc.provider = 'cloudflare';

codra-app-personal · 2026-06-02T06:16:01Z

-      INSERT INTO model_configs (model_id, rpm, tpm, rpd, provider)
-      VALUES ($1, 10, 131072, 300, 'cloudflare')
-      ON CONFLICT (model_id) DO NOTHING
+      INSERT INTO model_configs (model_id, rpm, tpm, rpd, provider, provider_id, model_name, updated_at)


Silent failure of model configuration inserts

The INSERT INTO model_configs statements (lines 319, 337, 354, 367) use a SELECT from llm_providers to fetch the provider_id. If the corresponding provider record (e.g., 'Cloudflare' or 'Google') is missing from the llm_providers table, these queries will execute successfully but insert zero rows. This creates a fragile dependency where seed data failure results in missing configurations without throwing an error.

codra-app-personal · 2026-06-02T06:16:02Z

+
  await query('DELETE FROM model_configs WHERE model_id = $1', [kimiK25Model]);
+
+  await query('ALTER TABLE model_configs ALTER COLUMN provider_id SET NOT NULL');


Risk of constraint violation during NOT NULL migration

Line 382 attempts to SET NOT NULL on provider_id. If the previous migration steps (lines 285-313) failed to assign a provider_id to every single row in model_configs (which is likely if the 'catch-all' Cloudflare logic is removed or fails), this migration will crash. A safer approach is to identify and handle orphaned rows before applying the NOT NULL constraint.

codra-app-personal · 2026-06-02T06:16:02Z

-      fallbacks: ['@cf/zai-org/glm-4.7-flash'],
-    },
-  ],
+const EMPTY_GLOBAL_CONFIG: RepoConfig['model'] = {


Potential runtime crash due to null model value

The removal of SERVER_DEFAULT_GLOBAL_CONFIG in favor of EMPTY_GLOBAL_CONFIG changes the main model property from a guaranteed string (e.g., 'gemma-4-31b-it') to null. If downstream consumers of getGlobalConfig do not explicitly handle the null case, this will likely lead to runtime TypeErrors or API failures when the system attempts to use the model name in a request. Verify that all callers of getGlobalConfig implement appropriate null-checks or fallback logic.

Suggested change

const EMPTY_GLOBAL_CONFIG: RepoConfig['model'] = {

// If a system-wide fallback is required, maintain a safe default or ensure the type system enforces handling of null:

const EMPTY_GLOBAL_CONFIG: RepoConfig['model'] = {

main: null,

fallbacks: [],

size_overrides: [],

};

devarshishimpi added 5 commits May 25, 2026 22:27

feat: implement dashboard-managed LLM provider system

e996a54

refactor: label management and add provider validation

ff77850

add: consolidate LLM provider management and make rate limits optional

0293b36

fix: db migration assistant on existing dbs

5eb8c1a

add: consolidate LLM provider management and make rate limits optional

04372bc

This comment was marked as resolved.

Sign in to view

codra-app-personal Bot added the review: needs-attention label May 26, 2026

fix: cleanup migrations, improve API robustness, and enhance settings UX

426bda0

This comment was marked as resolved.

Sign in to view

add: improve overall settings ui

1b58039

This comment was marked as resolved.

Sign in to view

devarshishimpi added 2 commits June 2, 2026 11:25

refactor: move job step completion to database layer

825e2f1

fix: database migrations, API methods, and label removal logic

b49a816

devarshishimpi linked an issue Jun 2, 2026 that may be closed by this pull request

Support OpenAI/Anthropic compatible APIs for broader LLM choice #4

Open

codra-app-personal Bot reviewed Jun 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add dashboard-managed LLM provider system with OpenAI-compatible API support#13

Add dashboard-managed LLM provider system with OpenAI-compatible API support#13
devarshishimpi wants to merge 9 commits into
devfrom
feature/support-openai-compatible-apis

devarshishimpi commented May 26, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

codra-app-personal Bot left a comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

codra-app-personal Bot Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	- `LLM_CONFIG_ENCRYPTION_KEY` for encrypting dashboard-managed provider API keys.
	- `LLM_CONFIG_ENCRYPTION_KEY` for encrypting dashboard-managed provider API keys.
	- LLM providers and model credentials from the Settings dashboard.

	model_id TEXT PRIMARY KEY,
	Restore the INSERT INTO model_configs statement with the required default model configurations before the DELETE and UPDATE operations.

-  rpm        INTEGER,
+rpm INTEGER NOT NULL,
+  tpm INTEGER NOT NULL,
+  rpd INTEGER NOT NULL,

-      UPDATE model_configs mc
+// Remove the catch-all assignment or add a check to ensure mc.provider is actually 'cloudflare'
+UPDATE model_configs mc
+SET provider_id = provider_record.id
+FROM llm_providers provider_record
+WHERE mc.provider_id IS NULL
+  AND provider_record.name = 'Cloudflare'
+  AND mc.provider = 'cloudflare';


		await query('DELETE FROM model_configs WHERE model_id = $1', [kimiK25Model]);

		await query('ALTER TABLE model_configs ALTER COLUMN provider_id SET NOT NULL');

-const EMPTY_GLOBAL_CONFIG: RepoConfig['model'] = {
+// If a system-wide fallback is required, maintain a safe default or ensure the type system enforces handling of null:
+const EMPTY_GLOBAL_CONFIG: RepoConfig['model'] = {
+  main: null,
+  fallbacks: [],
+  size_overrides: [],
+};

Conversation

devarshishimpi commented May 26, 2026

Description

Key Features:

Technical Changes:

Type of change

How Has This Been Tested?

Checklist:

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

codra-app-personal Bot left a comment

Choose a reason for hiding this comment

Codra Review

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

codra-app-personal Bot Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant