feat: add LiteLLM as AI gateway provider#559
Open
RheagalFire wants to merge 1 commit into
Open
Conversation
Author
|
cc @Huanshere |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Motivation
VideoLingo currently supports OpenAI-compatible APIs via configurable
base_url. Users wanting to use Anthropic, Google, or other providers must set up a third-party proxy like OpenRouter (issue #434 workaround). LiteLLM removes this friction: setprovider: litellmin config, use a prefixed model string likeanthropic/claude-sonnet-4-6orgemini/gemini-2.0-flash, and LiteLLM routes the request to the right provider automatically. No proxy server required.Changes
config.yaml- addedapi.provider: 'openai'config key (values:openai,litellm)core/utils/ask_gpt.py- extracted_call_openai()and_call_litellm()helpers;ask_gpt()routes based on provider configcore/st_utils/sidebar_setting.py- added provider dropdown in Streamlit sidebar with context-sensitive help textrequirements.txt- addedlitellm>=1.80.0,<1.87.0tests/test_litellm_provider.py- 11 unit tests covering dispatch,drop_params, credential forwarding, base_url handling, JSON format, and provider routingTests
1. Unit tests (11/11 passing):
2. Live E2E against Anthropic Claude Sonnet 4.6 (via Azure AI Foundry):
3. Live E2E - JSON mode (VideoLingo's primary usage pattern):
This proves the full chain:
ask_gpt()->_call_litellm()->litellm.completion()-> Anthropic via Azure Foundry -> parsed response.4. Pin resolution verified:
litellm>=1.80.0,<1.87.0resolves cleanly alongsideopenai>=1.55.3,<2.Risk / Compatibility
api.providerremainsopenai.drop_params=Truesilently drops provider-unsupported kwargs, preventing cross-provider failures (e.g.,response_formaton providers that don't support JSON mode).ask_gpt()work without changes since the function signature is unchanged.Example usage
1. Config (config.yaml):
**2. Python: **