feat(api): Implement connector support via static configuration

[jaideepr97]

What does this PR do?

Implements support for configuring static connectors in the stack via run.yaml. Major features of this PR:

• Adds a new configurable object called connector
• connectors can be configured under connectors in the run.yaml
• config allows stack admins to specify connector_id, server URL and label
• Adds connectors as a new internal API implementation
• Implements API surface outlined in feat(api): add readonly connectors API #4258
• updates meta-reference agent provider to have access to the connectors API implementation and handles plumbing
• updates responses API MCP tool input to accept either server_url or connector_id and handles resolution

Examples:

run.yaml config:

version: 2
image_name: llamastack-crimson
container_image: null
apis: ...
providers: ...
storage: ...
models: ...
connectors:
  - connector_id: kubernetes
    url: "http://localhost:8080/mcp"
    connector_type: mcp
...

API requests:

curl -s http://localhost:8321/v1alpha/connectors | jq 

Response:
{
  "data": [
    {
      "connector_type": "mcp",
      "connector_id": "kubernetes",
      "url": "http://localhost:8080/mcp",
      "server_label": null,
      "server_name": null,
      "server_description": null,
      "server_version": null
    }
  ]
}

curl -s http://localhost:8321/v1alpha/connectors/kubernetes | jq 

Response:                                                   
{
  "connector_type": "mcp",
  "connector_id": "kubernetes",
  "url": "http://localhost:8080/mcp",
  "server_label": null,
  "server_name": "kubernetes-mcp-server",
  "server_description": null,
  "server_version": "v0.0.55"
}

curl -s http://localhost:8321/v1alpha/connectors/kubernetes/tools/resources_list | jq 

Response:
{
  "toolgroup_id": null,
  "name": "resources_list",
  "description": "List Kubernetes resources and objects in the current cluster by providing their apiVersion and kind and optionally the namespace and label selector\n(common apiVersion and kind include: v1 Pod, v1 Service, v1 Node, apps/v1 Deployment, networking.k8s.io/v1 Ingress)",
  "input_schema": {
    "properties": {
      "apiVersion": {
        "description": "apiVersion of the resources (examples of valid apiVersion are: v1, apps/v1, networking.k8s.io/v1)",
        "type": "string"
      },
      "kind": {
        "description": "kind of the resources (examples of valid kind are: Pod, Service, Deployment, Ingress)",
        "type": "string"
      },
      "labelSelector": {
        "description": "Optional Kubernetes label selector (e.g. 'app=myapp,env=prod' or 'app in (myapp,yourapp)'), use this option when you want to filter the pods by label",
        "pattern": "([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]",
        "type": "string"
      },
      "namespace": {
        "description": "Optional Namespace to retrieve the namespaced resources from (ignored in case of cluster scoped resources). If not provided, will list resources from all namespaces",
        "type": "string"
      }
    },
    "required": [
      "apiVersion",
      "kind"
    ],
    "type": "object"
  },
  "output_schema": null,
  "metadata": {
    "endpoint": "http://localhost:8080/mcp"
  }
}

Client side usage example:

import os
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8321/v1", api_key=os.getenv("OPENAI_API_KEY"))
respB = client.responses.create(
        model=args.model,
        tools=[
            {
                "type": "mcp",
                "server_label": MCP_LABEL,
                "connector_id": "kubernetes",
                "require_approval": "never",
            }
        ],
        input=[{
            "role": "user",
            "content": (
                "List what kubernetes MCP tools you are allowed to use in this context.Tell me something about the cluster. \
                Try to call only the MCP tools that you have access to, and tell me which tools you called. If none are available, explain why."
            )
        }],
    )
    pretty_print_result("B: no restriction at the MCP tool (server) level, tool choice is mcp with server label and tool name", respB)

output:

=== B: no restriction at the MCP tool (server) level, tool choice is mcp with server label and tool name ===
Output text:
 I don't have access to user's context (e.g., cluster details). However, the available MCP tools for Kubernetes are:  
1. **configuration_view** - Shows Kubernetes configuration.  
2. **events_list** - Lists Kubernetes events.  
3. **namespaces_list** - Lists Kubernetes namespaces.  
4. **nodes_log** - Retrieves node logs.  
5. **nodes_stats_summary** - Gets node summary stats.  
6. **nodes_top** - Shows CPU and memory usage.  
7. **pods_list** - Lists all pods.  
8. **pods_list_in_namespace** - Lists pods in a namespace.  
9. **pods_delete** - Deletes pods.  
10. **pods_run** - Runs pods.  
11. **pods_get** - Retrieves pods.  
12. **resources_list** - Lists cluster resources.  

These tools help manage cluster configurations, track events, monitor nodes, and interact with pods. Let me know if you'd like to use specific ones!

Closes #4186 and #4061 (partially)

Test Plan

pending

NOTE: this PR builds on top of #4258 therefore it also contains changes from it. This PR should only be reviewed after #4258

[cdoern]

one comment so far, the implementation looks good especially compared to prompts which follows a similar structure.

[cdoern]

include this in the from llama_stack_api import. or If we missed these please add them to llama_stack_api's __init__.py thanks!

[jaideepr97]

updated, thanks!

[github-actions]

✱ Stainless preview builds

This PR will update the llama-stack-client SDKs with the following commit message.

feat(api): Implement connector support via static configuration

Edit this comment to update it. It will appear in the SDK's changelogs.

✅ llama-stack-client-node studio · code · diff

Your SDK built successfully.
generate ⚠️ → build ✅ → lint ✅ → test ✅

npm install https://pkg.stainless.com/s/llama-stack-client-node/6de68db8887ad05a02df05d2f86e51fe77ce90a1/dist.tar.gz

✅ llama-stack-client-kotlin studio · code · diff

Your SDK built successfully.
generate ⚠️ → lint ✅ → test ❗

✅ llama-stack-client-python studio · code · diff

Your SDK built successfully.
generate ⚠️ → build ✅ → lint ✅ → test ✅

pip install https://pkg.stainless.com/s/llama-stack-client-python/e30c1e431887bd5d892282810e37d8349a49bc55/llama_stack_client-0.4.0a14-py3-none-any.whl

✅ llama-stack-client-go studio · code · diff

Your SDK built successfully.
generate ❗ → lint ❗ → test ❗

go get github.com/stainless-sdks/llama-stack-client-go@4d517d6907f9eeb14dafe5acec7e900b5e964585

---

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
Last updated: 2025-12-24 14:21:56 UTC

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @jaideepr97 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[jaideepr97]

added unit tests for this feature. Looked into adding an integration test as well, but ran into a chicken and egg problem with the connector needing to be configured in the stack at boot time, but the mcp servers will need to be spun up at runtime with a port not known beforehand

This feature is currently incomplete with only read operations supported for connectors. IMO, we should wait until add/delete endpoints are available (#4239 ) for connectors so that the feature is truly complete and it is easier to write integration tests when connectors can be registered/unregistered on the fly

[jaideepr97]

cc @mattf @cdoern would be great if yall could also PTAL

[ashwinb]

lgtm

[mattf]

question about the future use of connectors: should each responses provider need to handle connectors or should the inference router convert connector_id to connector_url before dispatching to providers?

[mattf]

this is unconditional overwriting. idempotent would be to accept if the value is the same and reject otherwise.

[jaideepr97]

updated

[mattf]

to validate idempotency, you also need a negative test, one where the id is the same but the values are different and the service rejects the registration.

[jaideepr97]

added

[mattf]

why expose this detail?

[jaideepr97]

this field was added to be able to differentiate between stale config connectors and connectors that were registered via api
updated to exclude from schema

[jaideepr97]

question about the future use of connectors: should each responses provider need to handle connectors or should the inference router convert connector_id to connector_url before dispatching to providers?

I'm not sure at the moment since we only have 1 agents/responses provider (do we plan to have more?)
that might require setting up a separate resolver layer that is outside of the agent provider scope but would also need to be MCP tool-aware if it is going to manipulate tools, if we are okay with that

[cdoern]

thanks for all of the back and forth on this one!

[cdoern]

@mattf PTAL, this is the last PR linked to an issue in the 0.4.0 milestone

[mattf]

this shouldn't be exposed

[jaideepr97]

updated, should be fixed now

[mattf]

"idempotent - updates if already exists"

please self-review this code.

consider -

• why update if the values are the same?
• how will a user know their register attempt failed?

[jaideepr97]

here's the logic I ended up with after another look:

• config is the source of truth at boot time. config connectors can also never be updated, only registered. Therefore, config based connectors are allowed overwrites at registration time, so stack admins can take deliberate action to "update" any previously registered connector by updating the config and restarting the server. If the values are the same then it is a no-op
• if a new connector has a different source (api) and has conflicts with an existing connector, then the registration fails, since that is technically an update operation and should happen via the update endpoint (once it is implemented in Add /admin level Create/Update/Delete endpoints for the connectors API #4239 ) and not at registration time

as for

how will a user know their register attempt failed?

the user will always be a stack admin since only they can register new connectors (via config or /admin api), so I left it as a log statement that they can check instead of raising an error

[mattf]

there's a debate about how to handle resources in config vs api. to be consistent, you should not include the source as an input param and resources defined in config can be unregistered + registered via api.

an admin for the service may not be the same as the service operator.

[jaideepr97]

there's a debate about how to handle resources in config vs api. to be consistent, you should not include the source as an input param and resources defined in config can be unregistered + registered via api.

an admin for the service may not be the same as the service operator.

@ashwinb felt connectors should not be considered to be resources since they have no provider and are not routable and suggested to look at them as simple objects instead

Not sure how that informs the debate or connectors' subjectivity to it

[jaideepr97]

@mattf providing some more context/thoughts/questions here based on further looking at what exists -

to be consistent, you should not include the source as an input param

I found RegistryEntrySource being used by models at present. So source tracking exists, but it seems to be baked into ResourceWithOwner and currently only utilized by models. It also doesn't seem to distinguish between config and api (as you referenced) and treats them both as "api"

Connectors are not resources therefore they do not inherit from ResourceWithOwner and currently handle source tracking directly in their own definition. If we want to stay consistent with the precedent of source tracking but not keep it in the connector definition I can extract it out to some parent class and inherit it from there but that seems rather forced and unnecessary given that this field is already not exposed

alternatively, if connectors should stick with the pattern of not differentiating between config and api as sources, I see the problem of stale objects occurring (which is a gap within the existing implementation IMO). Resources/objects previously configured via config but removed later are not automatically deleted from the KV store. This can only be handled in 2 ways I think:

• identify if a seemingly stale object was registered via config or api - if api, leave it alone. if config, and it has been removed since, then that means it should be cleaned up at stack boot time.
• leave it up to the api to manually query what objects have been registered and clean up stale ones

option 1 assumes that the config is meant to be an accurate reflection of the state of things at any given time as opposed to just a point of initialization. Maybe that is a key point that will decide what approach should be followed
How do we view the role of the config in this project?

If I have misunderstood, and you only meant to remove source from the function signature for register_connector then I would say that the source field is not going to be exposed to either config or API users, it will be set internally based on where it is coming from

resources defined in config can be unregistered + registered via api.

if we extend this to all objects, this would still be possible for connectors once API operations (other than read) are implemented in #4239 , as long as it is done in 2 steps - unregister.+ register. The only API operation I have blocked here is re-registering an existing connector via API in a single action.

an admin for the service may not be the same as the service operator.

That's fair - users registering connectors via API would not see the log statement. I think that should be handled in the follow up PR when that API is the focus. This PR focuses on the config based workflow, for which the logging is enough since only stack admins can set the config and see the logs. How does that sound?

All of this is also encapsulated by the open question about whether non-resources are expected to follow the same standards and patterns as resources