Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
224 changes: 224 additions & 0 deletions docs/troubleshooting/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ This guide covers common issues and debugging techniques for the Copilot SDK acr

- [Enable Debug Logging](#enable-debug-logging)
- [Common Issues](#common-issues)
- [Silent Model Fallback / Wrong Model Being Used](#silent-model-fallback--wrong-model-being-used)
- [MCP Server Debugging](#mcp-server-debugging)
- [Connection Issues](#connection-issues)
- [Tool Execution Issues](#tool-execution-issues)
Expand Down Expand Up @@ -312,6 +313,229 @@ var client = new CopilotClient(new CopilotClientOptions
});
```

### Silent Model Fallback / Wrong Model Being Used

> See [GitHub Issue #250](https://github.com/github/copilot-sdk/issues/250)

**Symptoms:** You specify a model in your session configuration, but the session uses a
different model (typically `claude-sonnet-4.6`) without raising an error. This happens when the
requested model ID is invalid, misspelled, or not available for your account.

**Cause:** The SDK does not validate model IDs client-side — the name is forwarded directly
to the Copilot CLI server. When the server determines that the requested model is unavailable,
it silently falls back to the first available default model rather than returning an error.
The fallback order is `claude-sonnet-4.6` → `claude-sonnet-4.5` → `claude-haiku-4.5` → etc.
Warnings are logged server-side but are not surfaced to the SDK as errors.
Comment on lines +326 to +328
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Listing a specific fallback order is likely to become outdated and can mislead readers if the server behavior changes. Consider rephrasing to a more stable statement (e.g., “falls back to a default available model (often …); exact selection may vary by account/availability”) and avoid implying a guaranteed ordering.

Suggested change
it silently falls back to the first available default model rather than returning an error.
The fallback order is `claude-sonnet-4.6``claude-sonnet-4.5``claude-haiku-4.5` → etc.
Warnings are logged server-side but are not surfaced to the SDK as errors.
it silently falls back to a default available model rather than returning an error. This is often
a `claude-sonnet` or `claude-haiku` variant, but the exact model selected may vary by account,
configuration, and model availability. Warnings are logged server-side but are not surfaced to
the SDK as errors.

Copilot uses AI. Check for mistakes.

> **Note:** There is currently no SDK-level option to disable automatic model fallback. This
> is tracked in [#250](https://github.com/github/copilot-sdk/issues/250).

**Solution:**

1. **Validate the model before creating a session** by calling `listModels()` to check
available models:

<details open>
<summary><strong>Node.js / TypeScript</strong></summary>

```typescript
const models = await client.listModels();
const desiredModel = "gpt-5.4";

const isAvailable = models.some((m) => m.id === desiredModel);
if (!isAvailable) {
throw new Error(
`Model "${desiredModel}" is not available. Available: ${models.map((m) => m.id).join(", ")}`,
);
}

const session = await client.createSession({
model: desiredModel,
// ...
});
```
</details>

<details>
<summary><strong>Python</strong></summary>

```python
models = await client.list_models()
desired_model = "gpt-5.4"

available_ids = [m.id for m in models]
if desired_model not in available_ids:
raise ValueError(
f'Model "{desired_model}" is not available. '
f'Available: {", ".join(available_ids)}'
)

session = await client.create_session({"model": desired_model})
```
</details>

<details>
<summary><strong>Go</strong></summary>

<!-- docs-validate: hidden -->
```go
package main

import (
"context"
"fmt"
"strings"

copilot "github.com/github/copilot-sdk/go"
)

func main() {
client := copilot.NewClient(nil)
ctx := context.Background()
_ = client.Start(ctx)

models, _ := client.ListModels(ctx)
desiredModel := "gpt-5.4"

found := false
var ids []string
for _, m := range models {
ids = append(ids, m.ID)
if m.ID == desiredModel {
found = true
}
}
if !found {
panic(fmt.Sprintf("Model %q is not available. Available: %s",
desiredModel, strings.Join(ids, ", ")))
}
}
```
<!-- /docs-validate: hidden -->

```go
models, err := client.ListModels(ctx)
desiredModel := "gpt-5.4"

found := false
var ids []string
for _, m := range models {
ids = append(ids, m.ID)
if m.ID == desiredModel {
found = true
}
}
if !found {
panic(fmt.Sprintf("Model %q is not available. Available: %s",
desiredModel, strings.Join(ids, ", ")))
Comment on lines +417 to +430
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Go example is presented as a snippet, but it relies on fmt/strings without showing imports, and the indentation is not gofmt-style (spaces instead of tabs inside the for and if). Consider either showing a complete, gofmt’d snippet (with imports) or adding a brief note about required imports and formatting to keep the example copy/paste-friendly.

Suggested change
models, err := client.ListModels(ctx)
desiredModel := "gpt-5.4"
found := false
var ids []string
for _, m := range models {
ids = append(ids, m.ID)
if m.ID == desiredModel {
found = true
}
}
if !found {
panic(fmt.Sprintf("Model %q is not available. Available: %s",
desiredModel, strings.Join(ids, ", ")))
// Requires imports "fmt" and "strings".
models, err := client.ListModels(ctx)
desiredModel := "gpt-5.4"
found := false
var ids []string
for _, m := range models {
ids = append(ids, m.ID)
if m.ID == desiredModel {
found = true
}
}
if !found {
panic(fmt.Sprintf("Model %q is not available. Available: %s",
desiredModel, strings.Join(ids, ", ")))

Copilot uses AI. Check for mistakes.
}
```
</details>

<details>
<summary><strong>.NET</strong></summary>

<!-- docs-validate: skip -->

```csharp
var models = await client.ListModelsAsync();
var desiredModel = "gpt-5.4";

if (!models.Any(m => m.Id == desiredModel))
{
var available = string.Join(", ", models.Select(m => m.Id));
throw new InvalidOperationException(
$"Model \"{desiredModel}\" is not available. Available: {available}");
}

var session = await client.CreateSessionAsync(new SessionConfig
{
Model = desiredModel,
});
```
</details>

2. **Detect fallback at runtime** by comparing the model reported in `assistant.usage` events
against the model you requested:

<details open>
<summary><strong>Node.js / TypeScript</strong></summary>
Comment on lines +458 to +462
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says “All code examples cover the four SDKs (Node.js, Python, Go, .NET)”, but the “Detect fallback at runtime” workaround only includes Node.js and Python. Add Go and .NET examples in the same <details> pattern, or update the PR description/section text to reflect the actual coverage.

Copilot uses AI. Check for mistakes.

```typescript
const requestedModel = "gpt-5.4";
const session = await client.createSession({ model: requestedModel });

session.on("assistant.usage", (event) => {
if (event.data.model !== requestedModel) {
console.warn(
`Model fallback detected: requested "${requestedModel}", ` +
`got "${event.data.model}"`,
);
}
});
```
</details>

<details>
<summary><strong>Python</strong></summary>

```python
requested_model = "gpt-5.4"
session = await client.create_session({"model": requested_model})

@session.on("assistant.usage")
def on_usage(event):
if event.data.get("model") != requested_model:
print(
f'Model fallback detected: requested "{requested_model}", '
f'got "{event.data["model"]}"'
)
```
</details>

3. **Query the active model** after session creation to confirm what model is in use:
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prose says to “use rpc.model.getCurrent()”, but the Go and .NET sections use different-looking APIs (session.Model.GetCurrent / session.GetModelAsync). To avoid confusion, either (a) adjust the text to “use the SDK-specific equivalent of the model get-current RPC” or (b) align the examples/method naming consistently with the same conceptual API across languages.

Suggested change
3. **Query the active model** after session creation to confirm what model is in use:
3. **Query the active model** after session creation to confirm what model is in use. For each language, use the SDK-specific equivalent of the model get-current RPC:

Copilot uses AI. Check for mistakes.

<details open>
<summary><strong>Node.js / TypeScript</strong></summary>

```typescript
const result = await session.rpc.model.getCurrent();
console.log("Active model:", result.modelId);
```
</details>

<details>
<summary><strong>Python</strong></summary>

```python
result = await session.rpc.model.get_current()
print("Active model:", result.get("modelId"))
```
</details>

<details>
<summary><strong>Go</strong></summary>

```go
result, err := session.Model.GetCurrent(ctx)
fmt.Println("Active model:", result.ModelID)
```
</details>

<details>
<summary><strong>.NET</strong></summary>

<!-- docs-validate: skip -->

```csharp
var result = await session.GetModelAsync();
Console.WriteLine($"Active model: {result.ModelId}");
```
</details>

4. **Enable debug logging** (see [above](#enable-debug-logging)) to see server-side
fallback warnings in the CLI logs.

---

## MCP Server Debugging
Expand Down
Loading