Autocomplete does not register responses from server

[mmengler]

Before submitting your bug report

• I've tried using the "Ask AI" feature on the Continue docs site to see if the docs have an answer
• I'm not able to find a related conversation on GitHub discussions that reports the same bug
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:Ubuntu 24.04
- Continue version:1.2.22
- IDE version:VSCode 1.118.1
- Model:qwen3-coder-30b-a3b-instruct
- config:
  
  name: Local Config
  version: 0.0.1
  schema: v1
  models:
    - name: code autocomplete
      provider: openai
      model: qwen3-coder-30b-a3b-instruct
      apiBase: http://localhost:3000/v1
      useResponsesApi: false
      roles:
        - autocomplete
        - edit
      autocompleteOptions:
        disable: false

Description

I am trying to get continue running with a LLM that is hosted on a server that does not use a standard API. For this I wrote a proxy in python to translate the openai API to the one my server uses. For the chat I got that working without problems, but autocomplete does not register a response as seen in the following screenshot.

For the proxy server I am using the following python code (with a separate function for the chat). I have also tried using a streamed response with the same result.

from fastapi import FastAPI
from fastapi.responses import JSONResponse

app = FastAPI()

@app.post("/v1/completions")
async def completions(req):
        # communication with LLM server generates variable "text"
        ...

        response = {
        "id": "cmpl-" + str(int(time.time())),
        "object": "text_completion",
        "created": int(time.time()),
        "model": req.model,
        "choices": [
            {
                "index": 0,
                "message":{
                    "role": "assistant",
                    "content": text
                },
                "finish_reason": "stop"
            }
        ],
        #"usage": {
        #    "completion_tokens": 16,
        #    "prompt_tokens": 10,
        #    "total_tokens": 26
        #}
    }
    
    return JSONResponse(
        content=response
    )

The answers from the server typically have the format:

''' python
some completion
'''

I have also tried hardcoding the response as e.g.:

\n    pass

To reproduce

No response

Log output

[danyalahmed1995]

Hello , I might be missing something, but from the response you shared it looks like your /v1/completions endpoint is returning a chat style response.

The response has:

"message": {
  "role": "assistant",
  "content": "..."
}

For autocomplete, I think Continue may be expecting the older completions format instead, where the generated text is in choices[0].text.

Could you try changing the proxy response for /v1/completions to something closer to this?

{
  "id": "cmpl-123",
  "object": "text_completion",
  "created": 1234567890,
  "model": "qwen3-coder-30b-a3b-instruct",
  "choices": [
    {
      "text": "\n    pass",
      "index": 0,
      "finish_reason": "stop"
    }
  ]
}

If that works, then this may be a proxy response-shape issue rather than an autocomplete bug in Continue itself.

[mmengler]

At this point I have tried several different formats (to be sure I replaced the response in the proxy with the one you sent exactly), but got the same response.

[danyalahmed1995]

I See, one more thing I would check is the actual text being returned inside choices[0].text.

In the original issue, you mentioned the server typically returns something like a code block:

some completion

For autocomplete, I would try returning only the raw completion text, without Markdown fences, language labels, or extra formatting.

For example, instead of returning this:

```python
    pass

try returning only:

```text
    pass

It may also be worth logging the exact JSON response your proxy sends back to Continue after translation, not just the upstream model response, to confirm that /v1/completions is returning something like:

{
  "id": "cmpl-123",
  "object": "text_completion",
  "created": 1234567890,
  "model": "qwen3-coder-30b-a3b-instruct",
  "choices": [
    {
      "text": "    pass",
      "index": 0,
      "finish_reason": "stop"
    }
  ]
}

If the proxy is already returning raw choices[0].text, then the next thing I’d check is whether Continue is expecting a streaming completions response for autocomplete or whether the request/response is being rejected somewhere in the autocomplete provider path.

[danyalahmed1995]

If it still doesn't work , I might take a look into it during weekend.

[mmengler]

Thanks for the suggestions. It is working now with a streaming response.
I could have sworn I'd tried it that way before, but the format was probably wrong.

    def event_stream():
        payload = {
            "id": "1",
            "object": "text_completion",
            "choices": [
                {
                    "text": text,
                    "finish_reason": "stop"
                }
            ]
        }

        resp = f"data: {json.dumps(payload)}\n\n"
        yield resp
        yield "data: [DONE]\n\n"

    return StreamingResponse(event_stream(), media_type="text/event-stream")

[danyalahmed1995]

@mmengler glad I could helped 🙂