Update yutori-computer-use templates for n1-latest

[dprevoznik]

Summary

• Update yutori templates (Python + TypeScript) to the n1-latest model API, which uses OpenAI-compatible tool_calls format instead of the previous custom format
• Remove Playwright computer mode in favor of a single Computer Controls path (Playwright is still used for goto_url in kiosk mode)
• Add kiosk mode option that hides browser chrome and uses Playwright for navigation
• Rename max_tokens to max_completion_tokens, update viewport to 1280x800, simplify the sampling loop and tool mapping

Test plan

All 13 action types were tested against live Kernel browser sessions for both Python and TypeScript implementations (27 tests each, all passing):

Action	Status
left_click, double_click, triple_click, right_click	Pass
hover, drag	Pass
type (basic, clear_before_typing, press_enter_after)	Pass
key_press (single key, combos, modifier mapping)	Pass
scroll (up, down, left, right)	Pass
goto_url, go_back, refresh, wait	Pass
Error handling (unknown action, missing required fields, invalid direction)	Pass

Made with Cursor

---

Note

Medium Risk
Changes core agent-loop and action execution semantics (tool-calls, coordinate/action schema, screenshot encoding) and adds new dependencies, which could cause runtime regressions in browser automation if the API expectations differ.

Overview
Updates the Python and TypeScript yutori-computer-use templates to target Yutori n1-latest, switching the sampling loop from the prior JSON-in-content protocol to OpenAI-compatible tool_calls/tool messages and renaming token budgeting to max_completion_tokens.

Simplifies execution to a single Computer Controls tool path by removing the Playwright “mode” implementation, while adding an optional kiosk/kioskMode that launches the browser in kiosk mode and routes goto_url through browsers.playwright.execute when enabled.

Aligns defaults and payloads with the new API: viewport defaults move to 1280x800, actions change to coordinates plus distinct click types (left_click/double_click/triple_click/right_click), screenshots are converted from PNG to WebP (new Pillow/sharp deps), and QA/docs/examples are updated accordingly.

^{Written by Cursor Bugbot for commit b31643b. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[Sayan-]

Great stuff!

A few items to address before merge:

---

1. Python loop.py — raise api_error should be bare raise

except Exception as api_error:
    print(f"API call failed: {api_error}")
    raise api_error  # ← resets the traceback to this line

Should be:

except Exception as api_error:
    print(f"API call failed: {api_error}")
    raise  # preserves original traceback for debugging

---

2. TypeScript loop.ts — as unknown as string cast hides API extension

content: [
  {
    type: 'image_url',
    image_url: {
      url: `data:image/webp;base64,${result.base64Image}`,
    },
  },
] as unknown as string,

This double-cast is a lie to the type system. Consider replacing with // @ts-expect-error Yutori n1 accepts image content arrays in tool messages — it documents the API extension and will surface a diagnostic if the OpenAI SDK types ever add native support for this.

---

3. README action tables are incomplete (both Python and TypeScript)

Both READMEs document 8 actions but the code handles 13. Missing from the tables: key_press, hover, drag, wait, refresh.

---

4. Python loop.py — manual assistant message serialization is brittle (nit)

The manual dict construction:

assistant_dict: dict[str, Any] = {
    "role": "assistant",
    "content": assistant_message.content or "",
}
if assistant_message.tool_calls:
    assistant_dict["tool_calls"] = [...]

Works today but could silently break if the API adds fields. Consider using assistant_message.model_dump(exclude_none=True) to future-proof this (similar to how the TypeScript version pushes the SDK object directly).

[Sayan-]

Great stuff!