Grep tool fails with "invalid ripgrep output" when source files contain non-UTF-8 (e.g. GBK) characters

[SuperStrain]

Description

Bug description

The built-in Grep tool always fails with the error "invalid ripgrep output" when any matched file contains non-UTF-8 encoded characters (e.g. GBK/GB2312 Chinese comments or text files).

Steps to reproduce

1. Have a project that contains source files with GBK-encoded content
2. Use the Grep tool to search for any pattern that matches those files
3. Grep tool returns: invalid ripgrep output

Root cause

When ripgrep encounters bytes that cannot be decoded as valid UTF-8, it outputs the matched line as lines.bytes (base64-encoded) instead of lines.text in its JSON format.

Example of the JSON ripgrep actually emits:

{"type":"match","data":{"path":{"text":"./server/sctrl/main.cpp"},
"lines":{"bytes":"CS8vRU5BQkxFX0VYU0RL..."}, ← bytes, not text!
"line_number":1780, ...}}

opencode's ripgrep JSON parser only handles lines.text and throws "invalid ripgrep output" when it encounters lines.bytes, even though the rest of the results (all UTF-8 files) are perfectly valid.

This means a single non-UTF-8 file in the project completely breaks the Grep tool for ALL searches, regardless of whether the specific search matches that file.

Expected behavior

The Grep tool should handle lines.bytes gracefully — either by base64-decoding it, replacing it with a placeholder like [binary/non-UTF-8 line], or skipping only that specific match while returning all valid UTF-8 matches.

Workaround

Use the Bash tool with rg --text instead:
rg --text "PATTERN" .

The --text flag forces ripgrep to treat all files as text, avoiding the bytes field entirely.

Environment

• opencode version: latest
• OS: Ubuntu Linux (x86_64)
• ripgrep version: 15.1.0
• Affected file types: GBK-encoded .c / .cpp / .txt files

Plugins

No response

OpenCode version

No response

Steps to reproduce

No response

Screenshot and/or share link

No response

Operating System

ubuntu18

Terminal

No response

[kagura-agent]

I'd like to work on this. The root cause is in src/file/ripgrep.ts — the Match zod schema only accepts lines.text but ripgrep emits lines.bytes (base64) for non-UTF-8 content, causing the parse() function to throw. Same issue applies to path.bytes and submatches[].match.bytes.

Plan: update the schemas to accept either text or bytes, and base64-decode the bytes variant with a lossy UTF-8 fallback.

[SuperStrain]

Could this be related to this issue?
#27832

[kagura-agent]

Good catch @SuperStrain — #27832 is definitely related! Both stem from the same root cause: OpenCode's built-in tools assume UTF-8 encoding. #27832 proposes broader multi-encoding support across Read/Grep/Edit/Write, while this issue specifically targets the ripgrep output parser crash when lines.bytes (base64) is returned instead of lines.text.

This issue's fix (handling the bytes variant in the Zod schema) would be a targeted first step that also benefits the broader #27832 effort. The parser should gracefully handle non-UTF-8 content even before full multi-encoding support lands.