fix(t5): use prefix-redecode for streaming so byte-level BPE tokenizers work

[alvinttang]

Summary

Fixes #1021. t5/t5.py Tokenizer.decode called self._tokenizer.convert_ids_to_tokens(t) per token and only stripped SentencePiece ▁. The __main__ streaming loop fed it one token id at a time. For tokenizers whose ids don't map 1:1 to characters this is broken:

• Byte-level BPE (RobertaTokenizerFast used by Salesforce/codet5p-220m) emits raw GPT-2 byte markers Ġ (space) and Ċ (newline) which the ▁ replacement never touches.
• SentencePiece byte-fallback emits hex byte tokens that also need decoding.

Fix

Replace per-token convert_ids_to_tokens with a small streaming-state API that delegates to the HuggingFace tokenizer's own decode(prefix, skip_special_tokens=True) and yields the new substring each step — exactly the "redecode the entire prefix" path @awni suggested in the issue thread.

def new_stream_state(self) -> dict: ...
def stream_decode(self, state, token_id) -> str: ...
def decode(self, t) -> str:  # batch decode now also via HF
    return self._tokenizer.decode(t, skip_special_tokens=True)

__main__ updated to call stream_decode(state, token.item()) instead of decode([token.item()], with_sep=...). with_sep flag removed (only ever set in this one call site — checked via grep across the repo). +22/-4 production.

Test

t5/test_tokenizer_stream.py — parametrized: t5-small (SentencePiece) + Salesforce/codet5p-220m (byte-level BPE). Asserts streamed output equals full-prefix decode(ids, skip_special_tokens=True). Regression: asserts no Ġ / Ċ ever appear in streamed output for codet5p code generation.

TDD RED first (AttributeError: 'Tokenizer' object has no attribute 'new_stream_state'), then GREEN (3/3 pass). End-to-end repro encoded the exact buggy snippet from the issue ('\n print("Hello World")\n\n'), streamed via the new API, output matches expected with no markers leaked.

black + isort --profile=black clean.

Risk notes

• Tokenizer.decode(list_of_ids) no longer accepts with_sep kwarg. Only call site rewritten.
• Quadratic cost in output length per stream — acceptable for typical T5 lengths and explicitly accepted by the maintainer in the issue thread; alternative would be a proper incremental detokenizer (much larger change).

Refs #1021