Add buffered inference support for SALM models#15364

Merged

pzelasko merged 22 commits intomainfrom

dkaramyan/streaming-canary-qwen

Feb 9, 2026

Commits on Feb 3, 2026

draft version of streaming salm pipeline
naymaraq
committed
fix ids_to_text
naymaraq
committed
rm clone
naymaraq
committed
bring back .clone
naymaraq
committed

Commits on Feb 4, 2026

first working version
naymaraq
committed
change extra token count to 2
naymaraq
committed
rm prints
naymaraq
committed

Commits on Feb 5, 2026

fix device bug
naymaraq
committed
pre tokenize prompts
naymaraq
committed
choose rightmost lcs in the buffer, add test for lcs
naymaraq
committed
fix typos
naymaraq
committed
fix typo
naymaraq
committed
fix docstrings
naymaraq
committed

Commits on Feb 6, 2026

Commits on Feb 8, 2026

improvements
naymaraq
committed
Apply isort and black reformatting
naymaraq
committed
bugfix
naymaraq
committed