Offline AI

[LeanBitLab]

Model Name

Visheratin T5 Tiny

Device Specs (RAM/CPU)

4GB Ram sd720g Cpu

Feedback / Issue

Welcome to Offline AI discussion!

[ALMIGHTY-VC]

I tried the Xenova grammar small quantized based on T5, Pszemraj's grammar small quantized and yeah both don't work as intended.So they are out of consideration for now I think.

[LeanBitLab]

Only Visheratin T5 Tiny worked for me too

[ALMIGHTY-VC]

In this month my exam AR going on so,consider me mostly offline.Still,here are some of my interesting finds:- 1)Prithivida gramformer(quantized if available I'm unable to find it in this haste) 2)Falcon H1 Tiny 90M instruct 3)HRM Grammar Light V2(2026 edition) Also another question have you tried the qwen offline models? How about implementing a single model.onnx it will be better for the memory and processes.

…

On Sat, 31 Jan, 2026, 11:38 pm LeanBitLab, ***@***.***> wrote: Only Visheratin T5 Tiny worked for me too — Reply to this email directly, view it on GitHub <#16 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2YYNFFD6G4QCBEGKJNPX334JTVRZAVCNFSM6AAAAACTQYXALCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRVHE4DEMY> . You are receiving this because you commented.Message ID: ***@***.***>

[LeanBitLab]

I find only Qwen gguf models, when I tried the gguf engine with the Qwen model, only models of 600mb or above were barely usable since the gguf engine wasn't that customisable, I didn't find any ONNX models for Qwen

[vishaldebnath19-commits]

Instead of ONNX
Llama 1b . gguf
Works faster then other.
In offline ai
Please removed ONNX features instead please add gguf
There are so many options in gguf.
All downloadable

[LeanBitLab]

Instead of ONNX
Llama 1b . gguf
Works faster then other.
In offline ai
Please removed ONNX features instead please add gguf
There are so many options in gguf.
All downloadable

Actually, GGUf models were the first engine I used. I tried multiple GGUf models too. The issue is that the GGUf engine isn't tweakable enough for proofreading. There weren't any lightweight, proofreading-optimized GGUf models that were better than ONNX models.

[vishaldebnath19-commits]

First of All Hats off and Many Many Many Thank to You & LeanBitLab & Developer's for Creating such a Handsome Customizable keyboard.

So
As you tried using Qwen gguf .I Have a Good Smartphone ,And I have tried many gguf in my phone ,As per my experience Qwen gguf is the most slowest gguf. There are tons of gguf like EmbeddingGemma 300M or llama many more. Ggguf 1b models may work solw in many smartphone but 300m or 110m Gguf will work faster.

Instead you can add
task / litertlmfile. Which used in Google offline ai Edge Gallery
Options like TONE change / proofreading is important at offline ai keyboard.For this Custom AI Keys
Is Important please add this feature .

And can you please mention me on which versions you have used gguf feature so that i can try or download

[LeanBitLab]

First of All Hats off and Many Many Many Thank to You & LeanBitLab & Developer's for Creating such a Handsome Customizable keyboard.

So
As you tried using Qwen gguf .I Have a Good Smartphone ,And I have tried many gguf in my phone ,As per my experience Qwen gguf is the most slowest gguf. There are tons of gguf like EmbeddingGemma 300M or llama many more. Ggguf 1b models may work solw in many smartphone but 300m or 110m Gguf will work faster.

Instead you can add
task / litertlmfile. Which used in Google offline ai Edge Gallery
Options like TONE change / proofreading is important at offline ai keyboard.For this Custom AI Keys
Is Important please add this feature .

And can you please mention me on which versions you have used gguf feature so that i can try or download

I tried other models like Smollm, Gemma, and Tinylamma. I even tried to train a specified proofreading model, since the GGUf engine wasn't tweakable enough for proofreading or any specific task. The custom prompt wasn't enough, even with 1.5b qwen given mediocre proofreading results. If there exists a good, lightweight text correction specific model (less than 500 MB), I would gladly reuse the GGUf engine. I don't want the keyboard to consume too much battery or processing power.