vMLX - Home of JANG_Q - No other MLX inferencer can do this. Cont Batch, Prefix, Paged, KV Cache Quant, VL - Powers MLX Studio. Image gen/edit, OpenAI/Anth

Algorithm-System Co-design: accurate and efficient 2-bit KV cache quantization for LLM Inference..

Production-ready 2/4-bit KV Cache quantization for vLLM via Triton; 70% VRAM saving & 1.8x speedup