Google just dropped TurboQuant, and I'm about to get so much more out of my Mac Mini now lol. It makes LLMs 6x smaller and 8x faster with zero quality loss. Now I can run insane AI models locally for free. → Bigger context windows → Way faster processing → Completely secure Show more
@GoogleResearch
Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: goo.gle/4bsq2qI