TurboFieldfare is an open-source inference engine that enables running the 26B-parameter Gemma 4 model on M-series Macs with only 2 GB of RAM by streaming expert weights from SSD and overlapping computation with I/O. It achieves 5–35 tokens/second depending on hardware and includes an OpenAI-compatible local server.
Background
Running large language models on consumer devices typically requires significant memory; this project demonstrates a novel approach to offloading parts of the model to storage while maintaining performance through smart caching and parallelism.
- Source
- Hacker News (RSS)
- Published
- Jul 29, 2026 at 11:05 PM
- Score
- 8.0 / 10