KFlash: New Benchmarks for Apple Silicon Optimized Engines

Summary: Kapitsa is an open-source project that provides a fast inference engine optimized for Apple Silicon. This engine is specifically developed for Qwen and Nemotron models.

New benchmark results have been released for inference engines optimized for Apple Silicon processors. Engines like KFlash (Kapitsa) and Splash (DFlash2) were tested on large language models such as Qwen and Nemotron. This demonstrates the significant improvements in LLM inference performance on local devices.

The benchmarks were conducted on Apple Silicon hardware like the Macbook Pro M4 Pro and M1 Pro. KFlash reportedly achieved speeds of 65 tokens/s, while Splash performed around 50 tokens/s. These engines were specifically tested on quantized models like Qwen 3.8 27b/4b/35b 4Q. The optimizations aim to leverage the hardware’s specific architecture to achieve higher token/s rates.

For infrastructure engineers and developers, this increases the potential for performing local LLM inference on Apple Silicon-based workstations or servers (if available). Higher token/s rates can accelerate development and testing processes, and enable more efficient operation of AI models in edge computing scenarios. This offers opportunities to reduce cloud dependency and lower latency.

Running LLMs on local devices also offers advantages in terms of data privacy and security. Inference can be performed without the need to send sensitive data to the cloud. It is important for engineers to explore ways to integrate such optimized engines into their workflows and evaluate their performance.

🔗 Source: r/LocalLLaMA
💬 Community Discussion: Inspect
📡 Authoritative feeds & live tracking: Radar