“No Linux, no kernel. Flash NightRun to an SD card, boot a Raspberry Pi 5, and you’re chatting locally with Llama 3.2 1B, Qwen3 4B, or Granite 4.1 3B on the CPU. It is a single Rust UEFI application with handwritten NEON kernels. Models use standard GGUF files converted on the host, decode at roughly llama.cpp speed, and output is verified token-for-token against it.”
Of course even a 16Gb SDRAM Raspberry Pi lacks the serious computing power for fast LLM-use (Raspberry Pi cluster computing could fix that), might try this just the same. It’s developed with Claude in Rust, author admits the speed gain in comparison to running LLM’s on a regular Linux install is minimal. Not to keen on AI – still a Raspberry Pi fan and i should keep up with developments.
https://github.com/hardrave/NIGHTRUN
