NightRun is a runtime that boots directly from a USB stick or SD card and runs a local LLM chat without Linux or Windows in between. Turn on the PC, pick the boot media, and the chat screen comes up in place of an operating system’s startup sequence. It is a single application running on firmware (UEFI), written in no_std Rust, with everything from screen drawing to the tokenizer and the quantized inference kernels implemented in-house. The idea is to turn a PC or a Raspberry Pi 5 into a machine dedicated entirely to running a local LLM. It is developed by an individual who goes by hardrave, and the code is published under the MIT license.
Key Features
- A UEFI runtime that bypasses the OS: There is no bootloader and no kernel ─ the firmware launches a single UEFI application directly. With no OS or driver layer in between, the setup stays remarkably simple
- The model lives in RAM, and storage is sealed off: At startup the entire model (roughly 1.3 to 2.4 GB) is loaded into RAM, verified with CRC-32, and then storage access is sealed. No reads or writes happen after that
- Offline by construction, with no network stack: Networking is simply not implemented, so once it boots, it is fully offline. There is no path for data to be sent anywhere
- Hand-written quantized inference kernels: Inference runs on the CPU using matrix kernels written by hand for AVX2 (x86_64) and NEON (Arm). The Q8_0, Q4_K, and Q6_K quantization formats are supported
- A custom framebuffer UI: The graphics stack is also built from scratch, drawing the chat view, the input editor, and the scrollback
- Support for x86_64 PCs and the Raspberry Pi 5: x86_64 UEFI machines boot from a USB stick, and the Raspberry Pi 5 boots from an SD card. The Pi 5 build includes fan control
Pricing
| Plan | Price | What you get |
|---|---|---|
| Open source (MIT) | Free | All features. You can build, use, and modify the images |
Pricing is current as of September 2026. NightRun is self-hosted software published under the MIT license, and there are no paid plans. Check the official site for the latest information. Note that model weights are not bundled with NightRun and remain subject to their own licenses (the Llama Community License; Apache-2.0 for the Qwen and Granite builds).
Pros & Cons
✅ Pros
- There is no OS boot to wait through ─ you reach the chat within tens of seconds of powering on. The step of “starting up the PC in order to use AI” largely disappears
- Because networking is absent by construction, you can explain at the level of the design itself that nothing leaves the machine. That makes it easier to discuss in environments where taking data out is restricted
- A single USB stick, or one Raspberry Pi 5, is enough to build a dedicated AI device. Nothing is installed on the host PC, and pulling out the USB stick returns it to its original state
- The Rust code is written from scratch and published openly, so it also works as material for learning how inference engines and the boot process actually work
⚠️ Cons
- The models you can run are limited to a small set of families ─ quantized builds of Llama 3.2 (1B and 3B), Qwen3 4B Instruct, and Granite 4.1 3B ─ so you cannot freely bring your own GGUF files
- There is no GPU path, only CPU inference, which puts a ceiling on response speed. It is not suited to running large models quickly
- Booting requires Secure Boot to be disabled (the distributed images are unsigned). On managed machines you will need to check this against your organization’s rules
- The published performance figures come from benchmark logs produced by the developer, and no independent third-party verification could be confirmed
- The firmware portion for the Raspberry Pi 5 is explicitly documented as community-maintained
Comparison with Similar Services
| Service | Runs on | How it differs from NightRun |
|---|---|---|
| NightRun | UEFI (no OS) | Boots straight from removable media and has no network stack |
| Ollama | Linux / macOS / Windows | Runs as a background service on the OS, with free choice of models |
| llama.cpp | Linux / macOS / Windows | An inference engine running on the OS; NightRun uses it as a reference for accuracy checks |
| LM Studio | Desktop OS | An easy-to-use GUI client that needs no dedicated boot media |
If you just want to use a local LLM day to day, Ollama or LM Studio will take fewer steps. NightRun earns its place where “no OS in between” and “no network at all” are valuable in themselves.
Who Is It For
- People who want a standalone local AI device for handling sensitive documents offline
- Hardware enthusiasts who want to build a dedicated AI appliance out of a Raspberry Pi 5
- Developers who want to learn systems programming and Rust without an OS layer, from code that actually runs
- Anyone who wants to try a local LLM without installing anything on a work PC
Summary
“Running it locally” covers a wide range, and NightRun pushes all the way to one end of it. It removes the OS, removes networking, seals off storage access, and still makes a working chat out of what is left. In practical speed and model flexibility, tools that run on an OS often win, and this is not going to be anyone’s first choice for everyday use. As an answer to the question of what is genuinely required to run an AI model, though, it is refreshingly clear. A single USB stick is all you need to try it, so if you are curious, booting it on a machine you already have is a good place to start.