An on-device AI platform built by Mirai Labs. It provides an inference engine, a model conversion toolkit, and a quantization scheme so that open models such as Llama, Gemma, Qwen, and Mistral can run directly on macOS and iOS devices instead of being sent to the cloud. The core inference engine uzu (written in Rust) and the model optimization framework lalamo (Python/JAX) are released as open source under the MIT license, and quick-start guides for Swift, TypeScript, Python, and Rust let you begin integrating in minutes. The company’s position is that “on-device is not a smaller cloud, it’s a different system entirely,” so the stack is co-designed from the model down to the hardware. A hybrid setup that falls back to cloud inference when needed is also available.
Key Features
- An inference engine specialized for Apple Silicon (uzu): A high-performance inference engine written in Rust. It optimizes tensor operations for memory-constrained devices in a hardware-aware way, aiming to remove perceptible latency. The repository is published under the MIT license and had over 1,600 stars as of August 2026
- A model conversion and quantization toolchain (lalamo): A JAX-based model optimization framework. It converts open models for on-device deployment and compresses them with a quantization scheme designed to preserve interaction quality. It is also MIT licensed
- Support for major open models: Widely used open models such as Llama, Gemma, Qwen, and Mistral can be run locally. The full list of supported models is published on the Local models page of the official site
- Hybrid local and cloud inference: Work is handled locally by default, and processing that exceeds what the device can do can be routed to cloud inference. You can balance privacy requirements against processing power per use case
- A complete developer on-ramp: Quick-start guides for Swift, TypeScript, Python, and Rust, a CLI for interacting with models and serving a local API, and a chat application for macOS. Development and evaluation can stay on the same stack
- Android support announced: The official site lists Android inference as “coming soon”
Pricing
| Plan | Price | What’s included |
|---|---|---|
| Open source (uzu / lalamo) | Free | The inference engine and the model optimization framework. MIT licensed, commercial use permitted |
| macOS app / CLI | Free | Local chat application, plus a CLI for model management and serving a local API |
| Cloud inference / developer platform | Contact for details | Cloud inference as a fallback from local execution. No public price list could be confirmed |
Pricing is as of August 2026. No publicly available pricing page could be found on the official site, so it is best to confirm the terms for cloud usage and commercial support directly via the official site.
Pros & Cons
✅ Pros
- Inference stays on the device, so the data you enter is not sent anywhere. Well suited to apps handling medical, legal, or internal documents that are hard to put in the cloud
- With no network round trip, responses start returning quickly, and the app works offline
- The core libraries are published under the MIT license, so you can read the implementation. Vendor lock-in concerns are small
- The inference engine, model conversion, quantization, CLI, and app all come as one set, so you do not have to assemble the pieces yourself
- Beyond Swift, you can enter from TypeScript, Python, and Rust, which fits a wider range of existing project setups
⚠️ Cons
- Supported platforms center on Apple Silicon (macOS/iOS). Android is only announced, and Windows and general Linux are out of scope
- The size of models you can run locally is bound by device memory. It is not suited to running the latest large-scale models as-is
- The cloud pricing structure is not published, which makes it hard to estimate the cost of a hybrid setup in advance
- It is developer infrastructure rather than a finished consumer product, so adoption requires a certain amount of implementation knowledge
- It is a relatively new project, so APIs and supported models may change
Comparison with Similar Services
| Criteria | Mirai | Ollama | LM Studio | MLX (Apple) |
|---|---|---|---|---|
| Main use | On-device inference stack for embedding in apps | Running and managing models locally | Running local models through a GUI | Machine learning framework for Apple Silicon |
| Target users | iOS/macOS app developers | Developers and technical users | Individuals including non-engineers | Researchers and framework users |
| Platforms | macOS/iOS (Android announced) | macOS/Linux/Windows | macOS/Windows/Linux | macOS |
| Cloud fallback | Hybrid setup supported | Local only in principle | Local only in principle | Not applicable |
| License | uzu / lalamo are MIT | MIT | Free (proprietary) | MIT |
Who Is It For
- Developers who want to add AI features to iOS/macOS apps where user data cannot leave the device
- People building apps for offline environments, or where response speed is a hard requirement
- Engineers who want to read the internals of the inference engine and tune quantization and optimization themselves
- Product teams who want to avoid usage-based cloud API billing by shifting inference cost onto the device
- Anyone who wants to verify, in real code, what local inference on Apple Silicon can actually do
Summary
Mirai is an on-device AI stack designed on the premise that local inference is a different system, not a scaled-down cloud. Because the uzu inference engine and the lalamo optimization framework are published under the MIT license, the fastest way in is to try them on your own Mac from the GitHub repositories and the quick-start guides in the official documentation. The focus on Apple Silicon is a constraint, but for iOS/macOS apps with privacy and latency requirements it is a strong option. If you are considering a setup that also uses cloud inference, confirm the pricing terms with the vendor in advance.