NobodyWho

Product Introduction

1. Definition: NobodyWho is an open-source, cross-platform inference engine designed for running Large Language Models (LLMs) and Small Language Models (SLMs) fully on-device. It is a technical framework built on top of the high-performance llama.cpp library, providing type-safe, native bindings for multiple programming languages and game engines.
2. Core Value Proposition: It exists to enable private, offline AI inference without reliance on cloud APIs, subscription fees, or internet connectivity. Its primary value is delivering a production-ready, local LLM engine with automatic tool calling and multimodal capabilities directly to developers' native applications across mobile, desktop, and embedded platforms.

Main Features

1. Local & Offline Inference: The engine runs entirely on the user's device hardware (CPU/GPU). It loads models in the GGUF format and performs all computation locally, ensuring zero data privacy risk and no latency from network calls. It works by integrating the llama.cpp backend, which is optimized for efficient inference on consumer hardware.
2. Type-Safe Tool Calling with Automatic Grammar Generation: This feature allows developers to define functions in their native code (Kotlin, Swift, Python, etc.) and have the LLM call them with structured arguments. It works by automatically generating a formal grammar from the function's type signature, which constrains the LLM's output to valid JSON that matches the expected parameters. This eliminates the need for manual schema writing and parsing, reducing bugs.
3. Multimodal Input & Audio Synthesis/Recognition: Supports multimodal AI by allowing image and audio data to be provided as context to the LLM. It includes integrated Text-to-Speech (TTS) synthesis using backends like Kokoro and Supertonic to generate local WAV audio, and Speech-to-Text (STT) transcription using Whisper models. This enables fully local, voice-interactive AI applications.
4. Cross-Platform GPU Acceleration: Leverages Vulkan for GPU acceleration on Android, Linux, and Windows, and Metal for acceleration on macOS and iOS. This hardware acceleration significantly boosts inference speed compared to CPU-only execution, making on-device AI practical for real-time applications.
5. Conversation-Aware Preemptive Context Shifting: A memory management technique that retains full conversation history without arbitrary message length limits. It works by intelligently shifting and summarizing parts of the context window, allowing for long, coherent dialogues while maintaining performance, a common challenge in local LLM deployment.

Problems Solved

1. Pain Point: Data Privacy and Compliance Risk from sending sensitive information to third-party cloud AI APIs. NobodyWho solves this by keeping all data on-device.
2. Pain Point: Vendor Lock-in and Recurring Costs associated with per-token pricing models of services like OpenAI or Anthropic. It provides a one-time, free solution.
3. Pain Point: Development Complexity in implementing reliable, structured outputs (tool calling) from LLMs. Its automatic grammar generation abstracts away the complexity of JSON schema definition and validation.
4. Target Audience: Mobile App Developers (iOS/Swift, Android/Kotlin, React Native, Flutter) needing to embed private AI features; Game Developers using Godot Engine seeking AI-driven NPCs or narrative tools; Desktop Application Developers (Python, etc.) building offline-capable tools; IoT and Edge Computing Engineers requiring AI on resource-constrained, non-internet-connected devices.
5. Use Cases: Building a private AI chatbot within a notes app; creating an offline language tutor for travelers; implementing context-aware NPC dialogues in a Godot game; developing a local document analysis tool for legal or medical professionals; adding voice-controlled features to a smart home application without cloud dependency.

Unique Advantages

1. Differentiation: Unlike cloud-based API wrappers (OpenAI SDK) or single-language bindings, NobodyWho is a true multi-platform native engine. Compared to using llama.cpp directly, it provides high-level, type-safe APIs across six platforms, abstracting the complex C++ integration. Versus other local inference solutions, its automatic tool-calling grammar is a significant developer experience innovation.
2. Key Innovation: The type-driven, automatic grammar generation for tool calling is a core technical innovation. By deriving constraints directly from the host language's type system (e.g., Kotlin's data class, Swift's struct, Python's dataclass), it guarantees runtime safety and developer productivity unmatched by manual prompt engineering or schema-based approaches.

Frequently Asked Questions (FAQ)

1. What models are compatible with NobodyWho? NobodyWho is compatible with thousands of pre-trained models in the GGUF file format, which is the standard format for llama.cpp. You can download these models from platforms like Hugging Face, and the engine supports direct loading via hf:// URLs.
2. Can I use NobodyWho for commercial projects? Yes. Under its EUPL-1.2 license, you can use NobodyWho in proprietary and commercial projects free of charge. The copyleft condition applies only if you distribute a modified version of the NobodyWho engine itself, not the applications you build with it.
3. Does NobodyWho support GPU acceleration on Android and iOS? Yes. On Android, it uses Vulkan for GPU acceleration. On iOS and macOS, it uses Metal. This support is built into the respective Kotlin and Swift libraries, enabling faster inference on mobile devices.
4. How does the automatic tool calling work compared to OpenAI's function calling? While conceptually similar, NobodyWho's implementation is fully local and type-safe. Instead of sending a JSON schema in the prompt, it compiles your native function signatures into a deterministic grammar that guides the LLM's token generation at a fundamental level, ensuring the output is always syntactically and semantically valid for your code.
5. Is web browser (HTML5) export supported? Not currently. Desktop (Linux, macOS, Windows), Android, and iOS are the primary supported targets. WebAssembly (WASM) support for browser-based execution is technically challenging due to the engine's native dependencies and is a subject of future development.