Harbor

Product Introduction

1. Definition: Harbor is an open-source, command-line interface (CLI) and companion desktop application designed to orchestrate and manage a comprehensive, local AI development stack. It functions as a Docker Compose automation and configuration management layer specifically for Large Language Model (LLM) inference, frontends, and supporting AI tools.
2. Core Value Proposition: Harbor exists to eliminate the significant time and complexity involved in manually configuring, networking, and maintaining a local AI environment. Its primary value is providing a one-command local LLM stack setup, where all components—from backends like Ollama and vLLM to frontends like Open WebUI and supporting services like SearXNG for web search—are pre-wired to work together out of the box.

Main Features

1. Unified Service Orchestration: Harbor uses a declarative compose.yml file and a custom CLI to spin up Docker containers for dozens of AI services. It automatically handles cross-service networking, environment variable configuration, and volume management. The harbor up command is the primary entry point, launching specified services and their dependencies.
2. Pre-Configured Service Integrations: This is Harbor's technical cornerstone. It doesn't just run services in isolation; it configures them to communicate. For example, launching harbor up searxng automatically configures Open WebUI's Web RAG endpoint to point to the local SearXNG instance. Similarly, launching backends like llama.cpp or vLLM automatically registers them as available providers in connected frontends.
3. Configuration and Profile Management: Harbor provides a centralized system for managing runtime configurations. Users can save, switch between, and import configuration profiles (e.g., harbor profile save llama4). These profiles can store engine-specific arguments, model paths, and other settings, allowing quick context switching between different AI workflows or hardware optimizations.
4. Host Tool Integration (Launch Command): The harbor launch command bridges Harbor-managed backends with host-installed AI coding tools (e.g., Codex, Continue, OpenCode). It detects or starts a compatible Harbor backend (like Ollama), wires the specified model into the host tool's configuration, and can inject additional capabilities like Harbor Boost's web search tools, enabling local model use without manual provider setup.
5. Ecosystem and Utility Tools: Harbor bundles numerous utilities: harbor hf for interacting with Hugging Face; harbor bench for running benchmarks; harbor eject to generate a standalone Docker Compose file; harbor tunnel for secure internet exposure via Traefik; and harbor qr for generating phone-accessible links to local services.

Problems Solved

1. Pain Point: Configuration Hell and Fragmentation. Manually setting up a local LLM stack involves installing multiple independent tools (Ollama, Open WebUI, ComfyUI), configuring each one's settings, setting up networking between them (e.g., making Ollama's API visible to Open WebUI), and managing conflicting dependencies. This process is time-consuming, error-prone, and not reproducible.
2. Target Audience: AI Developers, Researchers, and Hobbyists. This includes developers prototyping LLM applications, researchers needing isolated testing environments, and tech enthusiasts exploring local AI models. It also benefits ML engineers who need consistent local staging environments and content creators who use local image/voice generation tools.
3. Use Cases: Rapid Prototyping of AI Applications: Spin up a complete backend (vLLM), frontend (Open WebUI), and RAG tool (SearXNG) in one command to test a new model. Unified Development Environment: Use harbor launch to ensure all your local coding agents (Aider, Codex) use the same configured model backend. Educational Exploration: Easily experiment with different inference engines (TGI, llama.cpp, Aphrodite) and UIs (LibreChat, Lobe Chat) without complex setup procedures.

Unique Advantages

1. Differentiation: Unlike standalone tools (Ollama App) or general DevOps platforms, Harbor is opinionated and pre-integrated. While Docker Compose alone can run containers, Harbor provides the crucial "glue" configuration that makes services interoperable. Compared to all-in-one platforms, Harbor is modular, open-source, and focused on local development rather than cloud deployment.
2. Key Innovation: Harbor's declarative service dependency and integration mapping is its core innovation. The system understands not just how to run a service, but how it connects to other services in the Harbor ecosystem. This is managed through internal configuration templates and sidecar containers that automate the setup of API endpoints, authentication keys (e.g., for Unsloth Studio), and plugin enabling, which would otherwise require manual, post-launch configuration.

Frequently Asked Questions (FAQ)

1. What is Harbor AI and how does it work? Harbor is an automation CLI that manages a local Docker-based AI toolstack. It works by using a central configuration to pull, run, and pre-configure Docker containers for services like LLM backends, chat UIs, and search tools, ensuring they are network-connected and ready to use upon launch.
2. Is Harbor a replacement for Ollama or Docker? No, Harbor is a management layer on top of them. It uses Ollama, Docker, and Docker Compose as underlying technologies. Harbor simplifies their orchestration and integration but does not replace their core functionality.
3. How do I use my own GGUF models with Harbor? You can use the integrated harbor hf download command to fetch models, or place your .gguf files in the designated shared model volume. For backends like llama.cpp, use harbor llamacpp model /path/to/model.gguf to specify the model, and Harbor will configure the service accordingly.
4. Can Harbor expose my local AI services to the internet safely? Harbor includes a tunnel command powered by Traefik for exposure. However, the documentation strongly warns about security risks. Exposing services is not recommended without implementing additional authentication, which is the user's responsibility to configure for each service.
5. What is the difference between Harbor CLI and Harbor App? The Harbor CLI is the core command-line tool for controlling services. The Harbor App is a complementary desktop GUI that provides a visual interface for starting/stopping services, viewing logs, and adjusting configurations, ultimately executing the same CLI commands under the hood.