Soup

Product Introduction

1. Definition: Soup is an open-source, Apache-2.0 licensed CLI tool and automation framework for the fine-tuning stack of large language models (LLMs). It is a comprehensive solution for machine learning engineers and researchers.
2. Core Value Proposition: Soup exists to eliminate the complexity, manual configuration, and hardware limitations associated with fine-tuning LLMs. Its primary value is enabling efficient, vendor-agnostic model training by automating the entire workflow from data preparation to deployment-ready model validation.

Main Features

1. Automated Configuration & Training Stack: Soup automates the entire fine-tuning pipeline. It analyzes your data to recommend the optimal training method (SFT, DPO, ORPO, etc.), automatically generates a complete soup.yaml configuration file with derived hyperparameters (learning rate, epochs), handles data formatting, and sets up evaluation suites. It supports 23 training methods and 142 pre-built recipes.
2. Layer Streaming (BETA): This is a breakthrough feature for limited hardware. Instead of loading the entire frozen base model into GPU VRAM, Soup streams it layer-by-layer from CPU RAM or NVMe storage into a dedicated CUDA stream. Combined with 4-bit quantization (NF4), this allows fine-tuning of models like Llama-3.1-8B on a GPU with as little as 4 GB of VRAM. The feature also works for preference tuning methods like DPO, where the reference model is the same streamed base.
3. Intelligent Data Curation & Model Governance: Soup provides tools for robust data and model lifecycle management. soup data doctor performs pre-flight checks, soup advise recommends tasks, and semantic deduplication removes reworded duplicates. The soup ship command acts as a final gatekeeper, running a suite of evaluations to produce a definitive SHIP or DON'T-SHIP verdict with committable evidence, ensuring model quality and safety.
4. Seamless Ecosystem Integration & Migration: Soup is built for existing ML stacks. It offers first-class integrations with Hugging Face, Ollama, vLLM, DeepSpeed, FlashAttention, W&B, and more. The soup migrate command can automatically convert training configurations from popular tools like LLaMA-Factory, Axolotl, and Unsloth into Soup's optimized format in seconds, eliminating vendor lock-in.
5. Advanced Training & Optimization Techniques: Beyond automation, Soup incorporates state-of-the-art research techniques. This includes Spectrum for layer importance ranking, LISA for layer-wise sampling, online DPO/GRPO training, Process Reward Models (PRM), and soup shrink for model compression via depth-pruning and distillation.

Problems Solved

1. Pain Point: The extreme complexity and manual effort required to configure, optimize, and execute LLM fine-tuning runs, especially on consumer or limited hardware.
2. Target Audience: ML Engineers and Researchers in small teams or with limited budgets; individuals experimenting with LLM fine-tuning; organizations needing efficient, reproducible, and governable model training pipelines without cloud dependency.
3. Use Cases: Fine-tuning a 8B+ parameter model on a single consumer laptop GPU; quickly converting an existing project from another framework to Soup; ensuring a fine-tuned model hasn't lost general knowledge or developed harmful behaviors before deployment; managing a library of LoRA adapters with diff, merge, and bisect operations.

Unique Advantages

1. Differentiation: Unlike tools that stop at launching a training job (like LLaMA-Factory, Axolotl), Soup automates the entire decide-train-ship-operate-secure lifecycle. Unlike cloud-based platforms, it is free, open-source, and runs entirely on your own hardware with no vendor lock-in.
2. Key Innovation: The Layer Streaming technology is its most distinctive technical innovation. By bounding peak VRAM usage to a single model layer rather than the full model, it fundamentally changes the hardware requirements for fine-tuning large models, making previously impossible training runs feasible on low-end GPUs.

Frequently Asked Questions (FAQ)

1. What is Soup CLI and how does it work? Soup CLI is an open-source command-line tool that automates fine-tuning for large language models. It works by analyzing your dataset, automatically generating an optimal training configuration, and executing the training run using advanced techniques like layer streaming to work on limited hardware.
2. Can I fine-tune Llama 3.1 8B on a 4GB GPU? Yes, using Soup's layer streaming feature (currently in BETA). By streaming the frozen base model layers from RAM/disk and applying 4-bit quantization, Soup can fine-tune the Llama-3.1-8B model on a GPU with only 4 GB of VRAM, a task typically requiring much more powerful hardware.
3. How does Soup compare to LLaMA-Factory or Axolotl? While LLaMA-Factory and Axolotl are excellent training libraries, Soup provides a higher level of automation, a more comprehensive lifecycle toolkit (data doctor, eval design, ship gating), and unique features like layer streaming. Soup can also automatically migrate configurations from these tools.
4. Is Soup completely free to use? Yes, Soup is free and open-source software released under the permissive Apache-2.0 license. It requires no sign-up, has no usage fees, and is designed to work offline on your own infrastructure.
5. What does the soup ship command do? The soup ship command is a quality gate. It runs a comprehensive, auto-generated evaluation suite against your fine-tuned model to check for task performance, general knowledge retention, and safety. It then outputs a definitive SHIP or DON'T-SHIP verdict with supporting evidence, preventing the deployment of degraded or unsafe models.