qlora Tools

Soup is a CLI tool that automates the entire fine-tuning stack for large language models. It solves the complexity of configuring training runs by automatically generating optimal settings, handling data preparation, and enabling training of large models on limited hardware through layer streaming. It's for ML engineers and researchers who want to fine-tune models efficiently without vendor lock-in or manual hyperparameter tuning.

AirLLM is a library that enables efficient inference for large language models like the 70B parameter class using just a single 4GB GPU. It solves the problem of high hardware requirements and costs typically associated with running state-of-the-art LLMs. This makes advanced AI capabilities accessible to developers, researchers, and small teams with limited computational resources.