llm-inference Tools
2 best llm-inference tools and apps, curated and ranked by community upvotes on ProductCool. Updated daily as new llm-inference products launch.
TensorFold is a high-performance LLM inference engine designed for developers and AI enthusiasts who want to run open models on their own Apple Silicon, NVIDIA, and AMD hardware. It solves the performance bottlenecks of local AI by delivering ultra-fast token generation, prompt reuse, and multi-machine scaling. Whether you are building creative workflows or managing enterprise workloads, TensorFold provides OpenAI- and Anthropic-compatible APIs to seamlessly power your local AI infrastructure.
Kimi-k3-in-c is a lightweight, portable C99 inference engine that allows consumer hardware to run massive Mixture of Experts AI models without needing GPUs, heavy frameworks, or complex clusters. By streaming sleeping experts directly from disk, it reduces a 1.56 TB model footprint down to just 8.24 GB of RAM on a single CPU. It is designed for developers, researchers, and AI enthusiasts who want to experiment with trillion-parameter language models locally on ordinary machines.