Compute:Arena

Product Introduction

1. Definition: Compute:Arena is a community-submitted benchmarking platform and command-line interface (CLI) for local AI model inference on any hardware, any runtime, and any quantisation. It aggregates signed throughput reports measured on community hardware, focusing on decode and prefill tokens per second for local models on edge silicon, Apple Silicon, discrete GPUs, and multi-GPU systems. Compute:Arena functions as a local LLM performance database, leaderboard, and comparison engine, with results tied to specific model identifiers, runtime, backend, quantisation, and chip.
2. Core Value Proposition: Compute:Arena exists to replace unverifiable local AI performance claims with reproducible, hardware-specific benchmarks. It helps users answer which model, quantisation, runtime, backend, and chip combination delivers the best decode and prefill throughput. Primary keywords include local AI benchmarks, LLM inference benchmarks, tokens per second, quantisation benchmarks, BaseRT, llama.cpp, Apple M5 Pro, Apple M5 Max, AMD Radeon RX 7900 XT, NVIDIA GeForce RTX 4060, PP512, and TG128.

Main Features

1. CLI-Driven Benchmark Capture with BaseRT and llama.cpp: The ComputeArena CLI installs on macOS and Linux via the command curl -LsSf https://computearena.ai/install.sh | sh -s launch. It requires curl, tar, and Python 3, and does not require sudo. After installation, an interactive terminal menu opens. Users choose BaseRT or llama.cpp as the runtime and run benchmarks offline. The CLI captures raw inference throughput, including decode tok/s and prefill tok/s, across models, quantisations, and chips. Example results include TinyStories Lay8 HS512 HD8 33M at 2,068.0 decode tok/s and 190,107 prefill tok/s on Apple M5 Pro with BaseRT Q4, and Qwen3 0.6B at 708.3 decode tok/s and 34,136 prefill tok/s on Apple M5 Max with BaseRT Q4.
2. Signed Community Reports and Offline-First Submission: Every public result originates from a signed report generated on community hardware. Reports remain on the user's machine until submission; no account is required to run benchmarks. Users sign in only when they choose to review and submit saved reports. This creates a provenance layer for local AI benchmarks, linking each result to a user, model identifier, runtime, quantisation, chip, backend, date, and measured workload. The platform shows entries such as Qwen/Qwen3-4B with qwen default-q4 on Apple M5 Pro, google/gemma-4-E4B-it with gemma4 default-q4 on Apple M5 Max, and Qwen/Qwen3.8-27B with llama.cpp Q4_0 on a 4x Tesla T10 configuration.
3. Model × Quantisation × Chip Leaderboard and Compare Engine: The leaderboard ranks compatible PP512 (512-token prefill) and TG128 (128-token decode) workloads, displaying decode tok/s and prefill tok/s beside every result. Filters include model, runtime, quantisation, chip, prefill size, and rank by decode or prefill. The sampled leaderboard contains 176 matching configurations. Supported runtimes include BaseRT and llama.cpp. Backends include Metal, BLAS + Metal, Vulkan, ROCm, and CUDA. Chips include Apple M5 Pro, Apple M5 Max, Apple M5, AMD Radeon RX 7900 XT (RADV NAVI31), AMD Radeon RX 7900 XT (ROCm), NVIDIA GeForce RTX 4060 Laptop GPU, and multi-GPU Tesla T10. Quantisations include default-q4, default-q8, Q4_0, Q4_K_M, Q8_0, Q2_K, and Q3_K_M. The compare engine generates URLs such as /compare?dimension=chip&a=apple-m5-pro&model=qwen3-0-6b&runtime=basert&quant=base-q4, enabling side-by-side chip, model, runtime, and quantisation analysis.

Problems Solved

1. Pain Point: Local AI users face fragmented, non-standard, and often unverifiable performance data. Vendor benchmarks may not reflect real quantised inference; community screenshots lack runtime, backend, and workload context; and hardware selection is complicated by model size, quantisation, memory bandwidth, and runtime optimizations. Compute:Arena addresses the problem of "how many tokens per second can this model actually run on this chip with this quantisation?" by standardizing PP512 prefill and TG128 decode workloads and requiring signed reports.
2. Target Audience: AI engineers, MLOps engineers, edge AI developers, local LLM enthusiasts, hardware reviewers, procurement teams, and model optimization researchers. Specific personas include developers deploying Qwen3, Gemma, Llama, GPT-OSS, Nemotron, or TinyStories models on Apple Silicon, AMD Radeon, NVIDIA GeForce, or Tesla GPUs; platform engineers comparing BaseRT vs llama.cpp; and technical buyers evaluating Apple M5 Pro vs M5 Max vs Radeon RX 7900 XT vs RTX 4060 Laptop for local inference.
3. Use Cases: Selecting a model and quantisation for a laptop, desktop, or server; comparing Q4_K_M, Q8_0, Q4_0, default-q4, and default-q8 trade-offs; measuring prefill throughput for RAG, document Q&A, and long-context prompts; measuring decode throughput for chat, code generation, and agentic loops; validating Metal vs Vulkan vs ROCm vs CUDA backends; benchmarking multi-GPU Tesla T10 configurations; and submitting signed reports to contribute reproducible local AI performance data.

Unique Advantages

1. Differentiation: Compute:Arena differentiates itself through community hardware provenance and granular configuration capture. Instead of a single headline score, it publishes decode tok/s and prefill tok/s for specific model, quantisation, runtime, backend, and chip combinations. It covers both BaseRT and llama.cpp, supports macOS and Linux, and includes Apple Silicon Metal, AMD Vulkan/ROCm, NVIDIA CUDA, and multi-GPU setups. The leaderboard is filterable by model, runtime, quantisation, chip, and prefill size, with 176 matching configurations in the sampled view.
2. Key Innovation: The key innovation is the signed, offline-first benchmark report pipeline. Users run benchmarks locally without an account, keep reports on their machine, and submit only what they choose. This preserves privacy while enabling public, attributable performance data. Standardized PP512 and TG128 workloads make cross-device comparisons more meaningful, and the model × quantisation × chip leaderboard turns raw benchmark data into actionable local AI deployment intelligence.

Frequently Asked Questions (FAQ)

1. What is Compute:Arena? Compute:Arena is a community-submitted benchmarking platform and CLI for local AI models. It measures decode and prefill throughput on any hardware, runtime, and quantisation, using signed reports from community hardware. It is used for local LLM performance comparison, edge AI benchmarking, and model × quantisation × chip leaderboards.
2. How does Compute:Arena measure local AI model performance? Compute:Arena uses the ComputeArena CLI with BaseRT or llama.cpp to run standardized workloads. Headline rankings compare PP512 (512-token prefill) and TG128 (128-token decode), reporting prefill tok/s and decode tok/s. Each result is linked to a model, quantisation, runtime, backend, chip, user, and measured workload.
3. Is Compute:Arena free and does it require an account? The ComputeArena CLI is installed on macOS or Linux via the install script and requires curl, tar, and Python 3. No account is required to run benchmarks offline. Users sign in only to review saved reports and submit selected benchmarks publicly. No sudo is required.
4. What hardware and runtimes does Compute:Arena support? Compute:Arena supports macOS and Linux, with runtimes including BaseRT and llama.cpp. Supported hardware examples include Apple M5 Pro, Apple M5 Max, Apple M5, AMD Radeon RX 7900 XT with RADV NAVI31 Vulkan or ROCm, NVIDIA GeForce RTX 4060 Laptop GPU with CUDA, and multi-GPU Tesla T10. Backends include Metal, BLAS + Metal, Vulkan, ROCm, and CUDA.
5. How do I compare quantisation impact on tokens per second? Use the Model × Quantisation × Chip leaderboard to filter by model, chip, runtime, and quantisation, then rank by decode or prefill. For example, you can compare Qwen3 0.6B default-q4 vs default-q8 on Apple M5 Max, or Q4_K_M vs Q8_0 on AMD Radeon RX 7900 XT. The compare URLs allow side-by-side analysis of chip, model, runtime, and quantisation.