KerasFormers

Product Introduction

1. Definition: KerasFormers is an open-source library of pretrained transformer models implemented natively in Keras 3. It provides a unified, framework-agnostic interface for state-of-the-art models across computer vision, natural language processing, multimodal, and speech tasks.
2. Core Value Proposition: It exists to eliminate framework lock-in and dependency bloat in deep learning deployment. By providing pure Keras 3 implementations with converted weights, it enables researchers and engineers to run any of its 118 model families on JAX, PyTorch, or TensorFlow backends with a single, consistent API, without requiring the original transformers or torch libraries at runtime.

Main Features

1. Framework-Agnostic Execution: Models are written in pure Keras 3 operations. By setting the KERAS_BACKEND environment variable to "jax", "torch", or "tensorflow", the identical model code runs on the chosen deep learning framework. This decouples model logic from backend-specific implementations, enabling seamless portability across research and production environments.
2. Unified Model Loading API: The from_weights() method is a single entry point for model construction. It intelligently dispatches to load preconverted Keras weights from the Hugging Face Hub, dynamically convert upstream checkpoints (e.g., from "qwen3-8b"), or load architecture and weights from any compatible Hugging Face repository using an hf: prefix. It automatically infers architecture details like class counts from the repository's configuration file.
3. Integrated Pre/Post-Processing: Every model is paired with a dedicated Processor class (e.g., DETRImageProcessor). This ensures that input normalization, resizing, and tokenization exactly match the original pretrained checkpoint. Processors also include task-specific post-processing methods (e.g., post_process_object_detection) to convert raw model outputs into interpretable results like bounding boxes or text.
4. Advanced Quantization Support: The library supports state-of-the-art weight-only quantization techniques directly during model loading. This includes INT8, INT4, FP8, and MXFP4 formats. A key innovation is the handling of Mixture-of-Experts (MoE) models like GPT-OSS 120B, where experts remain packed in a compressed format (e.g., MXFP4) and are dequantized on-the-fly during inference, drastically reducing memory footprint without altering the model call signature.

Problems Solved

1. Pain Point: Framework Fragmentation and Dependency Hell. Machine learning practitioners often struggle with incompatible dependencies between model repositories (e.g., transformers, timm), specific PyTorch versions, and custom CUDA toolkits. This creates reproducibility nightmares and deployment friction.
2. Target Audience: ML Engineers and Researchers requiring production deployment, MLOps specialists, educators teaching transformer architectures, and developers building applications that need to switch between CPU/GPU/TPU hardware or different inference runtimes.
3. Use Cases: Deploying a single model across diverse hardware stacks (JAX on TPU, PyTorch on GPU, TensorFlow on CPU); integrating multiple model types (vision, text, speech) into a unified application pipeline without managing separate dependency environments; rapidly prototyping with new model architectures using a consistent Keras API; and reducing serving costs by applying advanced quantization to large language models (LLMs) and vision transformers.

Unique Advantages

1. Differentiation: Unlike Hugging Face's transformers library, which is PyTorch-first and requires translation for other backends, or framework-native repositories, KerasFormers provides truly native implementations for all three major backends from a single codebase. It offers deeper framework integration than wrapper-based solutions, resulting in better performance and debugging.
2. Key Innovation: The "pure Keras 3" implementation strategy is its core innovation. By strictly using Keras ops, it leverages Keras 3's ability to compile to different backends. Combined with its sophisticated weight conversion pipeline and unified loading API, this creates a portable, self-contained model ecosystem that prioritizes developer experience and deployment flexibility over framework-specific optimizations.

Frequently Asked Questions (FAQ)

1. How does KerasFormers differ from using Hugging Face Transformers with Keras? KerasFormers models are implemented from the ground up in Keras 3, making them native citizens of the Keras ecosystem. In contrast, using Hugging Face Transformers with Keras often involves a compatibility layer or model wrapping, which can introduce overhead, limit access to low-level ops, and fail for complex architectures. KerasFormers provides a cleaner, more integrated, and backend-agnostic experience.
2. Can I fine-tune KerasFormers models? Yes. Since models are standard Keras 3 objects, you can use all Keras APIs for training, including model.compile() and model.fit(). You can load weights (or architecture-only with load_weights=False) and fine-tune them on your custom datasets using any supported backend (JAX, PyTorch, TensorFlow).
3. Is there a performance penalty for using the pure Keras 3 implementation? Typically, the performance is on par with or exceeds native framework implementations for inference because Keras 3 compiles the model graph to the underlying backend's native operations (e.g., it becomes a PyTorch nn.Module when KERAS_BACKEND="torch"). For training, performance is dependent on the backend's optimizer and is generally excellent, especially with JAX and Torch backends.
4. How are models and weights licensed? The KerasFormers library code is open-source. The weights are converted from original checkpoints and inherit the licensing of their source models. You must comply with the license of the original model (e.g., Llama 3's Meta license, Gemma's Google license). The kerasformers organization on Hugging Face Hub clearly indicates the source and license for each converted model.
5. Does it support loading LoRA adapters or other parameter-efficient fine-tuning (PEFT) weights? The core library focuses on the base model implementations and weight conversion. While you can manually merge LoRA weights into the model layers, native, streamlined support for loading standalone PEFT checkpoints is a potential area for future development as the ecosystem matures.