nvke

Product Introduction

1. Definition: NVIDIA Kubernetes Engine (NVKE) is an open-source, API-driven Kubernetes management system and Certified Kubernetes AI Platform designed specifically for AI factory operators. Operating under the Apache 2.0 license, NVKE functions as a specialized infrastructure orchestration layer built to provision, scale, upgrade, and manage production-grade GPU Kubernetes clusters directly on bare-metal hardware. It acts as the core orchestration component within DSX OS, NVIDIA's comprehensive software stack for modern AI factories.

2. Core Value Proposition: NVKE eliminates the extreme operational complexity of building custom cluster management systems for large-scale GPU fleets. By automating node provisioning, lifecycle operations, security attestation, and isolated control plane creation, NVKE empowers AI factory operators, cloud service providers, and enterprise infrastructure teams to deliver high-performance managed Kubernetes services to their customers efficiently and at scale.

Main Features

1. Automated Bare-Metal Provisioning and Lifecycle Management: NVKE integrates natively with NVIDIA Infra Controller (NICo) via CAPNICo (Cluster API Infrastructure Provider for NICo) to automate bare-metal node lifecycle operations. Using the nvkectl command-line interface, operators can bootstrap a temporary seed cluster to establish a highly available management cluster. Once operational, the management plane handles day-2 operations—including node pool scaling, Kubernetes version upgrades, automated node replacement for faulty hardware flagged by NVSentinel, and rolling certificate renewals—without manual intervention.

2. Secure Node Attestation and Supply Chain Provenance: To secure bare-metal environments, NVKE mandates that each node undergo strict cryptographic attestation using its Trusted Platform Module (TPM) via SPIFFE/SPIRE before receiving join credentials. Furthermore, NVKE works in tandem with NVIDIA Kubernetes Recipes (NVKR), its companion build system. NVKR builds immutable node OS images, container binaries, and Helm charts from pinned sources, generating a comprehensive software bill of materials (SBOM) and cryptographically signed artifacts with verifiable build provenance.

3. Advanced Telemetry and High-Performance Networking: NVKE features a robust, OpenTelemetry-compliant monitoring architecture known as the NVKE telemetry pipeline (NKTP). NKTP deploys OpenTelemetry Collectors across management and workload clusters to aggregate metrics, logs, and traces, exporting them securely via the OpenTelemetry Protocol (OTLP). For cluster networking, NVKE utilizes an NVIDIA-curated, open-source build of OVN-Kubernetes, ensuring low-latency, scalable network overlays tailored for intensive AI/ML workloads.

Problems Solved

1. Pain Point: AI factory operators face massive friction when attempting to assemble, integrate, and maintain disparate open-source and commercial tooling to build a reliable custom cluster management system. Managing isolated control planes, validating upstream Kubernetes releases against complex GPU architectures, handling bare-metal node identity, and ensuring zero-downtime upgrades across thousands of GPU machines usually requires massive engineering overhead and introduces severe maintenance debt.

2. Target Audience: AI factory operators, NVIDIA Cloud Partners (NCP), Kubernetes Independent Software Vendors (ISVs), cloud platform engineering teams, and enterprise administrators who deploy managed container services over dedicated NVIDIA GPU infrastructure.

3. Use Cases:

   • Deploying isolated, managed Kubernetes control planes as the foundational orchestration layer for hosted AI training, fine-tuning, and managed inference model endpoints.
   • Provisioning large-scale GPU bare-metal clusters integrated with NVIDIA AI Cluster Runtime (AICR) components such as the GPU Operator, Network Operator, and Dynamic Resource Allocation (DRA).
   • Delivering conformant, upstream-compliant Kubernetes environments where multi-tenant customers can run their own workloads, namespaces, and application secrets without administrative visibility into underlying host infrastructure.

Unique Advantages

1. Strengths & Limitations (Pros & Cons):

   • Pros: Full open-source transparency (Apache 2.0); deep native integration with NVIDIA hardware stacks (NICo, NVKR, NVSentinel, AICR); CNCF-certified conformant Kubernetes; automated cryptographic node attestation via TPM/SPIFFE; robust supply chain security with signed artifacts and SBOMs.
   • Cons: Currently designated as a preview release (APIs, configurations, and behaviors are subject to change between releases, making it unrecommended for strict production workloads without proper mitigation); requires significant upfront infrastructure planning (requires NICo and pre-prepared bare-metal node configurations); does not natively manage bare-metal firmware or low-level network discovery (relies entirely on external infrastructure controllers).

2. Key Alternatives & Differentiation:

   • Alternative 1: Commercial Managed Kubernetes Platforms (e.g., Red Hat OpenShift, VMware Tanzu). While these offer enterprise polish, they often introduce vendor lock-in, licensing costs, and lack native, low-level integration with specialized NVIDIA GPU orchestration tools (like AICR and NICo). NVKE is purpose-built as an open-source layer explicitly for AI factories running NVIDIA hardware.
   • Alternative 2: Vanilla Upstream Kubernetes with Custom Bash/Ansible Scripts. Building a custom lifecycle management system via raw Cluster API and homegrown scripts lacks integrated telemetry pipelines (NKTP), TPM-based node attestation out of the box, and verified supply chain registries provided by NVKR, leading to unsustainable maintenance overhead during Kubernetes and GPU driver upgrades.

Frequently Asked Questions (FAQ)

1. What is the primary function of NVIDIA Kubernetes Engine (NVKE)? NVKE is an open-source Kubernetes management system that enables AI factory operators to automatically provision, scale, upgrade, and manage GPU Kubernetes clusters on bare-metal infrastructure through an API-driven control plane.

2. Is NVKE recommended for production environments? No. NVKE is currently available as a preview release. APIs, configurations, and system behaviors are subject to change across releases, and NVIDIA does not recommend deploying NVKE for mission-critical production workloads during the preview phase.

3. What is the relationship between NVKE and NVIDIA Kubernetes Recipes (NVKR)? NVKR is the companion build system for NVKE. It compiles and builds the node OS images, binaries, container images, and Helm charts deployed by NVKE, ensuring every artifact is generated from pinned sources with cryptographic build provenance and a software bill of materials (SBOM).

4. How does NVKE handle node security and identity verification on bare metal? NVKE secures bare-metal node onboarding by requiring each machine to perform cryptographic attestation using its Trusted Platform Module (TPM) via SPIFFE/SPIRE before a join credential is ever issued to the node.

5. Does NVKE manage bare-metal hardware discovery and firmware updates? No. NVKE does not manage bare-metal machine provisioning or firmware updates directly. It delegates physical machine lifecycle management, discovery, and network configuration to the NVIDIA Infra Controller (NICo) via CAPNICo.