Codex GPU Queue

Product Introduction

1. Definition: The Codex GPU Queue is a local, Windows-based job orchestration and scheduling system designed for managing GPU workloads from multiple AI agent tasks. It falls under the technical categories of GPU resource management, local job queuing, and multi-agent orchestration.
2. Core Value Proposition: It exists to enable efficient sharing of a single NVIDIA GPU among several concurrent, agent-driven workloads (like those from OpenAI Codex), preventing conflicts and automating job execution. Its core value is providing a shared broker-managed queue with VRAM-aware admission control and automatic job dispatch, allowing submitting tasks to continue without manual intervention.

Main Features

1. Shared SQLite Queue with Idempotent Submissions: The system uses a centralized SQLite database as a persistent queue. Jobs submitted from multiple Codex tasks or a CLI are stored idempotently, preventing duplicate entries. A broker lease mechanism ensures only one management process is active per queue state directory, coordinating work.
2. VRAM-Aware and Policy-Based Scheduling: The scheduler performs VRAM admission control using estimates from job submissions and live telemetry from nvidia-smi. It supports explicit exclusive jobs, priority levels, job aging to prevent starvation, and backfill scheduling to utilize fragmented GPU memory. It manages Windows processes, handles bounded results, cancellation signals, and bounded retries for failed executions.
3. Read-Only Waiting Job Observer (CLI): A dedicated CLI tool (observe) provides a redacted, read-only view of the queue state. It connects to the live SQLite database in read-only mode to report why specific jobs are waiting, showing observed_blocker reasons (like explicit exclusivity or unmet dependencies) or undetermined status, without exposing private job details, commands, or logs. It uses SQLite's mode=ro and query_only for safe inspection.

Problems Solved

1. Pain Point: GPU contention and manual coordination when multiple AI agents or tasks on a single Windows workstation need to use the same NVIDIA GPU for inference, training, or other CUDA workloads. This leads to out-of-memory errors, failed processes, and inefficient resource utilization.
2. Target Audience: Developers and researchers running multiple AI agent workloads (e.g., with OpenAI Codex) on a single, GPU-equipped Windows machine. ML engineers needing to orchestrate local GPU experiments. Power users sharing a workstation between different GPU-intensive tasks.
3. Use Cases: Sequencing Codex-generated GPU jobs from separate chat sessions; managing a local inference server shared by multiple scripts; orchestrating batch processing of ML models where jobs have varying VRAM requirements and durations; providing visibility into queue bottlenecks without accessing sensitive job data.

Unique Advantages

1. Differentiation: Unlike system-level GPU virtualization or cluster schedulers (like Slurm), it is a lightweight, local, application-level queue requiring no kernel modules, complex infrastructure, or paid APIs. Compared to manual scripting, it provides automated, stateful scheduling with persistence and recovery.
2. Key Innovation: The integration of a privacy-preserving, read-only observer that shares the scheduler's explicit-exclusivity predicate for accurate blockage reporting, without compromising security or requiring a full production deployment. Its design uses bounded Windows Job Objects for process control and a shared broker model that returns control to the submitting agent upon job completion.

Frequently Asked Questions (FAQ)

1. How does the Codex GPU Queue prevent two jobs from using the GPU at the same time? It uses a centralized broker with a leasing system to manage a first-in, first-out queue enhanced by priority and aging policies. The broker admits and starts jobs only when the VRAM admission control check passes and no explicit exclusive jobs are running, ensuring serialized access.
2. Can I use the Codex GPU Queue without OpenAI Codex or MCP? Yes, the core functionality is accessible via its standalone Python CLI, allowing you to submit and manage GPU jobs from scripts or the command line. The MCP server integration is an optional layer for Codex/agent workflows.
3. What happens if my GPU job uses more memory than its declared --vram estimate? The VRAM admission is an estimate only and does not enforce a hard limit or provide an Out-Of-Memory (OOM) guarantee. The job may cause system instability or fail. Accurate estimates are crucial for scheduling quality.
4. Is the Codex GPU Queue a production-ready service? The reviewed candidate is a source package, not a certified product. It is derived from a core orchestrator and adds observational tools, but lacks features like installed services, automatic updates, or commercial support. It is intended for developer use.
5. How does the observe CLI protect my private job data? The observer uses hashed job references, redacted output, and a strictly read-only SQLite connection with a SELECT authorizer. It never reads command lines, environment variables, log payloads, or originating identities, displaying only allowlisted lifecycle event types and times.