Executive Summary
The software stack running today's most advanced AI models was designed for a fundamentally different era of computing. Linux — the dominant substrate for ML workloads — was architected in the early 1990s around a CPU-centric model of computation, with GPUs and accelerators treated as peripheral I/O devices. That architectural mismatch now imposes measurable performance penalties at every level of the stack: memory management, scheduling, hardware abstraction, and driver overhead.
This paper introduces the Machine Intelligence Operating System (MIOS): a proposed operating system architecture in which neural computation is a first-class kernel concern, not an application running atop a general-purpose OS. MIOS does not require discarding decades of systems research. It builds on that foundation while inverting the fundamental assumption that CPU processes are the primary unit of execution.
Three claims anchor this proposal:
- Legacy OS overhead is structural, not incidental. Existing workarounds — CUDA, cuDNN, XLA, NCCL — are impressive engineering responses to architectural constraints that should not exist at the kernel level.
- A purpose-built MI operating system is technically viable today. Mature NPU/TPU silicon, open-source microkernel research (seL4, Redox), and existing UEFI/bare-metal deployment patterns provide a realistic starting point.
- The strategic window for standardization is open, but not indefinitely. Proprietary hardware ecosystems (CUDA lock-in, Apple Silicon, Huawei Ascend) are converging toward fragmented, vendor-controlled stacks. An open standard established now prevents that fragmentation from hardening.
We propose the formation of the MIOS Open Standard Consortium — a multi-stakeholder collaboration between technology companies, national laboratories, and research universities — with a mandate to deliver a validated MIOS research prototype within 24 months and a production-ready V1.0 specification within 48 months, targeting data center and high-performance computing deployments.
The intelligence can no longer sit on top of the operating system as an application. The infrastructure must be redesigned from the ground up to treat neural computation as a primary workload.
The Problem: Structural Overhead in Legacy Operating Systems
When engineers at major AI labs describe infrastructure challenges, they rarely cite model architecture as the bottleneck. They cite memory bandwidth, PCIe transfer latency, driver overhead, scheduling jitter, and NUMA topology mismanagement. These are operating system problems — and they persist precisely because the OS was not designed for the workloads now running on it.
2.1 — GPU as a Peripheral: The Architectural Root
In a standard Linux kernel, a GPU is nominally managed via the DRM/KMS subsystem — the same framework used for display outputs. On paper, that implies every execution request pays for user-to-kernel transitions, PCIe crossings, and scheduler interruptions on the critical path. In practice, modern proprietary drivers have already routed around most of this: NVIDIA's CUDA runtime writes commands directly into a memory-mapped hardware ring buffer and signals execution via a hardware doorbell register, with the GPU's own DMA engine fetching and running commands asynchronously and with minimal Linux kernel scheduler involvement. AMD's ROCm stack does something similar via its user-queue mechanism. We state this plainly because an earlier draft of this paper overclaimed here: MIOS does not win by "eliminating the user-kernel boundary" for command submission, because CUDA has already eliminated most of it.
What legacy OS architecture still imposes is different and, we believe, still real: no single-node OS component has visibility across processes, models, and devices to schedule tensor work globally, preempt fairly under memory pressure, or reason about a workload's intent rather than just its syscalls — because that visibility was never designed as a first-class OS responsibility. CUDA's per-process user-space queues are fast in isolation but structurally blind to what's happening in the queue next to them. That's the gap MIOS is actually proposing to close: not raw submission latency, but cross-workload, intent-aware orchestration that isolated user-space runtimes cannot provide by construction. CUDA remains a remarkable and, for the command-submission problem specifically, largely solved piece of engineering; MIOS is not proposing to re-solve it.
2.2 — Memory Management at the Wrong Granularity
Linux's virtual memory system operates on 4KB base pages (with optional 2MB/1GB huge pages), designed for rapid context-switching between lightweight, independent processes. Large language model workloads operate at an entirely different scale: a 70B parameter model in BF16 requires approximately 140 GB of contiguous, high-bandwidth memory. A Mixture-of-Experts architecture like Mixtral 8x22B requires careful expert routing across memory regions that may span multiple NUMA nodes.
When the Linux kernel's OOM killer, transparent huge page compaction, or NUMA balancing daemon interrupts a training run to reclaim pages, the performance penalty is not marginal — it can trigger cascading tensor reallocation across an entire model-parallel job. Current mitigation strategies (memory locking via mlock(), cgroup isolation, NUMA pinning via numactl) require deep infrastructure expertise and are not portable across hardware configurations.
2.3 — Scheduler Blindness to Workload Semantics
The Linux Completely Fair Scheduler (CFS) treats all runnable threads as equal citizens, distributing CPU time proportionally across competing processes. This is a good model for interactive desktop applications. It is a poor model for a distributed training job where hundreds of CUDA streams must synchronize at a collective communication barrier across a multi-node cluster.
When even one process in an all-reduce operation stalls due to OS scheduling jitter, the entire synchronization group waits. GPU utilization drops. This is a known problem — Meta's research on FSDP training efficiency and Google's Pathways scheduling work both identify OS scheduler interactions as a significant source of hardware underutilization on large clusters.
Note on existing mitigations: Linux's cgroups v2, real-time scheduling classes (SCHED_FIFO), CPU isolation (isolcpus), and frameworks like SLURM represent significant progress in adapting Linux for ML workloads. MIOS acknowledges this work — the goal is not to dismiss it, but to recognize that these are shims over a fundamentally mismatched foundation. The engineering cost of these workarounds is itself an argument for architectural reconsideration.
2.4 — The Current Stack, Visualized
The following diagram illustrates how a typical LLM inference request traverses the current Linux-based ML stack, and where MIOS proposes to absorb or eliminate overhead layers:
┌─────────────────────────────────────────────┐ ┌─────────────────────────────────────────────┐ │ CURRENT LINUX-BASED ML STACK │ │ PROPOSED MIOS STACK │ ├─────────────────────────────────────────────┤ ├─────────────────────────────────────────────┤ │ Application Layer │ │ Application Layer │ │ Python / PyTorch / JAX │ │ Python / MIOS-ML / existing frameworks │ ├─────────────────────────────────────────────┤ ├─────────────────────────────────────────────┤ │ Middleware (overhead target) │ │ Kernel-Native Tensor Runtime │ │ CUDA Runtime / cuDNN / NCCL │ │ Neural graph scheduler, unified memory mgr │ │ XLA / OpenCL / ROCm │ │ (adds cross-workload visibility CUDA lacks) │ ├─────────────────────────────────────────────┤ ├─────────────────────────────────────────────┤ │ Kernel (overhead target) │ │ MIOS Kernel │ │ Linux CFS scheduler │ │ Intent-aware workload scheduler │ │ Virtual memory / page tables │ │ Unified ML memory model (regions, not pages)│ │ DRM/KMS GPU driver subsystem │ │ Direct-to-silicon HAL │ │ PCIe device abstraction │ │ Lightweight POSIX compatibility layer │ ├─────────────────────────────────────────────┤ ├─────────────────────────────────────────────┤ │ Hardware │ │ Hardware │ │ GPU / NPU / TPU (treated as I/O devices) │ │ GPU / NPU / TPU (first-class compute units) │ └─────────────────────────────────────────────┘ └─────────────────────────────────────────────┘
A Note on Terminology: From AI to MI
This paper uses the term Machine Intelligence (MI) in preference to Artificial Intelligence (AI). The distinction is not merely semantic. "Artificial" implies simulation — a machine approximating human cognition. That framing anchors infrastructure decisions to human-centric baselines and historically justified running ML on the same OS used for email and spreadsheets.
Understanding these systems as executing a genuinely distinct form of computation — massively parallel, probabilistic, operating over high-dimensional tensor spaces — leads to different infrastructure requirements. An MI workload is not a faster version of a traditional application. It has different memory access patterns, different parallelism characteristics, and different performance sensitivities.
Adopting MI terminology also supports a taxonomy for talking about more capable future systems. The first two tiers below describe things we can measure or reasonably extrapolate; the last two are speculative and are marked as such deliberately, since this document's actual scope is the current tier:
Current systems. Highly capable, domain-specialized. Infrastructure bottlenecks are measurable and addressable today. This is the scope of the MIOS proposal in this document.
Cross-domain reasoning with more autonomous knowledge transfer than current systems. A plausible near-term direction, though its timeline and exact character are genuinely uncertain and disputed among researchers.
Performance exceeding human capability across measurable cognitive domains. Speculative; included for taxonomic completeness, not as a design target of this proposal.
Whether subjective experience of any kind could arise in a computational system is an open scientific and philosophical question, not an engineering target. We discuss this as a long-horizon open question, deliberately separate from the MIOS technical proposal — see the research notes for the full caveat.
The MIOS Core Architecture
MIOS proposes a hybrid microkernel architecture in which a small, formally verifiable kernel core manages hardware resources and isolation, while MI-specific subsystems — the tensor runtime, the unified memory manager, and the workload scheduler — operate as privileged kernel services rather than user-space daemons or middleware libraries. This approach draws from seL4's capability-based security model and the L4 microkernel lineage, while incorporating lessons from Google's Fuchsia (component-based isolation) and the DPDK/SPDK data plane acceleration literature.
On microkernel IPC cost: a legitimate objection to any microkernel proposal is that inter-process communication between isolated kernel services is slow, and gets slower under hardware mitigations for speculative-execution vulnerabilities (Meltdown/Spectre-class). Even seL4's optimized fastpath IPC degrades substantially — by as much as 44% on some workloads — once those mitigations (e.g., KPTI) are enabled. MIOS's IPC-heavy tensor dispatch path would inherit this problem if implemented with traditional IPC. The mitigation this proposal relies on is retrofitting hardware memory-protection-key features (Intel PKU/ARM MPK-class mechanisms) to move trusted subsystems between user and kernel space dynamically while preserving isolation — an approach demonstrated in recent microkernel research to bring IPC roundtrip cost down by roughly 3x versus prior state of the art. This is called out explicitly as a load-bearing assumption of the architecture, not a solved problem we are taking for granted.
On the Hardware Abstraction Layer and existing standards: MIOS does not propose to build a competing open hardware abstraction standard from scratch. The Linux Foundation's UXL Foundation (evolved from Intel's oneAPI, with Arm, Google Cloud, Intel, Qualcomm, and Samsung already participating) is already developing "Level Zero," a multi-vendor, direct-to-metal accelerator interface that covers substantially the same ground as the HAL described below. MIOS's default position is to consume Level Zero as its hardware abstraction layer rather than duplicate it, and to scope the HAL pillar below as the kernel-side integration of that existing standard rather than a new one. Where Level Zero does not yet cover a MIOS-specific need (e.g., MMR-typed allocation metadata), we intend to propose extensions upstream before building a parallel spec.
The architecture rests on four integrated pillars:
Tensor operations and neural graph executions are first-class kernel entities, not user-space processes dispatching to drivers. The MIOS scheduler understands the dependency graph of an active neural computation — it can prioritize forward-pass operations, delay non-critical background tasks during a synchronization barrier, and schedule collective communication operations (all-reduce, all-gather) with awareness of network topology. Its value proposition is cross-workload, global visibility that per-process user-space runtimes (including CUDA's own submission path) do not have by construction — not the elimination of a user-kernel boundary that, on the critical path, largely no longer exists in modern GPU drivers (see §2.1).
MIOS defines a minimal Hardware Abstraction Layer (HAL) in which NPUs, GPUs, and TPUs are enumerated as primary compute resources at boot time — comparable to how modern systems treat CPU cores. HAL drivers are deliberately minimal: their sole function is DMA buffer management and command queue submission. All tensor operation semantics live in the kernel runtime above the HAL, enabling hardware-agnostic model execution across NVIDIA GPUs (via open kernel modules), Google TPU v4/v5, AMD Instinct MI300, and custom NPU designs. Rather than publishing a new open standard from scratch, MIOS's default plan (see above) is to implement this layer on top of the UXL Foundation's Level Zero specification, positioning MIOS as a consumer of that HAL rather than a competitor to it.
MIOS replaces the traditional virtual memory model for ML workloads with a region-based allocation system. Rather than managing 4KB pages, UMMA operates on model memory regions (MMRs): typed, variable-size allocations that carry metadata including precision (FP32/BF16/INT8), accelerator affinity, and persistence policy. A 70B parameter model would be represented as a set of named MMRs — "attention.layer.0.weights", "mlp.layer.3.activations" — that the kernel can prefetch, migrate between HBM and host DRAM, and checkpoint without application involvement. This eliminates the mlock/hugepage dance required by current PyTorch/CUDA workflows. Independent research on GPU memory managers (application-transparent multi-page-size allocation) and on hardware-native paging for LLM serving reports gains in this range from substantially less ambitious versions of this idea, which we take as supporting evidence rather than proof of MIOS's specific numbers.
Rather than time-slice-based CPU/GPU scheduling, MIOS uses a priority model based on workload intent declarations: structured annotations that specify the computational goal, SLA requirements, and acceptable resource tradeoffs for a given job. A training run declares "throughput-optimized, preemptible"; an inference server declares "latency-bound, 10ms P99 target". The IDRA scheduler uses these declarations, combined with real-time telemetry from hardware performance counters, to make allocation decisions. Conflicts are resolved via a configurable policy hierarchy (SLA-first, fairness-weighted, or administrator-defined priority tiers). Fairness across co-located workloads is maintained through bandwidth accounting and starvation prevention mechanisms adapted from Linux's BFQ I/O scheduler. Token-level GPU pooling deployed at hyperscale has independently validated the economic case for this class of scheduling, though via a proprietary, cluster-level system rather than an OS-level, single-node one — see §8.1.
4.1 — Kernel Architecture Diagram
┌──────────────────────────────────────────────────────────────────┐ │ POSIX Compatibility Layer │ │ (syscall translation for legacy Linux applications) │ ├──────────────────┬───────────────────────┬───────────────────────┤ │ Tensor Runtime │ UMMA Memory Service │ IDRA Scheduler │ │ Neural graph IR │ MMR allocator │ Intent parser │ │ Op dispatch │ Prefetch engine │ Hardware telemetry │ │ Collective comms │ Checkpoint manager │ Fairness arbiter │ ├──────────────────┴───────────────────────┴───────────────────────┤ │ MIOS Microkernel Core │ │ Capability-based IPC · Interrupt handling · Security │ ├──────────────────────────────────────────────────────────────────┤ │ Hardware Abstraction Layer │ │ NVIDIA (Open Kernel Modules) · AMD ROCm HAL · TPU HAL │ │ Custom NPU interface · NVLink/InfiniBand fabric driver │ └──────────────────────────────────────────────────────────────────┘
4.2 — Relationship to Existing Systems
MIOS is not a wholesale replacement of Linux for general-purpose workloads. The proposed deployment model for MIOS V1.0 targets dedicated ML inference and training nodes — machines whose primary function is neural computation. General-purpose cluster management, job submission, and network services would continue running on standard Linux nodes. MIOS nodes expose a stable API surface compatible with existing orchestration tools (Kubernetes, SLURM) via a thin compatibility layer.
Vendor cooperation is a precondition for the HAL, and vendors do not have symmetric incentives to provide it. NVIDIA has spent roughly two decades and billions of dollars building CUDA lock-in and has little commercial reason to help an initiative explicitly designed to commoditize it; we do not assume NVIDIA support for MIOS V1.0 and are not building a plan that depends on it. Instead, MIOS V1.0 targets hardware from vendors with a real incentive to break CUDA's moat: AMD Instinct (MI300-class, via ROCm's existing user-queue mechanism), Google TPU v4/v5 (in partnership with Google), and custom ASIC designs via the HAL. NVIDIA support, if it comes, is realistically a Phase 3+ outcome driven by market pressure once MIOS demonstrates value on secondary hardware — not a Year 1 assumption. Apple Silicon (M-series unified memory architecture) represents a compelling secondary target given its hardware-unified CPU/GPU memory model, which aligns closely with MIOS's UMMA design principles, though Apple's ecosystem currently lacks the CUDA/cuDNN/PyTorch-backend maturity MIOS would need to bootstrap on it first.
The Software Ecosystem: A Purpose-Built Language
A redesigned kernel creates the opportunity — but not the necessity — for a redesigned programming model. The current dominant stack (Python + PyTorch/JAX, with C++ performance-critical paths) has significant inertia, and MIOS does not require discarding it. MIOS V1.0 exposes a Python-compatible API that existing frameworks can target. The native MIOS language, described below, is a longer-horizon research initiative.
5.1 — Limitations of the Current Stack
Python's role in ML is fundamentally that of an orchestration layer. Its actual performance on compute-intensive paths comes entirely from compiled C++/CUDA extensions (PyTorch ATen, JAX XLA). This creates a permanent semantic gap: developers write high-level Python that is progressively rewritten by JIT compilers, XLA, and AOT compilation passes until it bears little resemblance to the original code. Debugging across that translation boundary is a chronic pain point for ML engineers.
C++ offers the performance but imposes manual memory management and deterministic execution semantics that are structurally ill-suited to probabilistic inference, dynamic compute graphs, and the speculative execution patterns common in modern transformers.
5.2 — The MIOS Native Language: Design Goals
The MIOS Programming Language (working name: Mira) is a statically typed, compiled language targeting the MIOS kernel tensor runtime directly. It is not a scripting language or a Python replacement — it is closer in spirit to a domain-specific Rust: safe, expressive, and with zero-cost abstractions for tensor operations. Key design principles:
- Tensor-native types: Tensors are a first-class type, with shape, dtype, and device locality encoded in the type system. Shape errors are caught at compile time, not at runtime during a 10-hour training run.
- Probabilistic execution blocks: Native syntax for stochastic sampling, beam search, and Monte Carlo operations — expressed as language constructs rather than library calls.
- Intent annotations: Functions annotated with
@inference,@training, or@realtimecompile to execution graphs with different optimization profiles, directly communicating workload intent to the IDRA scheduler. - Kernel-direct compilation: The Mira compiler targets the MIOS kernel's native tensor IR, bypassing the Python-to-XLA-to-LLVM-to-PTX translation stack. A training step that currently crosses five compiler boundaries crosses two.
5.3 — Illustrative Syntax Comparison
The following contrasts a simplified transformer attention computation in current PyTorch Python versus a proposed Mira equivalent. This is an illustrative sketch of language design direction, not a finalized specification:
# No shape guarantees at call site. Device placement implicit. # Runtime errors if Q, K, V are on different devices or have mismatched dtypes. def scaled_dot_product_attention(Q, K, V, mask=None): d_k = Q.size(-1) scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(d_k) if mask is not None: scores = scores.masked_fill(mask == 0, -1e9) weights = F.softmax(scores, dim=-1) return torch.matmul(weights, V)
// Shape and dtype checked at compile time. Device affinity declared. // @inference annotates IDRA scheduler: latency-sensitive, non-preemptible. @inference(latency_class: .realtime) fn scaled_attention[B: usize, H: usize, S: usize, D: usize]( Q: Tensor[B, H, S, D; bf16 @ npу], K: Tensor[B, H, S, D; bf16 @ npu], V: Tensor[B, H, S, D; bf16 @ npu], mask: Option[Tensor[B, H, S, S; bool]] ) -> Tensor[B, H, S, D; bf16 @ npu] { let scale = 1.0 / sqrt(D as f32); let scores = (Q @ K.T()) * scale; let masked = mask.map_or(scores, |m| scores.fill_where(!m, f32::NEG_INFINITY)); masked.softmax(dim: .seq) @ V }
The key differences are not syntactic novelty. They are semantic guarantees: shape correctness at compile time, explicit device locality, and scheduler intent communicated through the type system rather than through external configuration. Mira is designed to be learnable by existing PyTorch engineers — not to replace their mental model, but to move their runtime errors to compile time.
Adoption strategy: Mira is a Phase 3 initiative. MIOS V1.0 exposes a Python-compatible C API and a PyTorch backend that targets the MIOS tensor runtime. Mira development begins in parallel with V1.0 validation and targets production readiness at MIOS V2.0. Existing PyTorch/JAX codebases continue running unchanged on MIOS V1.0.
On the risk of language displacement: attempts to displace Python with a superior domain-specific compiled language have a poor track record — Julia and Swift for TensorFlow are the most direct precedents, and both stalled against Python's enormous ecosystem of auxiliary libraries rather than any technical deficiency in the language itself. We take this as a serious risk to Mira specifically, not a reason to be confident about it. Mira is scoped, deliberately, as an optional tool for teams that want compile-time shape and device-locality guarantees — never a required migration path. If Mira never reaches meaningful adoption, MIOS V1.0's value (kernel, scheduler, memory architecture) is unaffected, since it does not depend on Mira succeeding.
Prototype Path: How MIOS Starts Today
A common objection to clean-slate OS proposals is that they are compelling in theory but lack a credible starting point. This section addresses that directly. MIOS does not begin with a blank editor buffer. It begins with proven components, assembled in a staged approach that generates measurable results at each phase before committing to the next.
6.1 — Phase 0: Linux-Based Research Prototype (Months 1–12)
The first MIOS deliverable is not a new kernel. It is a modified Linux kernel with targeted modifications that demonstrate the core MIOS thesis: that kernel-level awareness of ML workloads produces measurable improvements over the current stack.
- Kernel module: MIOS Tensor Scheduler (MTS). A Linux kernel module that intercepts GPU command queue submissions (via the NVIDIA open kernel module hook points) and applies intent-aware scheduling. Benchmarked against baseline CFS + CUDA on LLaMA-3 70B inference throughput and P99 latency.
- Memory region prototype. A modified memfd + huge-page allocator implementing the MMR interface. Validated against current
mlock()+ NUMA pinning approaches on model loading latency for common open-weights models. - IDRA policy daemon. A user-space proof of concept for intent-based allocation, communicating with the MTS module via a Netlink interface. Provides data for the kernel-native implementation in Phase 1.
This phase produces three things: benchmark data validating the MIOS thesis, a team of engineers with deep familiarity with the target problem space, and an HAL interface draft derived from real hardware integration experience.
6.2 — Phase 1: Microkernel Core (Months 13–30)
With validated benchmarks, the consortium undertakes development of a functional MIOS microkernel core, built on the seL4 formal verification framework as a foundation for the capability-based security model. This phase produces a working core, not a formally verified one — full verification of a codebase this size is a multi-year undertaking in its own right (see §7.3) and is scoped separately, later in the roadmap. The MIOS tensor runtime and UMMA are implemented as privileged kernel services above the microkernel. A POSIX compatibility layer (similar in approach to Fuchsia's POSIX emulation) provides Linux application compatibility for cluster tooling.
Target hardware for Phase 1 validation: AMD MI300X and a TPU v4 pod slice via a Google research partnership, chosen because both vendors have a commercial incentive to support an open alternative to CUDA. NVIDIA hardware support is not assumed for this phase (see §4.2 and §7.3).
6.3 — Developer Workflow
Engineers targeting MIOS V1.0 interact with the system through a familiar workflow with a modified target layer:
# Existing PyTorch code runs unchanged — MIOS appears as a new backend $ pip install torch-mios-backend # Model training — scheduler receives intent from MIOS backend $ mios-run --intent=training --priority=throughput train.py # Inference server — latency-bound intent declared $ mios-run --intent=inference --sla=p99:8ms serve.py # Inspect active workloads and resource allocation $ miosctl workloads list WORKLOAD_ID INTENT SLA_TARGET NPU_ALLOC STATUS wl-0042 inference p99:8ms GPU0:72% ACTIVE wl-0041 training throughput GPU1:98% ACTIVE # MMR memory snapshot — named regions, not anonymous addresses $ miosctl memory snapshot --model=llama3-70b REGION SIZE DEVICE PRECISION llama3.embed.weights 2.1GB HBM0 bf16 llama3.attn.layer[0..31].weights 68.4GB HBM0-1 bf16 llama3.mlp.layer[0..31].weights 42.2GB HBM1 bf16 llama3.kv_cache.active 8.3GB HBM0 fp16
The mios-run launcher is the primary interface for Phase 0 and V1.0. It negotiates with the IDRA scheduler, sets up MMR regions for the model, and provides the MIOS backend hook to PyTorch/JAX. No code changes are required for existing model training scripts in Phase 0.
Execution Strategy: The MIOS Consortium
Foundational computing infrastructure with broad adoption has historically emerged from multi-stakeholder consortia, not from single-vendor mandates — even when the single vendor had superior technical resources. TCP/IP, USB, OpenGL, POSIX, and NVMe all followed this pattern. MIOS proposes the same governance model — while taking the historical timelines of those precedents seriously, which the table below does explicitly.
7.1 — Why Collaboration at the Infrastructure Level Makes Sense
Technology companies that compete intensely at the model and application layer share a common interest in a more efficient substrate. Each is currently spending substantial engineering resources on bespoke OS workarounds: custom NUMA configurations, proprietary CUDA tuning, internal memory allocators, and cluster-specific scheduler modifications. A shared, open foundation reduces that redundant overhead. Competition shifts to where it generates more value: model architecture, application design, and deployment infrastructure.
The CUDA ecosystem provides a useful cautionary parallel. NVIDIA's proprietary advantage at the software layer has created real lock-in effects — teams optimizing for CUDA cannot trivially migrate to AMD or TPU hardware. An open HAL standard breaks this dynamic, giving hardware vendors a level playing field and giving ML teams genuine hardware portability. We should be direct, however, that MIOS is not the only effort pursuing this: the UXL Foundation's Level Zero specification is already pursuing an open, multi-vendor accelerator interface with a multi-year head start and backing from Arm, Google Cloud, Intel, Qualcomm, and Samsung. MIOS's realistic path to an open HAL runs through building on Level Zero, not competing with it — see §4.
We should also be direct about who MIOS is actually competing with for attention and resources: hyperscalers are not standing still waiting for an open standard. Microsoft's Singularity, Google's Pathways, and Alibaba's Aegaeon are proprietary, cluster-scale scheduling systems already deployed in production that solve substantial parts of the problem MIOS describes — proof that the underlying diagnosis is correct, and a reminder that MIOS's differentiator has to be openness and single-node OS integration, not being first to notice the problem.
7.2 — Phased Roadmap
On timeline realism: an earlier draft of this roadmap targeted a production V1.0 within 48 months. Historical precedent for infrastructure standards of comparable or lesser complexity — TCP/IP, POSIX, OpenGL, USB, NVMe — argues that's the timeline for a specification, not a formally verified, production-ready operating system with a novel security model, hardware-agnostic drivers, and a new language. The table below shows the comparison; the roadmap after it separates "HAL specification" (achievable on something like the historical timeline) from "production OS" (which we now scope realistically at 6–10 years).
| Standard | Inception | First Spec | Broad Adoption | Time to Spec |
|---|---|---|---|---|
| TCP/IP | Early 1970s | 1980 | 1983 | ~10 years |
| POSIX | 1984 | 1988 | 1993+ | 4 years |
| OpenGL | 1991 | 1992 | 1997+ | 1.5 years |
| USB | 1994 | 1996 | 1998 | 2 years |
| NVMe | 2007 | 2011 | 2013 | 4 years |
USB — the fastest of these — was driven by seven tightly aligned companies solving a comparatively narrow problem (replacing serial/parallel ports), and still took two years to reach 1.0 and two more to see broad adoption. MIOS proposes a full operating system with a novel security model, hardware-agnostic drivers, and a new language — a materially larger undertaking.
Establish governing body and technical working groups (Kernel, HAL, Memory, Language, Security). Deliver Linux-based Phase 0 prototype (MTS kernel module, MMR allocator, IDRA daemon) and publish initial benchmark results. Begin engagement with the UXL Foundation on Level Zero rather than drafting a competing HAL from scratch. Target: 3–5 founding institutional members, prioritizing vendors with incentive to break CUDA lock-in (AMD, Google Cloud/TPU) over assuming NVIDIA participation.
Publish MIOS HAL v0.1 as an extension proposal against Level Zero rather than a standalone spec. Begin seL4-lineage microkernel core development, scoped for incremental formal verification (see §7.3) rather than a big-bang proof of the entire system. Ratify UMMA memory model specification. This is the point at which a defensible comparison to the historical "time to spec" precedents above is appropriate — not V1.0.
Deliver functional MIOS microkernel running on bare metal with AMD MI300-class and TPU targets. Integrate tensor runtime and UMMA as privileged kernel services. Port PyTorch to the MIOS tensor runtime backend so existing codebases run unchanged. First developer preview for consortium members. Begin incremental formal verification of the completed microkernel core.
Independent, adequately-resourced security audit of the verified microkernel core and HAL interface (see §7.3 for why this cannot be compressed into a "few months"). Performance validation against Linux baseline across training and inference benchmarks. POSIX compatibility layer stabilization for cluster tooling. Expand hardware support opportunistically, including NVIDIA if market conditions and vendor incentives have shifted.
Production-ready release for data center and HPC deployments. HAL extensions merged upstream into the relevant open standard. Mira language alpha release, evaluated strictly as an optional tool (see §5.3) — its absence of adoption would not block V1.0. Standardization engagement with IEEE/IETF working groups. We regard this as a realistic outer bound consistent with historical precedent, not a guarantee.
7.3 — Technical, Financial & Ecosystem Risks
The original seL4 verification took 11.5 person-years to produce roughly 500,000 lines of proof for 10,000 lines of C — independently estimated at around $400 per source line of code. A full ML operating system (kernel, MMR allocator, IDRA scheduler, tensor runtime) will run to hundreds of thousands of lines. Reverifying after every HAL or scheduler change would be untenable at that cost.
Mitigation: we scope formal verification incrementally — the microkernel core is held to full seL4-grade proof, while UMMA and IDRA are treated as a separate, smaller trusted computing base with lighter-weight verification, so iteration on scheduling logic doesn't force reverification of the proven core. Budget and timeline for this are carried explicitly in the Years 5–7 roadmap phase rather than compressed into a token audit line item.
Historical infrastructure standards of lesser scope (POSIX, USB, NVMe) took 2–4 years to reach a first specification and 6–10 years to reach the ecosystem dominance MIOS is ultimately aiming for. An earlier draft of this roadmap targeted full production readiness in 48 months, which is inconsistent with that precedent.
Mitigation: the roadmap in §7.2 now separates a HAL specification (achievable on something like the historical timeline) from a production V1.0 (realistically 6–10 years). We would rather be believed on a longer timeline than lose credibility over a short one we can't hit.
Hardware vendors may resist a unified HAL if it reduces their software differentiation advantage — and a standalone MIOS HAL would also duplicate the Linux Foundation's UXL/Level Zero effort, which has a multi-year head start and major vendor backing.
Mitigation: MIOS builds its HAL as an integration layer on top of Level Zero rather than a competing spec (see §4), proposing extensions upstream where gaps exist. Vendor-specific optimization layers remain valid above the HAL.
NVIDIA has no clear commercial incentive to support a standard explicitly designed to commoditize the hardware it dominates, and its absence would significantly limit initial credibility and reach.
Mitigation: V1.0 is scoped around vendors with an incentive to break CUDA lock-in (AMD, Google TPU) rather than assuming NVIDIA cooperation from the outset (see §4.2). NVIDIA support is treated as a possible later outcome of market pressure, not a dependency.
The Python/PyTorch/CUDA ecosystem has enormous momentum. Developer adoption requires compelling performance improvements, not just architectural elegance — and prior attempts to introduce a superior ML-native language (Julia, Swift for TensorFlow) have not displaced Python.
Mitigation: Phase 0 delivers benchmark data before any ecosystem commitment is required. V1.0 requires zero code changes for existing PyTorch workflows, and Mira (§5.3) is scoped as permanently optional rather than a required migration.
Microkernel architectures have historically paid a real IPC overhead penalty versus monolithic kernels, which worsens substantially under Meltdown/Spectre-class hardware mitigations (up to 44% on some seL4 workloads with KPTI enabled).
Mitigation: critical tensor operation paths rely on PKU/MPK-based intra-kernel isolation (see §4) to move trusted subsystems dynamically rather than paying full IPC cost — a technique shown in recent research to cut IPC roundtrip cost roughly 3x versus prior approaches. This is a load-bearing research dependency, not a settled fact, and is called out as such.
Strategic Impact
Control of foundational computing infrastructure has historically generated durable strategic advantage. The IBM 360 architecture defined enterprise computing for two decades. The Intel x86 ISA and Microsoft's OS position shaped the PC era. Google's Android defined mobile computing for the following decade. In each case, the organization that defined the infrastructure layer captured disproportionate value from every application built on top of it.
The ML infrastructure layer is presently fragmented and unsettled. CUDA dominates GPU computing but is a single-vendor proprietary standard. ROCm, OneAPI, and XLA each have momentum but lack the cross-vendor standardization that would enable genuine hardware portability. MIOS's open HAL standard addresses this gap directly — not by displacing existing frameworks, but by providing a hardware-agnostic interface beneath them.
8.1 — Economic Impact Scenarios
The economic case for MIOS rests on three compounding effects:
- Infrastructure cost reduction. Our 15–25% compute-overhead reduction estimate is conservative relative to recent point solutions in the exact domains MIOS targets: vLLM's PagedAttention demonstrated up to 24x throughput gains over naive baselines by treating GPU memory like OS-managed RAM; the follow-up vAttention work (ASPLOS 2025) showed a further 22–29% end-to-end serving improvement by restoring hardware-native paging; and academic OS-level schedulers like MSched report up to 57.88x gains on LLM inference under memory oversubscription. We read these as evidence that 15–25% is a credible floor for OS-native integration of the same ideas, not proof of MIOS's specific number, since none of these are full OS integrations.
- Hardware portability unlocks competition. Current CUDA lock-in creates a strong barrier to AMD, Intel Gaudi, and custom ASIC adoption. An open HAL standard that enables genuine hardware portability would accelerate competition in the accelerator market, driving price reduction — a significant indirect benefit to the entire ML industry.
- Edge and sovereign deployment. Nations and organizations seeking to deploy MI infrastructure without dependence on U.S. cloud providers (for data sovereignty or security reasons) require a portable, auditable software stack. MIOS, as an open standard, provides this — and creates a platform for U.S. technology leadership in the open-source infrastructure space.
- The scheduling case is already proven at hyperscale — just not openly. Alibaba's Aegaeon token-level GPU pooling system reduced the H20 GPUs required to serve its workload from 1,192 to 213 (an 82% reduction) in production. That's strong evidence intent-aware scheduling has real economic value; it's also a reminder that this value is currently being captured by proprietary, single-vendor systems, which is the gap an open MIOS-class standard would need to close rather than a gap it discovered.
8.2 — Industry Adoption Scenarios
| Sector | Current Pain Point | MIOS Value Proposition |
|---|---|---|
| Hyperscale Cloud | Bespoke OS customizations per cluster; CUDA vendor lock-in | Open HAL enables AMD/custom ASIC flexibility; shared OS investment reduces duplicated infra engineering |
| Drug Discovery & Genomics | Memory-bandwidth-bound protein folding workloads; HPC cluster fragmentation | UMMA's region-based model fits large genomic datasets; HPC-native deployment via SLURM compatibility |
| Autonomous Systems | Sub-millisecond inference latency limited by OS scheduling jitter | IDRA latency-class scheduling and real-time inference intent directly address this constraint |
| National Laboratories | Security-sensitive deployments require auditable, minimal-attack-surface software | Formally verified microkernel core; open source with auditable HAL interface; no proprietary blobs required |
| Deep-Space & Edge | Power constraints and communication latency require onboard autonomous inference | MIOS's minimal kernel surface and intent-aware power management suit constrained deployment environments |
Infrastructure standards are most effectively defined before the market consolidates around proprietary alternatives. The window for an open MI infrastructure standard is open today — it will not remain so indefinitely.
Conclusion & Call to Action
The case for MIOS is a straightforward systems argument: the infrastructure we use determines the capabilities we can achieve, and the gap between what current hardware can deliver and what current software allows is large enough to justify purpose-built solutions. This is not a novel observation — every major AI infrastructure team is managing this gap with bespoke workarounds. MIOS proposes to address the root cause rather than continue accumulating patches.
Critically, MIOS is not an all-or-nothing proposition. The phased prototype path described in Section 6 produces measurable results within 12 months, using existing kernel infrastructure, before any commitment to a clean-slate implementation is required. The Phase 0 Linux kernel module approach is a credible starting point for any well-resourced systems team. The benchmark data it generates will either validate the MIOS thesis or provide the evidence base for refining it.
This paper is an invitation to three distinct communities:
- Infrastructure engineers at technology companies who are currently writing bespoke OS workarounds: the Phase 0 prototype is a concrete, immediate contribution point. The HAL interface specification working group needs engineers with hands-on CUDA and ROCm driver experience.
- Academic computer science departments whose systems research groups are already working on microkernel design, formal verification, and hardware-software co-design: MIOS offers a research agenda with direct industrial relevance and a consortium deployment path for validated results.
- Policymakers and national laboratory directors who understand that foundational compute infrastructure is a strategic asset: MIOS proposes a concrete program with a phased roadmap, measurable milestones, and an open governance model that prevents single-vendor capture.
The right time to define foundational infrastructure is before the market consolidates around the alternatives. We are at that moment for machine intelligence infrastructure. The consortium model exists precisely for this situation: a problem too important to leave to market consolidation, and too large for any single organization to solve alone.
Join the Consortium
We are seeking founding members across technology companies, national laboratories, and research universities. Engineering contributors, hardware partners, and policy advisors are all welcome. Get in touch to express interest or discuss participation.