If you run a Kubernetes cluster with more than one H100 and more than one team, the GPU sharing question comes up in the first month. Someone wants a fifth of an H100 for a notebook. Someone else wants 40 GB of VRAM for a fine-tuning job. The third team needs the entire GPU for a serving workload. Until recently, the answers were NVIDIA-only: MIG for hardware-isolated partitions, time-slicing for round-robin bursts, and the GPU Operator to wrap the lot. HAMi added a third answer: a vendor-neutral, CNCF-governed device plugin and scheduler that runs across NVIDIA, AMD, Ascend, Cambricon, Hygon, Iluvatar, MetaX, and MThreads accelerators — and it just moved from CNCF Sandbox to Incubating on 2026-07-02. The tier change is the procurement and security-review unlock that a lot of teams have been waiting on.

What HAMi actually is

HAMi is two components. The first is a device plugin that runs as a DaemonSet on every GPU node and advertises partitioned GPU resources to the Kubernetes scheduler. The second is a scheduler extender that hooks into the default Kubernetes scheduler and chooses which partition a pod lands on based on memory, compute, and policy constraints. The resource model is the part that matters: a pod asks for nvidia.com/gpu with a memory request and an optional compute percentage, and the scheduler places the pod on a partition of a physical GPU that satisfies the request.

This is more flexible than MIG (which requires Ampere/Hopper/Blackwell and only exposes fixed partition shapes), and more bounded than time-slicing (which round-robins the GPU across pods with no memory isolation). The HAMi model is closer to what production teams actually want: a fraction of an H100 with a memory budget and a compute budget, scheduled alongside the rest of the Kubernetes workload.

The other half of the value is cross-vendor. The same scheduler that handles nvidia.com/gpu on an H100 node can also handle ascend.com/npu on a Huawei Atlas node, amd.com/gpu on a MI300X node, and so on. For clusters that have already moved past a single-vendor story — the pattern the custom AI silicon comparison tracks — HAMi is the layer that lets a single Kubernetes scheduler speak to the heterogeneous fleet without per-vendor scheduler forks.

What CNCF Incubating means in practice

CNCF has three project maturity tiers: Sandbox (early-stage), Incubating (production users, governance, security audit), and Graduated (broad adoption, security audits on a cadence, conservative feature pace). The CNCF project page for HAMi states that the project was accepted to CNCF on August 21, 2024 and moved to the Incubating maturity level on July 2, 2026. The two-year gap between acceptance and incubation is the typical CNCF cadence for a project that picks up real production users before the incubation review.

For teams evaluating GPU sharing tooling, the Incubating tier changes the procurement conversation. Sandbox projects typically require an exception path or a vendor-supported fork to get through security review; Incubating projects have a security audit on file and a governance process that survives a personnel change. If your security review has been the blocker on adopting HAMi, the tier change is the moment to re-open the conversation. The HAMi GitHub repository lists 4,400+ stars and a maintainer list that spans individual contributors plus vendor-affiliated maintainers from the China-cloud ecosystem — the multi-org contributor base that CNCF Incubating requires.

Why heterogeneous matters more in 2026

Eighteen months ago, the answer to "what accelerator should I buy" was "NVIDIA, full stop, don't waste my time." That answer is wrong now in three concrete ways. First, the price-per-token arithmetic on AMD MI300X and the AWS Trainium2 stack has crossed the threshold where a multi-vendor fleet is operationally cheaper than an all-NVIDIA fleet for inference workloads — the custom AI silicon comparison runs the numbers. Second, regional availability has become a procurement variable: Huawei Ascend is the dominant accelerator in Chinese data centers, and teams serving both US and APAC workloads are running heterogeneous fleets whether they planned to or not. Third, the inference economics on long-context and reasoning models favor accelerators with more memory bandwidth per dollar than the current NVIDIA lineup offers — which is the structural reason AMD and the in-house silicon programs at Google (TPU), AWS (Trainium), and Microsoft (Maia) have been gaining share.

HAMi is the layer that turns a heterogeneous fleet into a Kubernetes workload surface where a single deployment manifest can target a fleet without per-vendor scheduling code. A serving workload that prefers MI300X for memory-bandwidth reasons still gets scheduled correctly even if the cluster also carries H100 nodes — the deployment manifest asks for the partition, the scheduler picks the partition, the workload runs.

What HAMi actually does at scheduling time

The scheduler extender hooks the Kubernetes scheduling framework at the filter and score stages. At filter, it rejects pods whose memory and compute requests cannot be satisfied on any node. At score, it ranks candidate nodes by current utilization, MIG layout, and policy constraints (node affinity, taints, and the per-task-type memory and compute caps configured in the device plugin's configmap).

The resource model is straightforward. A pod that wants 20 GB of VRAM and 50% of an H100 asks for it like this:

resources:
  limits:
    nvidia.com/gpu: 1
    nvidia.com/gpumem: 20000  # MB of VRAM
    nvidia.com/gpucores: 50   # percent of SM throughput

The device plugin sees the nvidia.com/gpumem and nvidia.com/gpucores resource requests, allocates a partition on a physical GPU that satisfies both, and reports back the partition identifier to the scheduler. The pod starts on the partition; the partition is bounded by the requested memory and compute caps, and the scheduler accounts for the partition's capacity until the pod terminates.

This model has three operationally important properties. First, memory oversubscription is explicit: a node that has 80 GB of physical VRAM can host partitions that sum to more than 80 GB only if the device plugin's memAllocPolicy is set to allow it. The default is no-oversubscription, which matches what most teams want on a serving fleet. Second, the partition survives a pod crash and is reaped by the device plugin's cleanup loop on a configurable interval — operationally invisible until a pod is killed mid-flight, then operationally meaningful. Third, the scheduler records partition utilization on a Prometheus endpoint, so a Grafana panel can show "this H100 is at 80% partition utilization, this one is at 30%" — the operational telemetry the GPU monitoring for AI inference guide covers becomes partition-aware.

How HAMi compares to MIG and time-slicing

MIG (Multi-Instance GPU) is a hardware feature on NVIDIA Ampere, Hopper, and Blackwell that splits one GPU into up to seven hardware-isolated instances. Each instance has dedicated SMs, memory controllers, and L2 cache slices. The isolation is stronger than HAMi's: a buggy workload in one MIG instance cannot affect another, and the SM bandwidth is dedicated. The trade-off is flexibility: MIG instance shapes are fixed (1g.5gb, 2g.10gb, 3g.20gb, 4g.20gb, 7g.40gb on H100), and MIG is not available on consumer cards.

Time-slicing is the NVIDIA Device Plugin's round-robin scheduler. Multiple pods share one GPU and the driver round-robins SM time across them. Memory isolation is by nvidia.com/gpumem limits; compute isolation is best-effort. Time-slicing is the right answer when memory isolation matters but compute isolation does not — notebook workloads, low-priority batch jobs, CI runners.

HAMi sits between MIG and time-slicing. It gives you memory isolation (mandatory, bounded by the device plugin config) and soft compute isolation (capped by the device plugin config, but enforced via SM throttling rather than hardware partitioning). It runs on any GPU the NVIDIA Device Plugin supports, on AMD GPUs, and on the non-NVIDIA accelerators the project lists. The cross-vendor and the bounded-memory-oversubscription stories are what HAMi adds; the hardware-isolated SM story is what MIG still owns.

The practical pattern on a multi-tenant H100 cluster is to use MIG for the serving tier (where isolation matters), HAMi for the notebook and fine-tuning tier (where flexibility matters and isolation is a nice-to-have), and time-slicing for the batch and CI tier (where memory isolation is enough and the priority is throughput). The three modes can coexist on the same cluster and even on the same node — the GPU Operator deploys all three device plugins and the scheduler extender chooses at request time.

HAMi and the AWS Trainium / Inferentia angle

The AWS Trainium and Inferentia guide covers the silicon comparison; the angle that matters here is that Trainium2 and Inferentia2 are first-class accelerators in the cost arithmetic but ship with their own scheduler (the AWS Neuron device plugin) and their own Kubernetes integration. The multi-tenant sharing story on Trainium is less mature than on NVIDIA: Neuron exposes one accelerator per pod, and partition-style sharing is a Neuron-core-level primitive that requires manual scheduler code.

HAMi does not currently ship a Neuron device plugin in the upstream project, so a fleet that mixes Trainium with NVIDIA or AMD accelerators still needs the Neuron device plugin plus HAMi for the NVIDIA/AMD side. The practical answer for now is to treat the Neuron fleet as a separate scheduling pool and let HAMi handle the heterogeneous NVIDIA-plus-AMD pool — the architectural separation is clean even if the deployment manifest has to mention two schedulers. If you operate an AMD or NVIDIA-plus-Neuron fleet and want a single scheduler surface, the open question is whether the HAMi project will accept a Neuron device plugin — the maintainer conversation is worth tracking in the project's GitHub repository.

The FinOps side: HAMi and the per-GPU cost arithmetic

The Kubernetes cost optimization guide covers the showback and chargeback arithmetic that HAMi unlocks. With MIG or time-slicing alone, the per-pod cost of GPU time is "some fraction of a GPU that the cost model approximates." With HAMi, the per-pod cost is "20 GB of H100 VRAM for 47 minutes, at $X per GB-hour" — a number the device plugin reports and the FinOps pipeline can consume directly. The granularity is meaningful because it lets a chargeback rollup assign cost to a team based on the partitions they actually used rather than the GPU instances they nominally targeted.

The other FinOps angle is the utilization floor. On a cluster where the default scheduler leaves 30-40% of GPU capacity stranded because pods over-request, HAMi memory oversubscription (with the memAllocPolicy cap) can pull stranded memory into use without the safety risk of unbounded oversubscription. The LLM API cost calculator turns the per-call cost arithmetic into a forward-looking forecast; the same approach applied to per-partition GPU time gives a cluster-level cost-per-token view that the marketing-style benchmarks on NVIDIA's product pages do not.

What to validate before you adopt

If you are evaluating HAMi for a new cluster or planning to migrate an existing GPU-sharing setup, the questions worth answering in a two-week evaluation are:

  1. What is the per-partition utilization profile? Run a mixed workload of fine-tuning jobs (memory-bound) and serving jobs (latency-bound) on a HAMi-enabled node and capture the partition hit rate, the partition waste (memory reserved but not used), and the tail latency on the serving tier. HAMi memory oversubscription is bounded; the question is whether the bound is at the right level for your workload mix.
  2. Does the scheduler extender interact cleanly with your existing scheduler plugins? If you already run KEDA, Karpenter, or a custom scheduler plugin, validate that the HAMi scheduler extender does not regress your existing scheduling decisions. The KEDA autoscaling guide covers the queue-depth scaling path that pairs naturally with HAMi partitions.
  3. What is the failure mode when a partition exceeds its compute cap? HAMi throttles, which is the right answer for compute caps but the wrong answer for memory caps (the kernel OOM-kills). Make sure your monitoring catches the throttling signal before the workload notices, and make sure the partition's memory cap is set with headroom for transient spikes.
  4. What does the migration from a non-HAMi setup look like? Migrating from time-slicing to HAMi is straightforward: both run on top of the NVIDIA Device Plugin, the change is in the resource request shape and the scheduler extender. Migrating from MIG to HAMi is a cluster re-partition operation: pods have to be re-scheduled onto HAMi partitions, and the partition shape is now policy-driven rather than hardware-fixed.
  5. What does the failure mode look like when the device plugin dies? The device plugin runs as a DaemonSet; if it crashes on a node, that node's GPU capacity is no longer advertised to the scheduler and existing pods on the node stay alive but new pods cannot schedule there. The recovery is automatic on the device plugin restart, but the gap between crash and restart is a cluster-capacity loss you should plan for.

Where HAMi sits in the broader CNCF ecosystem

The CNCF projects that touch the GPU and accelerator story cluster around three problems: scheduling, observability, and application runtime. HAMi covers scheduling. The observability layer is split between Prometheus + DCGM for the traditional GPU metrics and OpenTelemetry for the inference-tracing surface (the OpenTelemetry for AI inference tracing guide walks through the latter). The application runtime story is being driven by the agent sandbox work the CNCF SIG Apps is incubating (the agent sandbox vs agent substrate guide covers the two projects). HAMi is upstream of all three — the partition the scheduler picks is what the observability layer instruments and what the application runtime lands on.

For teams that are also looking at the inference engine story (vLLM, SGLang, TGI, TensorRT-LLM), the practical integration pattern is HAMi for the GPU partition, vLLM for the inference serving, and OpenTelemetry for the tracing. The cross-reference in this corpus is the vLLM production monitoring guide for the serving layer and the LLM inference engine comparison for the engine choice.

What I would do this quarter

If you are running a multi-tenant NVIDIA-only cluster today and have not yet adopted HAMi, I would stage a HAMi evaluation on a non-production pool first. The evaluation should answer the five questions above, with explicit before-and-after measurements on partition utilization and tail latency. The upgrade from Sandbox to Incubating means the procurement and security-review path is now routine, so the remaining question is whether the workload behavior matches your assumptions.

If you are running a heterogeneous fleet that includes AMD or Ascend accelerators, I would adopt HAMi as the unified scheduler surface now. The cross-vendor story is the strongest argument for HAMi and the procurement story just got stronger with the Incubating tier.

If you are running a single-tenant cluster where every pod gets a whole GPU, HAMi is not the right starting point — the per-partition overhead is real and the cost arithmetic does not favor partition-style sharing until utilization drops below the per-pod waste threshold. Stay on the default NVIDIA Device Plugin and revisit HAMi when utilization becomes a problem.

The CNCF Incubating tier is not a destination. It is a milestone on the way to Graduated, which would imply a more conservative feature pace and a more aggressive security-audit cadence. For teams that have been waiting for HAMi to "settle" before adopting, the Incubating tier is the right point to start the conversation; for teams that have already adopted HAMi on the Sandbox promise, the tier change is the moment to re-validate the security review against the published audit and update the procurement paperwork.

The HAMi project keeps its project site and GitHub repository as the canonical sources for releases and documentation; the CNCF project page at cncf.io/projects/hami is the canonical source for governance and maturity status. The existing Kubernetes GPU Operator guide and the Kubernetes GPU scheduling for ML workloads guide are the right companion reads if you are evaluating the broader scheduler and operator story; this guide is the deep-dive on the HAMi piece specifically.