Choose an Ascend NPU Allocation Mode
Choose which Device Plugin will expose and allocate the Ascend NPUs in the complete scope managed by each NPUOperatorCtl instance before you install or configure workload resources.
Use direct whole-card allocation through the NPU Operator Ascend Device Plugin by default when workloads need exclusive access to one or more NPUs and do not need HAMi scheduling, sharing, or slicing. Evaluate HAMi when improving overall device utilization or using HAMi scheduling and resource management is more important than keeping the simplest direct-allocation path.
TOC
Compare the allocation modesConsider performance and resource utilizationPrepare the Ascend base for HAMiContinue with the selected modeCompare the allocation modes
If there is no clear requirement for HAMi scheduling, sharing, or slicing, keep direct whole-card allocation. Do not introduce HAMi only because a workload uses Ascend hardware.
Consider performance and resource utilization
Direct whole-card allocation provides a simpler device allocation path without HAMi sharing or virtualization controls. Choose it when workload performance, latency stability, or exclusive device access has priority over sharing unused capacity.
HAMi adds scheduling, sharing, virtualization, and resource-control capabilities. These mechanisms can introduce performance overhead, especially when workloads share or use slices of the same physical device. The actual impact depends on the hardware, allocation mode, workload characteristics, concurrency, Driver, CANN, and runtime environment.
Choose HAMi when the expected improvement in overall device utilization or the need for HAMi resource management is more important than maintaining the simplest direct-allocation path. Before production use, compare direct allocation and the selected HAMi mode with representative workloads in the target environment. Measure the application-level throughput and latency that define acceptance for the service; Pod scheduling and basic device access do not establish performance suitability.
The HAMi community publishes a performance benchmark that illustrates the possible overhead of HAMi resource control in a specific environment. The benchmark uses a particular NVIDIA GPU, workload, and configuration. Do not treat it as a fixed loss estimate for Ascend NPUs. Validate the selected Ascend allocation mode with the actual hardware and workloads before production use.
Prepare the Ascend base for HAMi
NPU Operator remains responsible for the Ascend base when HAMi manages allocation. Keep the NPU Operator-managed Driver, OCI Runtime, ascend RuntimeClass, and any required NPU Exporter components. Disable the NPU Operator Ascend Device Plugin for the complete NPUOperatorCtl scope that HAMi will manage.
HAMi-managed allocation adds HAMi scheduling and device-management components. Continue with the HAMi documentation for supported environments, installation, configuration, and known limitations.
Only one Device Plugin can expose and allocate the same physical NPUs in a managed scope. Do not run the NPU Operator Ascend Device Plugin and HAMi Ascend Device Plugin against the same devices or rely on resource-key differences to separate them.
Treat the switch as applying to the complete NPUOperatorCtl Device Plugin ownership scope. Use both paths in one cluster only when the installed product versions explicitly provide and validate mutually exclusive Device Plugin placement. Changing the owner of an existing scope is a maintenance operation and does not migrate running workloads.
Continue with the selected mode
- For NPU Operator direct whole-card allocation, continue with Request NPU Resources, Configure Workloads, and Verification.
- For HAMi-managed whole-card or vNPU allocation, continue with Choose a HAMi Deployment Scenario. Follow the HAMi documentation for installation, Device Plugin owner switching, resource requests, slicing behavior, compatibility, and troubleshooting.