HAMi on Ascend vNPU

Use this scenario when Ascend workloads need fixed-template hard slicing or hami-core soft slicing through HAMi. For whole-card allocation through HAMi, use HAMi on Ascend NPU.

WARNING

Confirm the supported hardware, environment, and known limitations for Alauda Build of HAMi Ascend Device Plugin v1.4.1 in Compatibility and Release Notes before use.

Choose hard or soft slicing

RequirementHard slicingSoft slicing
Use whenThe workload fits a predefined vendor vNPU template.The workload needs a software-controlled memory quota and optional compute scheduling weight.
Effective node annotationhami-vnpu-core=falsehami-vnpu-core=true
Workload resourcesCount plus memory.Count plus memory and optional core percentage.
Required Pod annotationNonehuawei.com/vnpu-mode: hami-core
Required RuntimeClassascendascend
Runtime signalASCEND_VNPU_SPECS, such as vir01.NPU_MEM_QUOTA, optional NPU_PRIORITY, and libvnpu.so preload.

Hard and soft slicing are separate node modes. When both modes are required in one cluster, use separate node pools and ensure that each workload selects a node pool configured for its mode.

Check the effective node mode

Check the effective annotation written to the target node:

kubectl get node <node-name> \
  -o go-template='hami-vnpu-core={{ index .metadata.annotations "hami-vnpu-core" }}{{ "\n" }}'

Use the node annotation as the effective mode seen by scheduling:

  • false: the node accepts whole-card requests and partial-memory hard-slice requests;
  • true: the node accepts whole-card requests and partial-memory soft-slice requests that set huawei.com/vnpu-mode: hami-core.

If the node and Pod modes do not match, the Pod can remain Pending with a ModeNotFit scheduling event. Check it with:

kubectl describe pod <pod-name>

See Configure Ascend Slicing Mode to inspect the owning custom resource and ConfigMaps or change the node mode.

Run Ascend workloads

The Pod manifests below use Ascend 310P3 as a concrete example. This hardware model is exposed through the huawei.com/Ascend310P resource family. For an Ascend 910-series device, replace the count, memory, optional core, and allocation-annotation names with the model-specific values from hami-scheduler-device. For hard slicing, also use a template and memory value defined for that model, and update the expected ASCEND_VNPU_SPECS value in the verification command. Replace the image with an ARM64 Ascend image that contains the CANN libraries required by your test application.

Check the CANN runtime before soft-slice acceptance

Ascend Driver 25.5 or later is required, but the Driver version alone does not prove that a workload image is compatible with the injected soft-slice runtime. Review the ABI constraint in Ascend Driver and CANN compatibility, then run a representative CANN operation with the exact workload image.

RuntimeClass injection

The examples set runtimeClassName: ascend explicitly. You may omit it only when the NPU Operator webhook or HAMi Ascend scheduler is configured to inject ascend; use one injector and verify the admitted Pod.

Run a hard-slice workload

Hard slicing rounds the requested memory up to a predefined template. For Ascend 310P3, the v1.4.1 configuration provides vir01 at 3072 MiB, vir02 at 6144 MiB, and vir04 at 12288 MiB. The following request selects vir01.

Hard-slice limitations in v1.4.1

An allocated hard-slice vNPU can be reclaimed before the workload first accesses it. Use this path only for evaluation, and use whole-card allocation when stable device access is required.

A partial-memory hard-slice request also supports only one device per container. Keep huawei.com/Ascend310P: "1". A request for multiple devices with partial memory is rejected by the admission webhook with vNPU not supported for multiple devices; the Pod might not be created, so check the kubectl apply error instead of waiting for a scheduling event.

apiVersion: v1
kind: Pod
metadata:
  name: hami-ascend310p-hard-slice
spec:
  schedulerName: hami-scheduler
  runtimeClassName: ascend
  restartPolicy: Never
  containers:
    - name: npu-workload
      image: <your-ascend-image>
      command: ["/bin/sh", "-c"]
      args:
        - |
          env | grep -E 'ASCEND_VISIBLE_DEVICES|ASCEND_VNPU_SPECS'
          sleep 3600
      resources:
        requests:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "3072"
        limits:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "3072"

Do not add huawei.com/vnpu-mode: hami-core or a core resource to a hard-slice workload.

Create and inspect the Pod:

kubectl apply -f hami-ascend310p-hard-slice.yaml
kubectl get pod hami-ascend310p-hard-slice -o wide
kubectl get pod hami-ascend310p-hard-slice \
  -o go-template='scheduler={{ .spec.schedulerName }}{{ "\n" }}runtimeClass={{ .spec.runtimeClassName }}{{ "\n" }}allocation={{ index .metadata.annotations "hami.io/Ascend310P-devices-allocated" }}{{ "\n" }}'
kubectl exec hami-ascend310p-hard-slice -- /bin/sh -c '
  test -n "$ASCEND_VISIBLE_DEVICES" &&
  test "$ASCEND_VNPU_SPECS" = vir01 &&
  test -z "$NPU_MEM_QUOTA"
'

The expected result is a non-empty HAMi allocation annotation and ASCEND_VNPU_SPECS=vir01. These signals confirm selection of the hard-slice runtime path. Pod Running and environment variables alone do not prove usable device access; run a representative CANN device-open or inference workload before accepting the environment.

Run a soft-slice workload

Soft slicing requires all of the following settings:

  • the target node has hami-vnpu-core=true;
  • the Pod has huawei.com/vnpu-mode: hami-core;
  • the Pod sets runtimeClassName: ascend;
  • the workload requests count plus memory and, optionally, the model-specific core resource;
  • the workload image contains the CANN runtime and the tools needed by the application.

The following Ascend 310P3 example requests 4096 MiB and supplies a core value of 1:

apiVersion: v1
kind: Pod
metadata:
  name: hami-ascend310p-soft-slice
  annotations:
    huawei.com/vnpu-mode: hami-core
spec:
  schedulerName: hami-scheduler
  runtimeClassName: ascend
  restartPolicy: Never
  containers:
    - name: npu-workload
      image: <your-ascend-image>
      command: ["/bin/sh", "-c"]
      args:
        - |
          env | grep -E 'ASCEND_VISIBLE_DEVICES|NPU_MEM_QUOTA|NPU_PRIORITY'
          grep -F /hami-vnpu-core/libvnpu.so /etc/ld.so.preload
          sleep 3600
      resources:
        requests:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "4096"
          huawei.com/Ascend310P-core: "1"
        limits:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "4096"
          huawei.com/Ascend310P-core: "1"

Create and inspect the Pod:

kubectl apply -f hami-ascend310p-soft-slice.yaml
kubectl get pod hami-ascend310p-soft-slice -o wide
kubectl get pod hami-ascend310p-soft-slice \
  -o go-template='scheduler={{ .spec.schedulerName }}{{ "\n" }}runtimeClass={{ .spec.runtimeClassName }}{{ "\n" }}mode={{ index .metadata.annotations "huawei.com/vnpu-mode" }}{{ "\n" }}allocation={{ index .metadata.annotations "hami.io/Ascend310P-devices-allocated" }}{{ "\n" }}'
kubectl exec hami-ascend310p-soft-slice -- /bin/sh -c '
  test -n "$ASCEND_VISIBLE_DEVICES" &&
  test "$NPU_MEM_QUOTA" = 4096 &&
  test "$NPU_PRIORITY" = 1 &&
  grep -F /hami-vnpu-core/libvnpu.so /etc/ld.so.preload
'

Interpret the results as follows:

  • mode=hami-core and a non-empty allocation annotation confirm HAMi soft-slice scheduling and allocation.
  • The libvnpu.so preload entry confirms selection of the HAMi vNPU runtime path. This release does not inject a PostStart hook.
  • NPU_MEM_QUOTA=4096 is the requested software memory quota.
  • The optional core resource is expressed as a percentage by the scheduler and injected as NPU_PRIORITY. It controls relative compute scheduling weight; it is not dedicated hardware AI Core isolation.

These checks prove selection of the soft-slice runtime path, but Pod Running is not an end-to-end functional test. Run a CANN memory-allocation or inference workload and confirm that it can access the assigned device and respects the requested quota.

For another chip model, obtain the resource keys and allocation-annotation suffix from hami-scheduler-device, and confirm the count capacity in node allocatable resources before creating the workload.

Next steps