Alauda Build of HAMi

HAMi on Ascend vNPU

Use this scenario when Ascend workloads need fixed-template hard slicing or hami-core soft slicing through HAMi. For whole-card allocation through HAMi, use HAMi on Ascend NPU.

Technical Preview

Alauda Build of HAMi Ascend Device Plugin v1.4.0 is a Technical Preview. Confirm the supported hardware, environment, and known limitations in Compatibility and Release Notes before use.

TOC

Choose hard or soft slicingCheck the effective node modeRun Ascend workloadsRun a hard-slice workloadRun a soft-slice workloadNext steps

Choose hard or soft slicing

RequirementHard slicingSoft slicing
Use whenThe workload fits a predefined vendor vNPU template.The workload needs a software-controlled memory quota and optional compute scheduling weight.
Effective node annotationhami-vnpu-core=falsehami-vnpu-core=true
Workload resourcesCount plus memory.Count plus memory and optional core percentage.
Required Pod annotationNonehuawei.com/vnpu-mode: hami-core
Required RuntimeClassascendascend
Runtime signalASCEND_VNPU_SPECS, such as vir01.NPU_MEM_QUOTA, optional NPU_PRIORITY, and libvnpu.so preload.

Hard and soft slicing are separate node modes. When both modes are required in one cluster, use separate node pools and ensure that each workload selects a node pool configured for its mode.

Check the effective node mode

Check the effective annotation written to the target node:

kubectl get node <node-name> \
  -o go-template='hami-vnpu-core={{ index .metadata.annotations "hami-vnpu-core" }}{{ "\n" }}'

Use the node annotation as the effective mode seen by scheduling:

  • false: the node accepts whole-card requests and partial-memory hard-slice requests;
  • true: the node accepts whole-card requests and partial-memory soft-slice requests that set huawei.com/vnpu-mode: hami-core.

If the node and Pod modes do not match, the Pod can remain Pending with a ModeNotFit scheduling event. Check it with:

kubectl describe pod <pod-name>

See Configure Ascend Slicing Mode to inspect the owning custom resource and ConfigMaps or change the node mode.

Run Ascend workloads

The Pod manifests below use Ascend 310P3 as a concrete example. This hardware model is exposed through the huawei.com/Ascend310P resource family. For an Ascend 910-series device, replace the count, memory, optional core, and allocation-annotation names with the model-specific values from hami-scheduler-device. For hard slicing, also use a template and memory value defined for that model, and update the expected ASCEND_VNPU_SPECS value in the verification command. Replace the image with an ARM64 Ascend image that contains the CANN libraries required by your test application.

Check the CANN runtime before soft-slice acceptance

Ascend Driver 25.5 or later is required, but the Driver version alone does not prove that a workload image is compatible with the injected soft-slice runtime. Review the ABI constraint in Ascend Driver and CANN compatibility, then run a representative CANN operation with the exact workload image.

Run a hard-slice workload

Hard slicing rounds the requested memory up to a predefined template. For Ascend 310P3, the v1.4.0 configuration provides vir01 at 3072 MiB, vir02 at 6144 MiB, and vir04 at 12288 MiB. The following request selects vir01.

Hard-slice limitations in v1.4.0

An allocated hard-slice vNPU can be reclaimed before the workload first accesses it. Use this path only for evaluation, and use whole-card allocation when stable device access is required.

A partial-memory hard-slice request also supports only one device per container. Keep huawei.com/Ascend310P: "1". A request for multiple devices with partial memory is rejected by the admission webhook with vNPU nor supported for multiple devices; the Pod might not be created, so check the kubectl apply error instead of waiting for a scheduling event.

apiVersion: v1
kind: Pod
metadata:
  name: hami-ascend310p-hard-slice
spec:
  schedulerName: hami-scheduler
  runtimeClassName: ascend
  restartPolicy: Never
  containers:
    - name: npu-workload
      image: <your-ascend-image>
      command: ["/bin/sh", "-c"]
      args:
        - |
          env | grep -E 'ASCEND_VISIBLE_DEVICES|ASCEND_VNPU_SPECS'
          sleep 3600
      resources:
        requests:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "3072"
        limits:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "3072"

Do not add huawei.com/vnpu-mode: hami-core or a core resource to a hard-slice workload.

Create and inspect the Pod:

kubectl apply -f hami-ascend310p-hard-slice.yaml
kubectl get pod hami-ascend310p-hard-slice -o wide
kubectl get pod hami-ascend310p-hard-slice \
  -o go-template='scheduler={{ .spec.schedulerName }}{{ "\n" }}runtimeClass={{ .spec.runtimeClassName }}{{ "\n" }}allocation={{ index .metadata.annotations "hami.io/Ascend310P-devices-allocated" }}{{ "\n" }}'
kubectl exec hami-ascend310p-hard-slice -- /bin/sh -c '
  test -n "$ASCEND_VISIBLE_DEVICES" &&
  test "$ASCEND_VNPU_SPECS" = vir01 &&
  test -z "$NPU_MEM_QUOTA"
'

The expected result is a non-empty HAMi allocation annotation and ASCEND_VNPU_SPECS=vir01. These signals confirm selection of the hard-slice runtime path. Pod Running and environment variables alone do not prove usable device access; run a representative CANN device-open or inference workload before accepting the environment.

Run a soft-slice workload

Soft slicing requires all of the following settings:

  • the target node has hami-vnpu-core=true;
  • the Pod has huawei.com/vnpu-mode: hami-core;
  • the Pod sets runtimeClassName: ascend;
  • the workload requests count plus memory and, optionally, the model-specific core resource;
  • the workload image provides bash, which the injected PostStart hook uses to start the limiter.

The following Ascend 310P3 example requests 4096 MiB and supplies a core value of 1:

apiVersion: v1
kind: Pod
metadata:
  name: hami-ascend310p-soft-slice
  annotations:
    huawei.com/vnpu-mode: hami-core
spec:
  schedulerName: hami-scheduler
  runtimeClassName: ascend
  restartPolicy: Never
  containers:
    - name: npu-workload
      image: <your-ascend-image>
      command: ["/bin/sh", "-c"]
      args:
        - |
          env | grep -E 'ASCEND_VISIBLE_DEVICES|NPU_MEM_QUOTA|NPU_PRIORITY'
          grep -F /hami-vnpu-core/libvnpu.so /etc/ld.so.preload
          sleep 3600
      resources:
        requests:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "4096"
          huawei.com/Ascend310P-core: "1"
        limits:
          huawei.com/Ascend310P: "1"
          huawei.com/Ascend310P-memory: "4096"
          huawei.com/Ascend310P-core: "1"

Create and inspect the Pod:

kubectl apply -f hami-ascend310p-soft-slice.yaml
kubectl get pod hami-ascend310p-soft-slice -o wide
kubectl get pod hami-ascend310p-soft-slice \
  -o go-template='scheduler={{ .spec.schedulerName }}{{ "\n" }}runtimeClass={{ .spec.runtimeClassName }}{{ "\n" }}mode={{ index .metadata.annotations "huawei.com/vnpu-mode" }}{{ "\n" }}allocation={{ index .metadata.annotations "hami.io/Ascend310P-devices-allocated" }}{{ "\n" }}'
kubectl get pod hami-ascend310p-soft-slice \
  -o jsonpath='{.spec.containers[0].lifecycle.postStart.exec.command}{"\n"}'
kubectl exec hami-ascend310p-soft-slice -- /bin/sh -c '
  test -n "$ASCEND_VISIBLE_DEVICES" &&
  test "$NPU_MEM_QUOTA" = 4096 &&
  test "$NPU_PRIORITY" = 1 &&
  grep -F /hami-vnpu-core/libvnpu.so /etc/ld.so.preload
'

Interpret the results as follows:

  • mode=hami-core and a non-empty allocation annotation confirm HAMi soft-slice scheduling and allocation.
  • The injected PostStart hook and libvnpu.so preload confirm selection of the HAMi vNPU runtime path.
  • NPU_MEM_QUOTA=4096 is the requested software memory quota.
  • The optional core resource is expressed as a percentage by the scheduler and injected as NPU_PRIORITY. It controls relative compute scheduling weight; it is not dedicated hardware AI Core isolation.

These checks prove selection of the soft-slice runtime path, but Pod Running is not an end-to-end functional test. Run a CANN memory-allocation or inference workload and confirm that it can access the assigned device and respects the requested quota.

For another chip model, obtain the resource keys and allocation-annotation suffix from hami-scheduler-device, and confirm the count capacity in node allocatable resources before creating the workload.

Next steps