Verification
Use this page to confirm that Alauda Build of NPU Operator installed the NPU stack and that workloads can consume NPU resources.
TOC
Verify operator statusVerify node resourcesVerify driver readinessVerify RuntimeClass integrationVerify a workloadVerify monitoringVerify operator status
Check the NPUOperatorCtl instance:
The Deployed condition should be True. A newly installed NPUOperatorCtl reports reason InstallSuccessful; after a successful Helm release upgrade, it reports UpgradeSuccessful.
Check component pods in their deployed namespaces:
With the default installation, the Operator, Driver, Runtime, and NPU Feature Discovery run in npu-operator; these resources follow the Operator installation namespace if you selected another one. The Ascend Device Plugin runs in kube-system, NPU Exporter runs in npu-exporter, optional MindCluster components run in mindx-dl, and optional MindIO DaemonSets run in default. Pods should be Running when their corresponding components are enabled; an empty optional component result is expected when those components are disabled.
Verify node resources
Check node allocatable resources:
Expected examples:
The exact key and value depend on the chip model, device plugin version, and card count. A 910-family node, including a 910B node, may report huawei.com/Ascend910. Use the resource key that appears in the target node allocatable resources.
Verify driver readiness
Check that the driver DaemonSet is ready on NPU nodes:
When runtime driver tree mode is enabled, the driver pod writes the ready gate on the host at /run/ascend/.ready/driver-ready and exposes the runtime driver tree from /run/ascend/driver.
On each selected NPU node, verify that the persisted runtime mode agrees with
the mode selected in NPUOperatorCtl. Run the first command through the
approved node-debug or host administration path:
The file should report installMode=host in scenarios 1 and 2.1, and
installMode=containerized in scenario 2.2. Its driverRoot should also match
the selected scenario (/usr/local/Ascend/driver or the configured
spec.driver.driverInstallDir). The runtime script rejects an
installMode mismatch by design; treat a driverRoot mismatch as a deployment
configuration error and correct it before running workloads. For either case,
follow Driver mode change is refused before proceeding.
For Ascend 910B, driver initialization can take several minutes after a node reboot because the chip PHY startup is slower than 310P. Wait until the driver pod is Ready before checking workload injection.
Verify RuntimeClass integration
Check the RuntimeClass and runtime integration DaemonSet on the target workload cluster:
The RuntimeClass name and handler should both be ascend. Keep the delivered operator.runtimeClass value at ascend in v26.6.0 so that it matches the containerd handler registered by the runtime integration DaemonSet. The DaemonSet installs the ascend-docker-runtime payload and maintains the corresponding containerd handler configuration. In precompiled driver mode, the host-side runtime Driver tree is staged under /run/ascend/driver. In pre-installed Driver and ordinary OS managed install modes, the runtime uses the Host Driver under /usr/local/Ascend/driver.
If the RuntimeClass exists but a workload fails before runtime setup, follow RuntimeClass exists but the containerd handler is missing. A node upgrade can remove the handler while leaving the RuntimeClass object unchanged.
Verify a workload
Run a small workload image on an NPU node. Use the resource key that the node actually advertises. Different Ascend fleets may report different huawei.com/Ascend* keys, so check node allocatable resources first instead of hard-coding one key for every cluster.
Replace the image with an internal image if your cluster is air-gapped or image whitelist policies are enforced.
With the default admission webhook enabled, do not add runtimeClassName to this example. The webhook adds runtimeClassName: ascend after it detects the Ascend resource request. If the webhook is disabled, set runtimeClassName: ascend explicitly.
Apply the YAML, then check logs:
The admitted Pod should use the ascend RuntimeClass. Device nodes and
npu-smi info prove device visibility only. Before accepting the node, run an
ACL probe that initializes the device, allocates and frees memory, resets the
device, and finalizes successfully, followed by the smallest supported NPU
application operation. Delete the Pod and confirm that another workload can
obtain the device. The workload image must provide CANN, the framework, and
the probe or application; the runtime supplies the device and Driver files.
npu-smi is available at /usr/local/bin/npu-smi when the runtime path is
working, and Driver libraries are mounted under /usr/local/Ascend/.... In
precompiled Driver mode, the host-side source path for the runtime Driver tree
is /run/ascend/driver.
Verify monitoring
If NPU Exporter is enabled, the operator creates its ServiceMonitor in the npu-exporter namespace:
Use ACP monitoring pages to manage dashboards. For dashboard operations, see Manage dashboards.