Alauda Build of HAMi

HAMi on NVIDIA GPU

Use this scenario when NVIDIA workloads need HAMi scheduling, whole-GPU allocation, memory or compute sharing, or HAMi-managed dynamic MIG. For direct NVIDIA GPU allocation without HAMi, use the NVIDIA GPU documentation instead.

TOC

Before you create a workloadUnderstand the resource requestRun a shared-GPU workloadNext steps

Before you create a workload

Confirm that:

  • Alauda Build of HAMi is installed with Enable NVIDIA enabled;
  • the target node is selected by the HAMi NVIDIA node label, commonly gpu=on;
  • HAMi is the only Device Plugin owner for the physical GPUs in that node pool;
  • the node reports the HAMi NVIDIA resource keys delivered by the installed package.

Run the following checks:

kubectl get ds -A -o wide \
  | grep -E 'hami-device-plugin|nvidia-device-plugin'
kubectl get node <node-name> \
  -o jsonpath='{.status.allocatable}'

The target node must report nvidia.com/gpualloc and the optional memory or compute keys required by the workload. If the same node also exposes direct-allocation resources from another NVIDIA Device Plugin, resolve the Device Plugin ownership conflict before continuing.

Understand the resource request

The example below uses nvidia.com/gpualloc, nvidia.com/gpucores, and nvidia.com/gpumem. See Resource Keys for their meanings, units, and the alternative percentage-based memory key. Do not use the absolute and percentage-based memory keys in the same container.

Run a shared-GPU workload

The following Pod manifest requests one physical GPU as the allocation base, 4096 MiB of GPU memory, and 50 percent of its compute capacity. Replace the image with a CUDA image that contains nvidia-smi and your test application.

apiVersion: v1
kind: Pod
metadata:
  name: hami-nvidia-share
spec:
  schedulerName: hami-scheduler
  restartPolicy: Never
  containers:
    - name: cuda-workload
      image: <your-cuda-image>
      command: ["/bin/sh", "-c"]
      args:
        - |
          env | grep -E 'CUDA_DEVICE_MEMORY_LIMIT|CUDA_DEVICE_SM_LIMIT'
          grep -F libvgpu.so /etc/ld.so.preload
          nvidia-smi
          sleep 3600
      resources:
        requests:
          nvidia.com/gpualloc: "1"
          nvidia.com/gpucores: "50"
          nvidia.com/gpumem: "4096"
        limits:
          nvidia.com/gpualloc: "1"
          nvidia.com/gpucores: "50"
          nvidia.com/gpumem: "4096"

Create and inspect the Pod:

kubectl apply -f hami-nvidia-share.yaml
kubectl get pod hami-nvidia-share -o wide
kubectl get pod hami-nvidia-share \
  -o go-template='scheduler={{ .spec.schedulerName }}{{ "\n" }}allocation={{ index .metadata.annotations "hami.io/vgpu-devices-allocated" }}{{ "\n" }}'
kubectl exec hami-nvidia-share -- /bin/sh -c '
  test "$CUDA_DEVICE_MEMORY_LIMIT_0" = 4096m &&
  test "$CUDA_DEVICE_SM_LIMIT" = 50 &&
  grep -F libvgpu.so /etc/ld.so.preload &&
  nvidia-smi
'

Interpret the results as follows:

  • scheduler=hami-scheduler confirms that the workload used the HAMi scheduler.
  • A non-empty hami.io/vgpu-devices-allocated annotation confirms that HAMi assigned a physical GPU and sharing quota to the container.
  • CUDA_DEVICE_MEMORY_LIMIT_0=4096m, CUDA_DEVICE_SM_LIMIT=50, and the libvgpu.so preload entry confirm that the HAMi-Core sharing runtime was injected.
  • nvidia-smi confirms basic device visibility, but it does not prove enforcement by itself. Run a CUDA memory-allocation or compute workload for end-to-end quota verification.

If only nvidia.com/gpualloc is requested, HAMi allocates a whole GPU through the same HAMi-managed path. For HAMi-managed dynamic MIG, continue with Configure Dynamic MIG.

Next steps