#How to Define a Whole GPU Cost Model

This procedure registers NVIDIA whole-GPU allocation and project-quota usage as billing items. It does not measure GPU utilization percentage.

#What this configuration provides

• Allocation: one allocated GPU UUID is counted as one GPU, then accumulated as GPU-hours.
• Project quota (optional): the project's requests.nvidia.com/gpu hard quota can be accumulated as GPU-hours when project-quota billing is required.
• Model pricing: allocation records retain modelName, so different GPU models can have different prices.
• Project-quota pricing: quota records do not contain a GPU model, so use one project-level base price.

#1. Create the recording rule

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: alauda-gpu-billing-recording
  namespace: kube-system
  labels:
    prometheus: kube-prometheus
spec:
  groups:
    - name: alauda.billing.gpu
      interval: 30s
      rules:
        - record: alauda_gpu_allocated_count
          expr: |
            count by (UUID, label_modelName, namespace) (
              label_replace(
                DCGM_FI_DEV_GPU_UTIL{
                  namespace!="kube-system",
                  pod!~"nvidia-dcgm-exporter.*",
                  container!="nvidia-dcgm-exporter"
                },
                "label_modelName",
                "$0",
                "modelName",
                ".*"
              )
            )
        - record: alauda_gpu_project_quota_count
          expr: |
            avg by (project, cluster) (
              avg_over_time(
                cpaas_project_resourcequota{
                  resource="requests.nvidia.com/gpu",
                  type="project-hard"
                }[5m]
              )
            )

The first rule emits one value-1 series for each allocated GPU UUID. The second rule emits the project's GPU quota. The second rule is optional: remove it and the matching project-quota collection and display entries when project-quota billing is not required. Replace the source expressions if the exporter or quota metric has different names in your environment.

Verify the rules in Prometheus:

alauda_gpu_allocated_count
alauda_gpu_project_quota_count

Both queries must return the expected non-empty series before continuing.

Confirm that the GPU exporter target is healthy in the active Prometheus instance before troubleshooting the recording rule.

#2. Create the collection configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: slark-agent-gpu-config
  namespace: cpaas-system
  labels:
    cpaas.io/slark.collection.config: "true"
data:
  config: |
    - kind: GPU
      category: GPU
      item: gpuAllocationHours
      period: Hourly
      usage:
        query: alauda_gpu_allocated_count
        step: 5m
        mappers:
          name: UUID
          namespace: namespace
          cluster: ""
          project: ""
      labels:
        query: alauda_gpu_allocated_count
        mappers:
          name: UUID
          namespace: namespace
          cluster: ""
          project: ""
    - kind: Project
      category: GPU
      item: projectGPUQuotaHours
      period: Hourly
      usage:
        query: alauda_gpu_project_quota_count
        step: 5m
        mappers:
          name: project
          namespace: ""
          cluster: cluster
          project: project

The label_ prefix is required in the Prometheus label source. Cost Management stores label_modelName as the billing label modelName.

After applying the ConfigMap, reload the Cost Management Agent so it reads the new collection configuration:

kubectl -n cpaas-system delete pod \
  -l service_name=slark-agent

#3. Create the display configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: slark-display-config-for-gpu
  namespace: kube-public
  labels:
    cpaas.io/slark.display.config: "true"
data:
  config: |
    - name: GPU
      displayname:
        zh: "整卡 GPU"
        en: "Whole GPU"
      methods:
        - name: Allocation
          displayname:
            zh: "分配量"
            en: "Allocation Usage"
          item: gpuAllocationHours
          divisor: 1
          unit:
            zh: "卡时"
            en: "GPU-hours"
        - name: ProjectQuota
          displayname:
            zh: "项目配额"
            en: "Project Quota"
          item: projectGPUQuotaHours
          divisor: 1
          unit:
            zh: "卡时"
            en: "GPU-hours"

After applying the ConfigMap, reload the Cost Management Server so it reads the new display configuration:

kubectl -n cpaas-system delete pod \
  -l service_name=slark-server

#4. Add prices to the cost model

In the platform console, open Metering and Billing > Cost Models and create or edit the cost model linked to the GPU cluster.

Add the following billing items:

For model-specific allocation prices, add a label override:

Key: modelName
Value: <the exact value reported by the GPU exporter>
Price: <price per GPU-hour>

The label value is case-sensitive. Confirm it with:

count by (UUID, label_modelName) (alauda_gpu_allocated_count)

#5. Verify the result

Wait for at least one hourly billing window and allow the collection and billing processes to finish. Verify the result in this order:

1. Usage: confirm that the GPU usage record exists for the expected cluster, namespace, and GPU UUID.
2. Bill: confirm that a bill is generated from that usage record and that the expected price is applied.
3. Console: confirm that the same item and amount appear in cost details and statistics.

Confirm that:

• Allocation Usage contains the allocated GPU UUID and, when available, modelName;
• Project Quota contains the project and cluster dimensions;
• the expected model-specific price is applied to allocation records;
• project-quota records use the project-level base price.

For common issues, see FAQ.