Configure Project-Level HAMi Quota Metrics

HAMi exposes quota usage with the namespace label quotanamespace. If a dashboard or alert needs project-level usage, add a recording rule that joins the HAMi scheduler metric with kube_namespace_labels and records QuotaUsedByProject.

This recording rule is for monitoring and reporting. It is not required for HAMi scheduling or quota enforcement. Apply it only when the monitoring or dashboard package used by your platform consumes QuotaUsedByProject.

Before you begin

  • HAMi scheduler metrics are scraped by the target Prometheus or VictoriaMetrics instance.
  • kube_namespace_labels from kube-state-metrics is scraped by the same monitoring instance.
  • You have permission to create a monitoring rule in the workload cluster.

VictoriaMetrics-compatible monitoring

Use the vmcluster label when the monitoring stack exposes it:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: hami-project-quota
  namespace: kube-system
  labels:
    app.kubernetes.io/component: hami-scheduler
    prometheus: kube-prometheus
spec:
  groups:
    - name: quotaused-project-quotaname.rules
      interval: 30s
      rules:
        - record: QuotaUsedByProject
          expr: |
            sum by (quotaName, project, vmcluster) (
              QuotaUsed
              * on(quotanamespace, vmcluster) group_left(project)
                label_replace(kube_namespace_labels, "quotanamespace", "$1", "namespace", "(.*)")
            )
          labels:
            description: QuotaUsed metric aggregated by project and quotaname

Apply the rule in the workload cluster:

kubectl apply -f hami-project-quota-prometheusrule.yaml

Prometheus monitoring

Use this variant when the monitoring stack does not expose vmcluster:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: hami-project-quota
  namespace: kube-system
  labels:
    app.kubernetes.io/component: hami-scheduler
    prometheus: kube-prometheus
spec:
  groups:
    - name: quotaused-project-quotaname.rules
      interval: 30s
      rules:
        - record: QuotaUsedByProject
          expr: |
            sum by (quotaName, project) (
              QuotaUsed
              * on(quotanamespace) group_left(project)
                label_replace(kube_namespace_labels, "quotanamespace", "$1", "namespace", "(.*)")
            )
          labels:
            description: QuotaUsed metric aggregated by project and quotaname

Apply the rule and verify the recorded series:

kubectl apply -f hami-project-quota-prometheusrule.yaml

Query QuotaUsedByProject from the same monitoring backend. If the series is empty, first check that the scheduler exposes QuotaUsed, that kube_namespace_labels contains the expected project label, and that both metrics are scraped by the same backend.

ACP quota metadata is documented separately

The ConfigMaps that make HAMi resources selectable in ACP project and namespace quota pages are platform metadata, not recording rules. Follow

ACP Accelerator Resource Quota

for those ConfigMaps and their resource-key fields.