Configuring Metrics for the Monitoring Stack

The OpenTelemetry Collector can be integrated with the platform monitoring stack to expose its own operational metrics via Prometheus. As a cluster administrator, you can set up this integration to achieve the following:

  • Automatically generate Prometheus ServiceMonitor resources that scrape both the Collector's internal pipeline metrics and any configured Prometheus exporter endpoints.
  • Use the Prometheus receiver within the Collector to pull metrics from the cluster's built-in monitoring infrastructure.

Sending Metrics to the Monitoring Stack

The OpenTelemetryCollector custom resource (CR) supports automatic creation of Prometheus ServiceMonitor or PodMonitor resources. These resources instruct the monitoring stack to collect metrics from the Collector's internal telemetry endpoint as well as from any Prometheus exporter endpoints defined in the pipeline.

Enabling Automatic ServiceMonitor Creation

When you enable the enableMetrics option in the Collector CR, the Alauda build of OpenTelemetry v2 Operator automatically provisions the necessary ServiceMonitor or PodMonitor resources for metric collection.

The following example shows a Collector CR configured with a Prometheus exporter and automatic metrics collection enabled:

apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  labels:
    prometheus: kube-prometheus
spec:
  mode: deployment
  replicas: 1
  observability:
    metrics:
      enableMetrics: true
  config:
    exporters:
      prometheus:
        endpoint: 0.0.0.0:8889
        resource_to_telemetry_conversion:
          enabled: true # by default resource attributes are dropped
    service:
      telemetry:
        metrics:
          readers:
          - pull:
              exporter:
                prometheus:
                  host: 0.0.0.0
                  port: 8888
                  without_scope_info: true
                  without_type_suffix: true
                  without_units: true
      pipelines:
        metrics:
          exporters: [prometheus]
  1. prometheus=kube-prometheus is required in ACP prometheus.
  2. Enables the Operator to automatically create ServiceMonitor or PodMonitor resources that target the Collector's metrics endpoints, including both internal telemetry and Prometheus exporter ports.
NOTE

Turning on enableMetrics results in two separate ServiceMonitor resources being created:

  • A ServiceMonitor targeting the <instance_name>-collector-monitoring service, which collects the Collector's own internal operational metrics.
  • A ServiceMonitor targeting the <instance_name>-collector service, which collects metrics published by any Prometheus exporter configured in the pipeline.

Using a Custom PodMonitor

For scenarios that require more granular control over metric collection — such as filtering out duplicate labels introduced during Prometheus scraping — you can create a PodMonitor resource manually instead of relying on the automatic ServiceMonitor provisioning.

The following example demonstrates a PodMonitor that targets the Collector pods and applies relabeling rules to drop redundant labels:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  labels:
    prometheus: kube-prometheus
  name: otel-collector
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: <cr_name>-collector
  podMetricsEndpoints:
  - port: metrics
  - port: promexporter
    relabelings:
    - action: labeldrop
      regex: pod
    - action: labeldrop
      regex: container
    - action: labeldrop
      regex: endpoint
    metricRelabelings:
    - action: labeldrop
      regex: instance
    - action: labeldrop
      regex: job
  1. prometheus=kube-prometheus is required in ACP prometheus.
  2. Must match the name of your OpenTelemetry Collector CR, following the pattern <cr_name>-collector.
  3. The port exposing the Collector's internal operational metrics. This port is always named metrics.
  4. The port exposing metrics from the Prometheus exporter configured in the Collector pipeline.