Monitoring Ceph RGW with the Ceph RGW Exporter
Alauda Storage Exporter is an Operator that collects usage, bucket, and user metrics from the Ceph RGW (RADOS Gateway) Admin Ops API and exposes them in Prometheus format. Rook's built-in monitoring covers the health of the Ceph cluster itself, but not object-storage tenants: how much data each user stores, how many objects a bucket holds, or whether a quota is about to be exhausted. This Operator fills that gap.
Once installed, the Operator watches every CephObjectStore in the cluster and automatically deploys one exporter per object store, together with a read-only RGW user and a ServiceMonitor. No credentials need to be handled manually.
TOC
TerminologyPrerequisitesHow it worksProcedureCustomizing an exporterMetrics referenceTroubleshootingUninstallingTerminology
Prerequisites
- Alauda Storage is installed and at least one
CephObjectStoreis in theReadyphase. See Ceph Distributed Storage. - The Alauda Storage Exporter Operator package is available in OperatorHub. On air-gapped clusters, upload it first as described in Upload Packages.
- To view the metrics in the platform, the monitoring plugin (Prometheus or VictoriaMetrics) is installed on the cluster. See Monitoring Installation. The exporter works without it, but nothing scrapes the metrics.
How it works
- The Operator watches
CephObjectStoreresources. For each object store<store>it creates, in the same namespace:- a
CephObjectStoreUsernamedrgw-exporter-user-<store>with read-onlybucket,user,metadata, andusagecapabilities; - a
CephRGWExporternamedrgw-exporter-<store>, annotated withceph.alauda.io/auto-created: "true".
- a
- For each
CephRGWExporter, the Operator reconciles a Deployment, a Service (port9242, namedhttp-metrics), a ServiceAccount with a namespace-scoped Role, and aServiceMonitor, all sharing the CR's name. - At startup the exporter resolves the referenced
CephObjectStoreUser, readsEndpoint,AccessKey, andSecretKeyfrom its Secret, and starts polling the RGW Admin Ops API. The Operator itself never reads the keys. - When a
CephObjectStoreis deleted, the auto-createdCephRGWExporterandCephObjectStoreUserare removed with it. Resources you created manually are left untouched.
Procedure
Install the Operator
-
Go to Administrator.
-
In the left navigation bar, click Marketplace > OperatorHub.
-
Find Alauda Storage Exporter and click Install. Select the
alphachannel, install it into therook-cephnamespace (the namespace where Rook-Ceph runs), and set the upgrade approval to Manual. For a detailed description of the installation dialog, refer to Operator.Alternatively, apply a
Subscriptionfrom the command line on the control plane node of the cluster. Therook-cephnamespace already contains anOperatorGroupcreated by Rook-Ceph, so no new one is needed.Approve the generated
InstallPlan: -
Verify that the Operator is running.
Expected output:
Verify the auto-discovered exporters
Within about a minute of the Operator starting, one exporter exists for each CephObjectStore.
-
List the exporters.
Expected output (
ceph-cluster-objectis the object store name in this example):PHASEisPendingwhile the exporter Pod starts,Runningwhen it is ready, andFailedif the referencedCephObjectStoreUseror its Secret cannot be resolved. Details are recorded instatus.conditions. -
Confirm the read-only RGW user that was provisioned for the exporter.
Expected output:
-
Confirm the child resources.
Verify the metrics
-
Forward the exporter Service to a local port and request the metrics endpoint.
Expected output:
Bucket metrics (
radosgw_usage_bucket_*) only appear once at least one bucket exists in the object store. -
If the monitoring plugin is installed, the
ServiceMonitoris discovered automatically: it carries theprometheus: kube-prometheuslabel that the platform uses to select scrape targets, and scrapes thehttp-metricsport every300sby default. After one scrape interval, queryradosgw_usage_scrape_duration_secondsin the platform's monitoring dashboards or through PromQL to confirm the data is being ingested. See Management of Metrics for how to work with custom metrics.
Customizing an exporter
The auto-created CephRGWExporter uses default settings. You can edit its spec in place, or create your own CephRGWExporter (any name that is not rgw-exporter-<store>) when you need a different configuration. Manually created resources are never modified or deleted by the auto-discovery logic.
Parameters
Metrics reference
All metrics carry the store label. Usage and bucket metrics additionally carry bucket and owner, plus one label per key in tagList.
A quota size of -1 means the quota is unlimited.
Example queries:
Troubleshooting
Uninstalling
Delete the Operator from Marketplace > OperatorHub, or delete the Subscription and ClusterServiceVersion from the rook-ceph namespace. Exporter Deployments, Services, and ServiceMonitor objects are owned by their CephRGWExporter and are garbage-collected when the custom resources are deleted. The auto-created RGW users are not removed automatically; delete them explicitly if they are no longer needed:
Ceph is a trademark or registered trademark of Red Hat, Inc. or its subsidiaries in the United States and other countries. Rook® is a registered trademark of The Linux Foundation. Alauda is an independent vendor. This product is not affiliated with, endorsed by, or sponsored by LF Projects, LLC or The Linux Foundation. All trademarks are the property of their respective owners and are used here for identification purposes only.