#Installation

This article will introduce how to install the components required for metering and billing.

#Preinstall Considerations

• Download the latest version of the Cost Management Plugin from Custom Portal and upload it to the platform by violet.

• The Cost Management Server, Cost Management Essentials and Cost Management Agent work together to provide the overall metering and billing functionality of the platform. The server-side component supports data storage capabilities related to metering and billing, while the client-side component handles data collection and billing capabilities.

• The Cost Management Server component relies on ClickHouse as its data storage solution. You can either install the ACP Log Storage with Clickhouse plugin provided by the platform or install ClickHouse on your own for integration.

• The Cost Management Essentials needs to integrate with the Cost Management Server component.

• The Cost Management Agent component requires that the corresponding cluster has a monitoring component installed, either ACP Monitor with Prometheus or ACP Monitor with VictoriaMetrics. Please ensure that one of them is successfully installed in advance. For VictoriaMetrics clusters running an Alauda Container Platform version earlier than 4.4.0, see Enable Required Metrics for VictoriaMetrics on Versions Earlier Than 4.4.0.

#Install the Cost Management Server Plugin via Console

#Prerequisites

• The Cost Management Server Plugin component relies on ClickHouse as its data storage solution. You can either install the ACP Log Storage with Clickhouse plugin provided by the platform or install ClickHouse on your own for integration.

#Procedure

1. Log in to the cluster where the ACP Log Storage with Clickhouse plugin is installed and obtain ClickHouse integration information:

   # Obtain Clickhouse connection address
   kubectl get feature log-center -o jsonpath='{.spec.accessInfo.clickhouse.address}'
   
   # Obtain Clickhouse username and password:
   kubectl get secret -n cpaas-system clickhouse-basic-auth -o jsonpath={.data.username}|base64 -d
   kubectl get secret -n cpaas-system clickhouse-basic-auth -o jsonpath={.data.password}|base64 -d

2. In the left navigation bar, click Marketing Place > Plugin Management.

3. Select the cluster which you plan to install the plugin.

4. Find the Cost Management Server plugin and click the Install button in the right actions bar.

5. Refer to the following description to configure the relevant parameters.

   Parameter	Description
   Clickhouse Address	Configure the Clickhouse connection address
   Clickhouse Cluster Name	The cluster name for Clickhouse; the default for platform-created Clickhouse is replicated.
   Clickhouse Secret Name	Configure the Secret for Clickhouse username and password
   Currency Unit	Choose the currency unit to display expenses.

6. Click Install.

#Install the Cost Management Server Plugin via YAML

#Prerequisites

• The Cost Management Server Plugin component relies on ClickHouse as its data storage solution. You can either install the ACP Log Storage with Clickhouse plugin provided by the platform or install ClickHouse on your own for integration.

#Procedure

1. Check available versions

   Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the global cluster:

   # kubectl get moduleplugin | grep cost-server
   cost-server                       30h
   # kubectl get moduleconfig | grep cost-server
   cost-server-v4.1.0                30h

   This indicates that the ModulePlugin cost-server exists in the cluster and version v4.1.0 is published.

2. Create a ModuleInfo

   Create a ModuleInfo resource to install the plugin without any configuration parameters:

   kind: ModuleInfo
   apiVersion: cluster.alauda.io/v1alpha1
   metadata:
     name: global-cost-server
     labels:
       cpaas.io/cluster-name: global
       cpaas.io/module-name: cost-server
       cpaas.io/module-type: plugin
   spec:
     version: v4.1.0
     config:
       global:
         clickhouse:
           address: https://xxx.xxx.xxx.xxx:xx/clickhouse
           authSecret: clickhouse-basic-auth
           cluster: replicated
         currency: CNY(¥)

   YAML field reference (cost-server):

   Field path	Description
   metadata.labels.cpaas.io/cluster-name	Target cluster name where the plugin is installed.
   metadata.labels.cpaas.io/module-name	Must be cost-server.
   metadata.labels.cpaas.io/module-type	Must be plugin.
   metadata.name	ModuleInfo name (e.g., <cluster>-cost-server).
   spec.version	Plugin version to install.
   spec.config.global.clickhouse.address	Configure the Clickhouse connection address.
   spec.config.global.clickhouse.authSecret	Configure the Secret for Clickhouse username and password.
   spec.config.global.clickhouse.cluster	The cluster name for Clickhouse; the default for platform-created Clickhouse is replicated.
   spec.config.global.currency	Choose the currency unit to display expenses. Support: CNY(¥)/USD($)/THB(฿)/RUB(₽)/AUD(AU$)/CAD(CA$)/CHF(Fr.)/DKK(kr.)/EUR(€)/GBP(£)/IDR(Rp)/INR(₹)/JPY(JP¥)/HKD(HK)/SGD(SG$)

3. Verify installation

   Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

   kubectl get moduleinfo -l cpaas.io/module-name=cost-server
   NAME                                             CLUSTER         MODULE            DISPLAY_NAME     STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
   global-e671599464a5b1717732c5ba36079795          global          cost-server       cost-server      Running   v4.1.0           v4.1.0            v4.1.0

   Field explanations:

   • NAME: ModuleInfo resource name
   • CLUSTER: Cluster where the plugin is installed
   • MODULE: Plugin name
   • DISPLAY_NAME: Display name of the plugin
   • STATUS: Installation status; Running means successfully installed and running
   • TARGET_VERSION: Intended installation version
   • CURRENT_VERSION: Version before installation
   • NEW_VERSION: Latest available version for installation

#Install the Cost Management Essentials Plugin via Console

#Prerequisites

• The Cost Management Essentials Plugin needs to integrate with the Cost Management Server component. So you need to install the Cost Management Server plugin first.

#Procedure

1. In the left navigation bar, click Marketing Place > Plugin Management.

2. Select the global cluster.

3. Find the Cost Management Server plugin and click the Install button in the right actions bar.

4. Select the cluster name which installed the Cost Management Server plugin

5. Click Install.

#Install the Cost Management Essentials Plugin via YAML

#Prerequisites

• The Cost Management Essentials Plugin needs to integrate with the Cost Management Server component. So you need to install the Cost Management Server plugin first.

1. Check available versions

   Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the global cluster:

   # kubectl get moduleplugin | grep cost-api
   cost-api                       30h
   # kubectl get moduleconfig | grep cost-api
   cost-api-v4.1.0                30h

   This indicates that the ModulePlugin cost-api exists in the cluster and version v4.1.0 is published.

2. Create a ModuleInfo

   Create a ModuleInfo resource to install the plugin without any configuration parameters:

   kind: ModuleInfo
   apiVersion: cluster.alauda.io/v1alpha1
   metadata:
     name: global-cost-api
     labels:
       cpaas.io/cluster-name: global
       cpaas.io/module-name: cost-api
       cpaas.io/module-type: plugin
   spec:
     version: v4.1.0
     config:
       crossClusterDependency:
         costServer: business-1

   YAML field reference (cost-api):

   Field path	Description
   metadata.labels.cpaas.io/cluster-name	Must be global.
   metadata.labels.cpaas.io/module-name	Must be cost-api.
   metadata.labels.cpaas.io/module-type	Must be plugin.
   metadata.name	ModuleInfo name (e.g., <cluster>-cost-api).
   spec.version	Plugin version to install.
   spec.config.crossClusterDependency.costServer	Configure the cluster name which installed the Cost Management Server plugin

3. Verify installation

   Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

   kubectl get moduleinfo -l cpaas.io/module-name=cost-api
   NAME                                             CLUSTER         MODULE            DISPLAY_NAME     STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
   global-e671599464a5b1717732c5ba36079795          global          cost-api          cost-api         Running   v4.1.0           v4.1.0            v4.1.0

   Field explanations:

   • NAME: ModuleInfo resource name
   • CLUSTER: Cluster where the plugin is installed
   • MODULE: Plugin name
   • DISPLAY_NAME: Display name of the plugin
   • STATUS: Installation status; Running means successfully installed and running
   • TARGET_VERSION: Intended installation version
   • CURRENT_VERSION: Version before installation
   • NEW_VERSION: Latest available version for installation

#Install the Cost Management Agent Plugin via Console

#Prerequisites

• The Cost Management Agent Plugin component requires that the corresponding cluster has a monitoring component installed, either ACP Monitor with Prometheus or ACP Monitor with VictoriaMetrics. Please ensure that one of them is successfully installed in advance. For VictoriaMetrics clusters running an Alauda Container Platform version earlier than 4.4.0, see Enable Required Metrics for VictoriaMetrics on Versions Earlier Than 4.4.0.

• The Cost Management Agent Plugin needs to integrate with the Cost Management Server component. So you need to install the Cost Management Server plugin first.

#Procedure

1. In the left navigation bar, click Marketing Place > Plugin Management.

2. Select the cluster where this plugin needs to be installed.

3. Find the Cost Management Agent plugin and click the Install button in the right actions bar.

4. Select the cluster name which installed the Cost Management Server plugin

5. Click Install.

#Install the Cost Management Agent Plugin via YAML

#Prerequisites

• The Cost Management Agent Plugin component requires that the corresponding cluster has a monitoring component installed, either ACP Monitor with Prometheus or ACP Monitor with VictoriaMetrics. Please ensure that one of them is successfully installed in advance. For VictoriaMetrics clusters running an Alauda Container Platform version earlier than 4.4.0, see Enable Required Metrics for VictoriaMetrics on Versions Earlier Than 4.4.0.

• The Cost Management Agent Plugin needs to integrate with the Cost Management Server component. So you need to install the Cost Management Server plugin first.

#Procedure

1. Check available versions

   Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the global cluster:

   # kubectl get moduleplugin | grep cost-agent
   cost-agent                       30h
   # kubectl get moduleconfig | grep cost-agent
   cost-agent-v4.1.0                30h

   This indicates that the ModulePlugin cost-agent exists in the cluster and version v4.1.0 is published.

2. Create a ModuleInfo

   Create a ModuleInfo resource to install the plugin without any configuration parameters:

   kind: ModuleInfo
   apiVersion: cluster.alauda.io/v1alpha1
   metadata:
     name: global-cost-agent
     labels:
       cpaas.io/cluster-name: global
       cpaas.io/module-name: cost-agent
       cpaas.io/module-type: plugin
   spec:
     version: v4.1.0
     config:
       crossClusterDependency:
         costServer: business-1

   YAML field reference (cost-agent):

   Field path	Description
   metadata.labels.cpaas.io/cluster-name	Target cluster name where the plugin is installed.
   metadata.labels.cpaas.io/module-name	Must be cost-agent.
   metadata.labels.cpaas.io/module-type	Must be plugin.
   metadata.name	ModuleInfo name (e.g., <cluster>-cost-agent).
   spec.version	Plugin version to install.
   spec.config.crossClusterDependency.costServer	Configure the cluster name which installed the Cost Management Server plugin

3. Verify installation

   Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

   kubectl get moduleinfo -l cpaas.io/module-name=cost-agent
   NAME                                             CLUSTER         MODULE            DISPLAY_NAME     STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
   global-e671599464a5b1717732c5ba36079795          global          cost-agent        cost-agent       Running   v4.1.0           v4.1.0            v4.1.0

#Enable Required Metrics for VictoriaMetrics on Versions Earlier Than 4.4.0

Metering and billing supports both Prometheus and VictoriaMetrics as the monitoring component of the metered cluster.

On Alauda Container Platform 4.4.0 and later, VictoriaMetrics collects all metrics required by metering and billing, so no extra configuration is needed. On earlier versions, the kube-state-metrics allowlist shipped with VictoriaMetrics does not contain all of them, and the missing metrics have to be enabled manually.

#Prerequisites

• The monitoring component of the metered cluster is ACP Monitor with VictoriaMetrics, and the Alauda Container Platform version is earlier than 4.4.0.
• You can access the global cluster with a user who is allowed to update the ModuleInfo resources.

#Procedure

Perform the following steps on the global cluster, once for each metered cluster whose monitoring component is VictoriaMetrics.

1. Locate the VictoriaMetrics ModuleInfo of the metered cluster:

   kubectl get moduleinfo -l cpaas.io/module-name=victoriametrics,cpaas.io/cluster-name=<cluster-name>

2. Add the missing metrics to spec.valuesOverride:

   kubectl patch moduleinfo <moduleinfo-name> --type=merge -p '{
     "spec": {
       "valuesOverride": {
         "ait/chart-victoriametrics": {
           "exporter-kube-state": {
             "additionalKeepMetrics": [
               "kube_namespace_annotations",
               "kube_node_labels",
               "kube_persistentvolume_capacity_bytes",
               "kube_persistentvolumeclaim_info",
               "kube_persistentvolumeclaim_resource_requests_storage_bytes",
               "kube_pod_annotations",
               "kube_pod_container_status_running",
               "kube_pod_owner"
             ]
           }
         }
       }
     }
   }'

   additionalKeepMetrics is merged with the built-in allowlist, and applies both to the metrics exposed by kube-state-metrics and to the metrics kept when they are scraped. Do not patch the rendered Deployment or VMServiceScrape objects directly: such a change is recorded as a ResourcePatch and replayed on every subsequent chart deployment, and patching only one of the two places has no effect.

3. Wait about 2 minutes for the plugin to converge, and confirm that all releases are ready:

   kubectl get moduleinfo <moduleinfo-name> -o jsonpath='{range .status.appReleases[*]}{.name}{"="}{.ready}{" "}{end}'

#Verification

1. On the metered cluster, confirm that kube-state-metrics exposes the metrics:

   kubectl -n cpaas-system get deploy kube-prometheus-exporter-kube-state \
     -o jsonpath='{.spec.template.spec.containers[0].command}' | tr ',' '\n' | grep metric-allowlist

2. On the metered cluster, confirm that the metrics are kept when they are scraped:

   kubectl -n cpaas-system get vmservicescrape kube-prometheus-exporter-kube-state \
     -o jsonpath='{.spec.endpoints[0].metricRelabelConfigs[0].regex}' | tr '|' '\n' | grep kube_pod_owner

The eight metrics listed above must appear in the output of both commands. After they are collected, the Cost Management Agent starts producing Pod-level metering data from the next collection cycle.