#Installation

The platform's logging system consists of three plugins:Alauda Container Platform Log Essentails , Alauda Container Platform Log Collector and Alauda Container Platform Log Storage. This chapter will introduce you to the installation of these plugins.

#Installation Planning

Alauda Container Platform Log Storage plugins can be installed in any cluster, and any cluster's log storage component can be selected for log collection to interface with the storage data.

So , before installing the log storage plugin , you need to plan the cluster and node where the log storage component will be installed.

• Separate log storage by scope. Deploy log storage in the global cluster for the global cluster's own logs, so that the global cluster collects, stores, and queries its own logs independently. Centralize application logs from workload clusters into a dedicated log storage workload cluster rather than into the global cluster, so that a global cluster failure does not disrupt log-based issue resolution for applications, and application log load does not affect the management cluster.

• If the global cluster is deployed with disaster recovery, do not send workload cluster logs to either the primary or the standby global cluster. Disaster recovery synchronizes only the global cluster's etcd data and excludes namespaces such as cpaas-system, so log data is not replicated to the standby cluster. Each global cluster stores and queries only its own logs and never uses the other cluster's log storage; otherwise application logs become unavailable after a failover. For the synchronization scope, see Global Cluster Disaster Recovery.

• Prioritize centralizing logs to a single log storage cluster. If log volume exceeds maximum capacity thresholds, distribute logs across multiple storage clusters.

• Deploy at least one log storage instance per network zone to aggregate logs locally, minimizing cross-data-center public network traffic (which incurs high costs and latency).

• Dedicate exclusive nodes for log storage, avoiding co-deployment with other applications or platform components. Log storage requires high I/O throughput and may be affected by interference.

• Mount dedicated SSD disks for log storage to significantly enhance performance.

#Host Paths on Alauda OS Nodes

The default host paths in this chapter use the /cpaas prefix, which is the layout of the traditional operating system. Alauda OS nodes are immutable: only /var/cpaas is writable, so a path under /cpaas is not valid on those nodes.

When you install a log storage plugin on Alauda OS nodes, replace the /cpaas prefix with /var/cpaas in the host path field. The host path is set in the plugin ModuleInfo; the console install form does not expose it.

Back the selected path with a persistent disk that survives node re-provisioning, and size that disk for the retention period you plan to use. Set the path before you install, because the host path cannot be changed afterwards.

#Install Alauda Container Platform Log Essentials via console

1. Navigate to App Store Management > Cluster Plugin and select the global cluster.

2. In the Plugins tab, click the action button to the right of Alauda Container Platform Log Essentials > Install.

3. Click Install.

#Install Alauda Container Platform Log Essentials via YAML

#1. Check available versions

Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the globalcluster:

# kubectl get moduleplugin | grep log-api
log-api                       30h
# kubectl get moduleconfig | grep log-api
log-api-v4.2.0                30h

This indicates that the ModulePlugin log-api exists in the cluster and version v4.2.0 is published.

#2. Create a ModuleInfo

Create a ModuleInfo resource to install the plugin without any configuration parameters:

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  annotations:
    cpaas.io/display-name: log-api
    cpaas.io/module-name: '{"en": "Alauda Container Platform Log Essentials", "zh":
      "Alauda Container Platform Log Essentials"}'
  labels:
    cpaas.io/cluster-name: global
    cpaas.io/module-name: log-api
    cpaas.io/module-type: plugin
    cpaas.io/product: Platform-Center
  name: global-b4a67e6adb67bad47544354e9bf2bdff
spec:
  version: v4.2.0
  valuesOverride:
    ait/chart-alauda-log-api:
      global:
        nodeSelector:
          node-role.kubernetes.io/infra: "true"
        tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
          operator: Equal

YAML field reference:

#3. Verify installation

Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

kubectl get moduleinfo -l cpaas.io/module-name=log-api
NAME                                             CLUSTER         MODULE      DISPLAY_NAME   STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
global-e671599464a5b1717732c5ba36079795          global          log-api   log-api      Running   v4.2.0          v4.2.0           v4.2.0

Field explanations:

• NAME: ModuleInfo resource name
• CLUSTER: Cluster where the plugin is installed
• MODULE: Plugin name
• DISPLAY_NAME: Display name of the plugin
• STATUS: Installation status; Running means successfully installed and running
• TARGET_VERSION: Intended installation version
• CURRENT_VERSION: Version before installation
• NEW_VERSION: Latest available version for installation

#Install Alauda Container Platform Log Storage with ElasticSearch via console

1. Navigate to App Store Management > Cluster Plugin and select the target cluster.

2. In the Plugins tab, click the action button to the right of Alauda Container Platform Log Storage with ElasticSearch > Install.

3. Refer to the following instructions to configure relevant parameters.

   Parameter	Description
   Connect External Elasticsearch	Keep closed to install the log storage plugin within the platform.
   Component installation Settings	LocalVolume: Local storage, log data will be stored in the local storage path of the selected node. The advantage of this method is that the log component is directly bound to local storage, eliminating the need to access storage over the network and providing better storage performance. StorageClass: Dynamically create storage resources using storage classes to store log data. The advantage of this method is a higher degree of flexibility; when multiple storage classes are defined for the entire cluster, administrators can select the corresponding storage class for the log components based on usage scenarios, reducing the impact of host malfunction on storage. However, the performance of StorageClass may be affected by factors such as network bandwidth and latency, and it relies on the redundancy mechanisms provided by the storage backend to achieve high availability of storage.
   Retention Period	The maximum time logs, events, and audit data can be retained on the cluster. Data exceeding the retention period will be automatically cleaned up. Tip: You may back up data that needs to be retained for a long time. If you need assistance, please contact technical support personnel.

4. Click Install.

#Install Alauda Container Platform Log Storage with ElasticSearch via YAML

#1. Check available versions

Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the globalcluster:

# kubectl get moduleplugin | grep logcenter
logcenter                       30h
# kubectl get moduleconfig | grep logcenter
logcenter-v4.1.0                30h

This indicates that the ModulePlugin logcenter exists in the cluster and version v4.1.0 is published.

#2. Create a ModuleInfo

Create a ModuleInfo resource to install the plugin without any configuration parameters:

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  annotations:
    cpaas.io/display-name: logcenter
    cpaas.io/module-name: '{"en": "Alauda Container Platform Log Storage for Elasticsearch", "zh": "Alauda Container Platform Log Storage for Elasticsearch"}'
  labels:
    cpaas.io/cluster-name: go
    cpaas.io/module-name: logcenter
    cpaas.io/module-type: plugin
    cpaas.io/product: Platform-Center
  name: <cluster>-log-center
spec:
  config:
    clusterView:
      isPrivate: "true"
    components:
      elasticsearch:
        address: ""
        basicAuthSecretName: ""
        hostpath: /cpaas/data/elasticsearch # On Alauda OS nodes: /var/cpaas/data/elasticsearch
        httpPort: 9200
        install: true
        k8sNodes:
        - 192.168.139.75
        masterK8sNodes: []
        masterReplicas: 0
        masterResources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: 200m
            memory: 256Mi
        masterStorageSize: 5
        nodeReplicas: 1
        nodeStorageSize: 200
        resources:
          limits:
            cpu: "4"
            memory: 4Gi
          requests:
            cpu: "1"
            memory: 1Gi
        tcpPort: 9300
        type: single
      kafka:
        address: ""
        auth: true
        basicAuthSecretName: ""
        exporterPort: 9308
        install: true
        k8sNodes:
        - 192.168.139.75
        port: 9092
        storageSize: 10
        tls: true
        zkElectPort: 3888
        zkExporterPort: 9141
        zkLeaderPort: 2888
        zkPort: 2181
      kibana:
        install: false
      storageClassConfig:
        name: elasticsearch-local-log-sc
        type: LocalVolume
      zookeeper:
        storageSize: 1
    ttl:
      audit: 180
      event: 180
      logKubernetes: 7
      logPlatform: 7
      logSystem: 7
      logWorkload: 7
  version: v4.1.0
  valuesOverride:
    ait/chart-alauda-log-center:
      global:
        nodeSelector:
          node-role.kubernetes.io/infra: "true"
        tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
          operator: Equal

YAML field reference:

#3. Verify installation

Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

kubectl get moduleinfo -l cpaas.io/module-name=logcenter
NAME                                             CLUSTER         MODULE      DISPLAY_NAME   STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
global-e671599464a5b1717732c5ba36079795          global          logcenter   logcenter      Running   v4.0.12          v4.0.12           v4.0.12

Field explanations:

• NAME: ModuleInfo resource name
• CLUSTER: Cluster where the plugin is installed
• MODULE: Plugin name
• DISPLAY_NAME: Display name of the plugin
• STATUS: Installation status; Running means successfully installed and running
• TARGET_VERSION: Intended installation version
• CURRENT_VERSION: Version before installation
• NEW_VERSION: Latest available version for installation

#Install Alauda Container Platform Log Storage with Clickhouse via console

1. Navigate to App Store Management > Cluster Plugin and select the target cluster.

2. In the Plugins tab, click the action button to the right of Alauda Container Platform Log Storage with Clickhouse > Install.

3. Refer to the following instructions to configure relevant parameters.

   Parameter	Description
   Component installation Settings	LocalVolume: Local storage, log data will be stored in the local storage path of the selected node. The advantage of this method is that the log component is directly bound to local storage, eliminating the need to access storage over the network and providing better storage performance. StorageClass: Dynamically create storage resources using storage classes to store log data. The advantage of this method is a higher degree of flexibility; when multiple storage classes are defined for the entire cluster, administrators can select the corresponding storage class for the log components based on usage scenarios, reducing the impact of host malfunction on storage. However, the performance of StorageClass may be affected by factors such as network bandwidth and latency, and it relies on the redundancy mechanisms provided by the storage backend to achieve high availability of storage.
   Retention Period	The maximum time logs, events, and audit data can be retained on the cluster. Data exceeding the retention period will be automatically cleaned up. Tip: You may back up data that needs to be retained for a long time. If you need assistance, please contact technical support personnel.

4. Click Install.

#Install Alauda Container Platform Log Storage with Clickhouse via YAML

#1. Check available versions

Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the globalcluster:

# kubectl get moduleplugin | grep logclickhouse
logclickhouse                       30h
# kubectl get moduleconfig | grep logclickhouse
logclickhouse-v4.1.0                30h

This indicates that the ModulePlugin logclickhouse exists in the cluster and version v4.1.0 is published.

#2. Create a ModuleInfo

Create a ModuleInfo resource to install the plugin without any configuration parameters:

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  name: global-logclickhouse
  labels:
    cpaas.io/cluster-name: global
    cpaas.io/module-name: logclickhouse
    cpaas.io/module-type: plugin
spec:
  version: v4.1.0
  config:
    components:
      storageClassConfig:
        type: LocalVolume
        name: ""
      clickhouse:
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: 200m
            memory: 256Mi
        k8sNodes:
          - xxx.xxx.xxx.xx
        hostpath: /cpaas/data/clickhouse # On Alauda OS nodes: /var/cpaas/data/clickhouse
        nodeReplicas: 1
        nodeStorageSize: 200
        type: single
      razor:
        resources:
          limits:
            cpu: "2"
            memory: 1Gi
          requests:
            cpu: 10m
            memory: 256Mi
      vector:
        resources:
          limits:
            cpu: "4"
            memory: 1Gi
          requests:
            cpu: 10m
            memory: 256Mi
    ttl:
      audit: 180
      event: 180
      logKubernetes: 7
      logPlatform: 7
      logSystem: 7
      logWorkload: 7
  valuesOverride:
    ait/chart-alauda-log-clickhouse:
      global:
        nodeSelector:
          node-role.kubernetes.io/infra: "true"
        tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
          operator: Equal

YAML field reference (ClickHouse):

#3. Verify installation

Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

kubectl get moduleinfo -l cpaas.io/module-name=logclickhouse
NAME                                             CLUSTER         MODULE      DISPLAY_NAME   STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
global-e671599464a5b1717732c5ba36079795          global          logclickhouse   logclickhouse      Running   v4.0.12          v4.0.12           v4.0.12

Field explanations:

• NAME: ModuleInfo resource name
• CLUSTER: Cluster where the plugin is installed
• MODULE: Plugin name
• DISPLAY_NAME: Display name of the plugin
• STATUS: Installation status; Running means successfully installed and running
• TARGET_VERSION: Intended installation version
• CURRENT_VERSION: Version before installation
• NEW_VERSION: Latest available version for installation

#Install Alauda Container Platform Log Collector Plugin

1. Navigate to App Store Management > Cluster Plugin and select the target cluster.

2. In the Plugins tab, click the action button to the right of Alauda Container Platform Log Collector > Install.

3. Select the Storage Cluster (where Alauda Container Platform Log Storage has been installed) and click Select/Deselect log types to set the scope of log collection in the cluster.

4. Click Install.

#Install Alauda Container Platform Log Collector Plugin via YAML

#1. Check available versions

Ensure the plugin has been published by checking for ModulePlugin and ModuleConfig resources, in the globalcluster:

# kubectl get moduleplugin | grep logagent
logagent                       30h
# kubectl get moduleconfig | grep logagent
logagent-v4.1.0                30h

This indicates that the ModulePlugin logagent exists in the cluster and version v4.1.0 is published.

#2. Create a ModuleInfo

Create a ModuleInfo resource to install the plugin without any configuration parameters:

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  annotations:
    cpaas.io/display-name: logagent
    cpaas.io/module-name: '{"en": "Alauda Container Platform Log Collector", "zh": "Alauda Container Platform Log Collector"}'
  labels:
    cpaas.io/cluster-name: go
    cpaas.io/module-name: logagent
    cpaas.io/module-type: plugin
    cpaas.io/product: Platform-Center
    logcenter.plugins.cpaas.io/cluster: go
  name: <cluster>-log-agent
spec:
  config:
    crossClusterDependency:
      logcenter: go
      logclickhouse: null
    dataSource:
      audit: true
      event: true
      kubernetes: true
      platform: false
      system: false
      workload: true
    storage:
      type: Elasticsearch
  version: v4.1.0
  valuesOverride:
    ait/chart-alauda-log-agent:
      global:
        nodeSelector:
          node-role.kubernetes.io/infra: "true"
        tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
          operator: Equal

YAML field reference (Log Collector):

#3. Verify installation

Since the ModuleInfo name changes upon creation, locate the resource via label to check the plugin status and version:

kubectl get moduleinfo -l cpaas.io/module-name=logagent
NAME                                             CLUSTER         MODULE      DISPLAY_NAME   STATUS    TARGET_VERSION   CURRENT_VERSION   NEW_VERSION
global-e671599464a5b1717732c5ba36079795          global          logagent   logagent      Running   v4.0.12          v4.0.12           v4.0.12