#How to Interface with External ES Storage Clusters

You can interface with external Elasticsearch or Kafka clusters by writing YAML configurations. Depending on your business requirements, you can choose to interface with only the external Elasticsearch cluster (while installing Kafka in the current cluster), or you can interface with both the external Elasticsearch and Kafka clusters simultaneously.

#Resource Preparation

Before interfacing, you need to prepare the required credential information.

1. In the left navigation bar, click on Cluster Management > Resource Management, then switch to the cluster that needs the plugin installation.

2. Click on Create Resource Object, and fill in the code box after modifying the parameters according to the code comments.

• Credentials required for interfacing with external Elasticsearch:

  apiVersion: v1
  type: Opaque
  data:
    password: dEdWQVduSX5kUW1mc21acg==  # Must be base64 encoded. Reference command: echo -n <password_value>| base64
    username: YWRtaW4=                  # Must be base64 encoded. Reference command: echo -n <username_value>| base64
  kind: Secret
  metadata:
    name: elasticsearch-basic-auth      # Credential name. Ensure that the value of elasticsearch.basicAuthSecretName in the log storage YAML matches this parameter.
    namespace: cpaas-system             # The namespace where the Elasticsearch component is located, generally cpaas-system.
  

• If you need to use an external Kafka cluster, you will also need to create credentials for interfacing with the external Kafka cluster:

  apiVersion: v1
  type: Opaque
  data:
    password: dEdWQVduSX5kUW1mc21acg==  # Must be base64 encoded. Reference command: echo -n <password_value>| base64
    username: YWRtaW4=                  # Must be base64 encoded. Reference command: echo -n <username_value>| base64
  kind: Secret
  metadata:
    name: kafka-basic-auth              # Credential name. Ensure that the value of kafka.basicAuthSecretName in the log storage YAML matches this parameter.
    namespace: cpaas-system             # The namespace where the Kafka component is located, generally cpaas-system.
  

3. Click on Create.

#Operating Procedures

1. In the left navigation bar, click on App Store > Plugin Management.

2. In the top navigation, select the Cluster Name where you want to install the ACP Log Storage with Elasticsearch plugin.

3. Click the action button on the right side of ACP Log Storage with Elasticsearch > Install.

4. Enable the Interface with External Elasticsearch switch, configure the YAML file, with the interfacing example and parameter descriptions as follows:

• Interfacing with the external Elasticsearch cluster while installing Kafka in the current cluster:

  elasticsearch:
    install: false
    address: http://fake:9200                # External ES access address, e.g., http://192.168.143.252:11780/es_proxy
    basicAuthSecretName: elasticsearch-basic-auth # Credentials required for interfacing with external Elasticsearch created in the prerequisites.
  storageClassConfig:
    type: "LocalVolume"   # Default is LocalVolume. Options are "LocalVolume" or "StorageClass".
  kafka:
    auth: true                               # Whether to enable authentication.
    k8sNodes:
    - log1                                   # Node name, obtained from kubectl get nodes.
    - log2
    - log3
    storageSize: 10                          # Storage size, in Gi, default is 10 Gi.
  

• Interfacing with both the external Elasticsearch cluster and the external Kafka cluster:

  elasticsearch:
    install: false
    address: http://fake:9200                # External ES access address, e.g., http://192.168.143.252:11780/es_proxy
    basicAuthSecretName: elasticsearch-basic-auth # Credentials required for interfacing with external Elasticsearch created in the prerequisites.
  kafka:
    auth: true                               # Whether to enable authentication.
    install: false
    basicAuthSecretName: kafka-basic-auth  # Credentials required for interfacing with external Kafka created in the prerequisites.
    address: 192.168.130.169:9092,192.168.130.187:9092,192.168.130.193:9092     # Kafka access addresses, separated by commas.