#Create a Cluster on Huawei Cloud Stack

This document provides instructions for creating Kubernetes clusters on the Huawei Cloud Stack platform. YAML-based cluster creation is available through Cluster API manifests. If Fleet Essentials 1.0.2 or later is installed, you can also create clusters through a guided web UI.

#Prerequisites

Before creating clusters, ensure all of the following prerequisites are met:

#1. Required Plugin Installation

Before you create the first workload cluster on Huawei Cloud Stack, upload and install the following plugins on the 's global cluster. This applies regardless of the IaaS platform that the global cluster itself runs on:

• Alauda Container Platform Cluster Bootstrap Provider
• Alauda Container Platform Infrastructure Provider for Huawei Cloud Stack

For the procedure, see Install the Provider.

#2. HCS Infrastructure Input Preparation

Prepare all HCS-specific inputs before writing any YAML in this document:

• HCS credential Secret values
• Provider-recognized compute values such as imageName, flavorName, and availabilityZone
• Cluster network inventory, including the subnets and free IP ranges used by the cluster
• Control plane ELB address planning, including vipAddress, vipSubnetName, and fixed L4 and L7 IPs
• Static IP pool planning for control plane and worker nodes

See Infrastructure Resources for Huawei Cloud Stack for the complete checklist, source information, and constraints.

#3. OS Release Registration

This prerequisite applies to the web UI workflow only. YAML-based creation sets the OS image and component versions directly and does not need it.

The web UI does not ask you to type the OS image or its Kubernetes and component versions. It reads them from an OS-release ConfigMap that you register once per Alauda OS image. The wizard resolves the platform Distribution version from the global cluster, finds the registered OS image whose supportedACPVersions includes that Distribution, and derives the node image and control plane component versions from it.

If no registered OS image matches the current Distribution, the wizard blocks creation with an error such as:

No OS registered for the current platform Distribution v4.3.2. Register an OS-release ConfigMap first.

Register the OS image on the global cluster before you open the wizard:

apiVersion: v1
kind: ConfigMap
metadata:
  # Any DNS-1123 name. The name is not a lookup key.
  # Do not copy the image version verbatim: it contains "_" (for example, x86_64), which is invalid in a ConfigMap name.
  name: alaudaos-43-m55-x86-64
  namespace: cpaas-system
  labels:
    cpaas.io/os-release: ""
data:
  osImageVersion: "alaudaos-43.m55.202606251459-0.x86_64"
  supportedACPVersions: "v4.3.2"
  osVariant: "m55"
  architecture: "x86_64"
  osBuild: "202606251459-0"
  kubernetesVersion: "v1.34.5-3"
  containerdVersion: "2.2.1-5"
  coreDNSImageTag: "1.14.2-v4.3.11"
  etcdImageTag: "v3.5.28-260625"
  pauseImageTag: "3.10"

Apply it with kubectl apply -f os-release.yaml.

Field sources:

The osImageVersion must name an image that already exists on the HCS platform. To read the exact name used in an environment that already has clusters, list an existing machine template:

kubectl -n cpaas-system get hcsmachinetemplate \
  -o custom-columns=NAME:.metadata.name,IMAGE:.spec.template.spec.imageName

#4. Workload Cluster Image Registry

Decide which registry this cluster pulls its platform components from — Kube-OVN and the platform modules that Alauda Container Platform installs on the cluster. This choice does not affect the images kubeadm manages, which come from KubeadmControlPlane.spec.kubeadmConfigSpec.clusterConfiguration.imageRepository.

Either way, the registry must already hold Alauda Container Platform Core for the target version plus the packages for every platform plugin you intend to install on this cluster, and every workload cluster node must reach its address.

For the dedicated-registry procedure and its constraints, see Choose the Image Registry for a Workload Cluster.

#5. Node Labels and Taints

To give the nodes of a pool their own role label and taint — for example to reserve dedicated pools for monitoring, logging, or ingress — declare them on the pool's resources as described in Infra Nodes. Plan them before you create the pools. This applies to every node pool, whether the cluster is created through the web UI or with YAML; the web UI node pool steps do not set labels or taints.

#Using the Web UI

Requirement: This workflow requires Fleet Essentials 1.0.2 or later. Without it, create clusters with the YAML manifests described in Using YAML.

The web UI loads VPCs, subnets, security groups, flavors, and assignable IP addresses from the HCS platform using the Infrastructure Credential you select. Make sure the prerequisites above are met before you start, and that an HCS credential Secret exists in the cpaas-system namespace.

#Creation Workflow

The cluster creation follows a 6-step wizard:

Step 1: Basic Info
  ↓
Step 2: Networking
  ↓
Step 3: Infrastructure
  ↓
Step 4: Control Plane Node Pool
  ↓
Step 5: Worker Node Pools
  ↓
Step 6: Review

Navigation: Clusters → Clusters → Create Cluster → Select Huawei Cloud Stack

#Step 1: Basic Info

#Step 2: Networking

#Step 3: Infrastructure

This step has two parts: the HCS infrastructure resources and the control plane load balancer.

Infrastructure resources:

Control Plane Load Balancer: The HCS provider creates an Elastic Load Balance (ELB) for the Kubernetes API server and owns its VIP. Hybrid Load Balancing requires fixed L4 and L7 virtual subnet IPs.

#Step 4: Control Plane Node Pool

The control plane node pool is fixed at 3 replicas for high availability.

#Machine Config entry

Each entry in Machine Configs describes one node:

• Hostname (required): Lowercase letters, numbers, hyphens, and dots, up to 253 characters (for example, master-1 or master-1.example.org). For how a dotted hostname affects the node, see Hostname behavior on the node.
• Networks (required): One row per NIC. Each row has an IP Address picker and a Subnet Name. Add rows for multi-NIC nodes.
• Persistent Disks: The platform-required /var/cpaas disk is pre-filled; its size is adjustable but the row cannot be removed. Add rows only for extra host-bound data that must survive node rebuild.

#Step 5: Worker Node Pools

Click Add Worker Node Pool to open the pool dialog. You can add multiple pools.

#Step 6: Review

Review the Basic Info, Infrastructure, Control Plane Load Balancer, Control Plane Node Pool, Worker Node Pools, and Networking sections. Click View YAML to inspect the exact resources that will be created. Click Create to start the cluster creation process.

#Using YAML

#Cluster Creation Overview

At a high level, you'll create the following Cluster API resources in the 's global cluster to provision infrastructure and bootstrap a functional Kubernetes cluster.

Before you write any YAML in this page, complete the preparation checklist in Infrastructure Resources for Huawei Cloud Stack. This checklist covers the values that the provider expects, where to get them, and which values must be planned before you fill the manifests.

The cluster creation process follows this order:

1. Configure HCS authentication (Secret)
2. Create machine configuration pool (HCSMachineConfigPool)
3. Configure machine template (HCSMachineTemplate)
4. Configure KubeadmControlPlane
5. Configure HCSCluster
6. Create the Cluster
7. Deploy worker nodes (HCSMachineConfigPool, HCSMachineTemplate, KubeadmConfigTemplate, and MachineDeployment). See Deploy Worker Nodes.

#Control Plane Configuration

The control plane manages cluster state, scheduling, and the Kubernetes API. This section shows how to configure a highly available control plane.

#Configure HCS Authentication

HCS authentication information is stored in a Secret resource in cpaas-system.

Author the credential file with an editor rather than typing the values at a shell prompt. A shell records the whole command, secret key included, in its history; a file written by an editor leaves no such copy.

accessKey=<access-key>
secretKey=<secret-key>
projectID=<project-id>
region=<region>
externalGlobalDomain=<domain>
schema=https

Restrict the file, create the Secret from it, then remove it:

chmod 600 hcs-credential.env

HCS_SECRET_NAME="<credential-secret-name>"

kubectl create secret generic "${HCS_SECRET_NAME}" \
  --namespace cpaas-system \
  --from-env-file=hcs-credential.env

rm -f hcs-credential.env

Write the values in plain text — kubectl base64-encodes them when it creates the Secret, so there is no separate encoding step. Each line is key=value. Do not quote the values: kubectl takes quote characters literally. A value may contain =; only the first = on a line separates the key from the value.

Existing credential Secrets created without schema continue to work unchanged. Only set schema if your HCS IAM endpoint uses http instead of the default https.

You can reuse an existing HCS credential Secret. Its name does not need to match the cluster name, but HCSCluster.spec.identityRef.name must reference this Secret.

#Configure Machine Configuration Pool

The HCSMachineConfigPool defines pre-configured hostnames, static IP addresses, and any pool-managed persistent disks for VMs.

Use one subnet selector per networks[] entry. For new manifests, set either subnetName or subnetId, but not both. Existing manifests may keep the deprecated subenetName field; if you also add subnetName while updating that manifest, its value must exactly match subenetName. Do not supply conflicting values across subenetName, subnetName, and subnetId.

If you use subnetName in the machine configuration pool, include the same subnet name in HCSCluster.spec.network.subnets.

For the initial cluster create flow, listing an existing subnet by name is enough because the controller resolves subnet metadata before the cluster becomes Ready. If you later add another subnet to an existing Ready HCSCluster, do not append only name. Patch the parent HCSCluster.spec.network.subnets entry with the full subnet object so later machine or ELB operations can reuse the resolved subnet metadata.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineConfigPool
metadata:
  name: <cluster-name>-cp-pool
  namespace: cpaas-system
  labels:
    cluster.x-k8s.io/cluster-name: <cluster-name>
spec:
  configs:
    - hostname: master-1
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.11
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs
    - hostname: master-2
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.12
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs
    - hostname: master-3
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.13
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs

*For new manifests, set either subnetName or subnetId. Existing manifests may continue to use subenetName, and may add subnetName only if both fields use the same value. Do not provide conflicting subnet selector values.

Persistent disk fields are required when persistentDisks is specified.

Use persistentDisks[] for node-local state that must survive VM replacement. Do not declare the same mount path in HCSMachineTemplate.spec.template.spec.dataVolumes[].

Note: The CRD schema lists subnetName, subenetName, and subnetId as optional fields and does not express their allowed combinations. Follow the provider-level rules above when writing manifests.

Note: To attach multiple NICs to one node, add multiple networks[] entries. The provider only uses these entries to attach NICs and assign subnet selectors plus static IPs. It does not support declaring per-NIC roles, default gateways, static routes, or per-NIC DNS settings.

#Hostname behavior on the node

The provider derives the node's POSIX hostname and FQDN from hostname as follows:

The dotted form is the path to set up applications that depend on FQDN resolution (hostname -f, certificates with SAN entries, log labels). The provider sets prefer_fqdn_over_hostname: false and enables cloud-init manage_etc_hosts only when a dotted hostname is supplied, so POSIX tools continue to see the short name and hostname -f returns the full FQDN.

Invalid hostnames (leading dot, trailing dot, all-dot strings, uppercase letters, or anything that violates the field constraint) are rejected before the VM boots. The error is set on the owning Machine.status and surfaced on the Cluster.status.conditions so it shows up under kubectl describe cluster <name> and kubectl get machines -n cpaas-system -o wide.

#Configure Machine Template

The HCSMachineTemplate defines the VM specifications for control plane nodes.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineTemplate
metadata:
  name: <cluster-name>-control-plane
  namespace: cpaas-system
spec:
  template:
    spec:
      imageName: <vm-image-name>
      flavorName: <instance-flavor>
      availabilityZone: <availability-zone>
      rootVolume:
        type: SSD
        size: 100
      configPoolRef:
        name: <cluster-name>-cp-pool
      dataVolumes:
        - size: 10
          type: SSD
          mountPath: /var/lib/etcd
          format: xfs
        - size: 100
          type: SSD
          mountPath: /var/lib/kubelet
          format: xfs
        - size: 100
          type: SSD
          mountPath: /var/lib/containerd
          format: xfs

*Required when dataVolumes is specified.

dataVolumes[] are recreated with the ECS. Do not use them for /var/cpaas or any other path that must survive rolling replacement. Put those paths in HCSMachineConfigPool.spec.configs[].persistentDisks[].

Note: Do not set runtime identity fields such as providerID or serverId in HCSMachineTemplate manifests. The provider assigns these values when it creates HCS instances.

Note: The ACP web UI loads flavors and availability zones through the selected HCS credential. When you write YAML, get the exact provider-recognized Flavor.Name and ZoneName values from the HCS administrator because the native HCS tenant console might not expose them.

#Configure KubeadmControlPlane

The KubeadmControlPlane defines the Kubernetes control plane configuration.

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>-kcp
  namespace: cpaas-system
spec:
  replicas: 3
  version: <kubernetes-version>
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
  kubeadmConfigSpec:
    files:
      - path: /etc/kubernetes/admission/psa-config.yaml
        owner: root:root
        permissions: "0644"
        content: |
          apiVersion: apiserver.config.k8s.io/v1
          kind: AdmissionConfiguration
          plugins:
          - name: PodSecurity
            configuration:
              apiVersion: pod-security.admission.config.k8s.io/v1
              kind: PodSecurityConfiguration
              defaults:
                enforce: "privileged"
                enforce-version: "latest"
                audit: "baseline"
                audit-version: "latest"
                warn: "baseline"
                warn-version: "latest"
              exemptions:
                usernames: []
                runtimeClasses: []
                namespaces:
                - kube-system
                - cpaas-system
      - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
        owner: root:root
        permissions: "0644"
        content: |
          {
            "apiVersion": "kubelet.config.k8s.io/v1beta1",
            "kind": "KubeletConfiguration",
            "imagePullCredentialsVerificationPolicy": "NeverVerify",
            "protectKernelDefaults": true,
            "tlsCertFile": "/etc/kubernetes/pki/kubelet.crt",
            "tlsPrivateKeyFile": "/etc/kubernetes/pki/kubelet.key",
            "streamingConnectionIdleTimeout": "5m",
            "clientCAFile": "/etc/kubernetes/pki/ca.crt"
          }
      - path: /etc/kubernetes/encryption-provider.conf
        owner: root:root
        permissions: "0600"
        content: |
          apiVersion: apiserver.config.k8s.io/v1
          kind: EncryptionConfiguration
          resources:
          - resources:
            - secrets
            providers:
            - aescbc:
                keys:
                - name: key1
                  secret: bootstrap-placeholder
      - path: /etc/kubernetes/audit/policy.yaml
        owner: root:root
        permissions: "0644"
        content: |
          apiVersion: audit.k8s.io/v1
          kind: Policy
          rules:
          - level: Metadata
    clusterConfiguration:
      imageRepository: <image-repository>
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      apiServer:
        extraArgs:
          audit-log-format: json
          audit-log-mode: batch
          audit-log-path: /etc/kubernetes/audit/audit.log
          audit-policy-file: /etc/kubernetes/audit/policy.yaml
          admission-control-config-file: /etc/kubernetes/admission/psa-config.yaml
          encryption-provider-config: /etc/kubernetes/encryption-provider.conf
          kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
          profiling: "false"
          tls-min-version: VersionTLS12
        extraVolumes:
          - name: vol-dir-0
            hostPath: /etc/kubernetes
            mountPath: /etc/kubernetes
            pathType: Directory
      controllerManager:
        extraArgs:
          bind-address: "::"
          flex-volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          profiling: "false"
          tls-min-version: VersionTLS12
      scheduler:
        extraArgs:
          bind-address: "::"
          profiling: "false"
          tls-min-version: VersionTLS12
    postKubeadmCommands:
      - chmod 600 /var/lib/kubelet/config.yaml
    initConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          protect-kernel-defaults: "true"
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
    joinConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
  machineTemplate:
    nodeDrainTimeout: 1m
    nodeDeletionTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: HCSMachineTemplate
      name: <cluster-name>-control-plane

The HCS controller also injects files while resolving cloud-init data. It writes /etc/kubernetes/pki/kubelet.crt, /etc/kubernetes/pki/kubelet.key, and /etc/kubernetes/encryption-provider.conf for control plane machines. For the first control plane machine, the controller generates the encryption provider configuration. After the control plane is initialized, it tries to reuse the existing kube-apiserver encryption provider configuration. If you include a bootstrap file at /etc/kubernetes/encryption-provider.conf, treat it as a placeholder because the controller-generated or synchronized file takes precedence.

Note: Configure apiServer.extraArgs and apiServer.extraVolumes together. If the volume is not mounted, kube-apiserver cannot read the files written under /etc/kubernetes.

Note: The rolloutStrategy.rollingUpdate.maxSurge: 0 example above is for highly available static-IP control planes. Keep this setting for fixed-size control plane pools with at least three replicas so replacements happen in a scale-down-then-scale-up order. If you create a single-control-plane HCS cluster (spec.replicas: 1), do not copy the rolloutStrategy block into the create manifest. KubeadmControlPlane validation rejects that scale-in style rollout configuration for a single replica.

Note: HCS also supports creating a single-control-plane cluster by setting spec.replicas: 1 and preparing one control plane config entry in the referenced HCSMachineConfigPool. Treat this as a creation-only topology, and leave the rollout strategy unset in that create manifest. A single-control-plane HCS cluster cannot be upgraded.

Use the OS Support Matrix only for the component versions it explicitly lists, such as coredns and etcd image tags for supported Alauda OS images. It is not a complete source for all HCS manifest values. Before you apply this YAML, also use the approved release baseline for values such as imageRepository, DNS image repository, Kube-OVN version, Kube-OVN join CIDR, Pod CIDR, and Service CIDR.

#Configure HCSCluster

The HCSCluster resource defines the HCS infrastructure configuration.

The HCS provider creates an Elastic Load Balance (ELB) on the HCS platform for the Kubernetes API server. This ELB must keep Hybrid Load Balancing enabled so cluster nodes can also reach the API server through the ELB address.

For the documented HCS workflow, provide vipAddress, elbVirsubnetL4Ips, and elbVirsubnetL7Ips. Each elbVirsubnetL4Ips[].ips and elbVirsubnetL7Ips[].ips entry must contain two IPs.

Leave vipDomainName unset — cluster access by domain is not supported; see the field table below.

List every cluster subnet in spec.network.subnets before you reference it anywhere else. vipSubnetName, elbVirsubnetL4Ips[].subnetName, elbVirsubnetL7Ips[].subnetName, and the subnetName values used by HCSMachineConfigPool must all exist in spec.network.subnets.

For the initial cluster create flow, the controller can resolve existing subnet metadata from name. For an existing Ready cluster, append a full subnet object instead of only name. Include id, and include neutronSubnetId for any subnet that the control plane ELB will use. Keep cidr, gatewayIp, primaryDNS, and secondaryDNS in the subnet inventory as well.

Do not disable Hybrid Load Balancing on the provider-created ELB after the cluster is created. The cluster depends on that ELB mode so nodes can reach the API server through the ELB address.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSCluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
spec:
  # Optional. Include only when you want provider-managed control-plane ECS placement.
  controlPlaneHA:
    enabled: true
    policy: soft-anti-affinity
  controlPlaneLoadBalancer:
    vipAddress: <control-plane-vip-address>
    vipSubnetName: <vip-subnet-name>
    elbVirsubnetL4Ips:
      - subnetName: <subnet-name>
        ips:
          - <l4-ip-1>
          - <l4-ip-2>
    elbVirsubnetL7Ips:
      - subnetName: <subnet-name>
        ips:
          - <l7-ip-1>
          - <l7-ip-2>
  networkType: kube-ovn
  network:
    vpc:
      name: <vpc-name>
    subnets:
      - name: <subnet-name>
    securityGroup:
      name: <security-group-name>
  identityRef:
    name: <credential-secret-name>

Do not include spec.controlPlaneEndpoint in the create manifest. In the HCS create flow, the controller derives and populates this field from spec.controlPlaneLoadBalancer after the HCSCluster is created. Do not set controlPlaneEndpoint manually, and do not add an empty controlPlaneEndpoint object. If controlPlaneEndpoint is explicitly present in the manifest, it must include both host and port.

controlPlaneHA is optional. When you include it, both enabled and policy are required. Use anti-affinity for strict host separation. Use soft-anti-affinity when you prefer host spread but do not want ECS creation to fail only because HCS cannot satisfy the hard placement rule. For planning guidance, including the rolling replacement requirement when enabling the feature on an existing cluster, see Control Plane HA Placement Plan.

#Configure Cluster

The Cluster resource in Cluster API declares the cluster and references the control plane and infrastructure resources.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
  annotations:
    cpaas.io/sentry-deploy-type: Baremetal
    cpaas.io/alb-address-type: ClusterAddress
    capi.cpaas.io/resource-group-version: infrastructure.cluster.x-k8s.io/v1beta1
    capi.cpaas.io/resource-kind: HCSCluster
    cpaas.io/kube-ovn-join-cidr: <kube-ovn-join-cidr>
    cpaas.io/kube-ovn-version: <kube-ovn-version>
  labels:
    cluster-type: HCS
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - <pods-cidr>
    services:
      cidrBlocks:
        - <services-cidr>
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: <cluster-name>-kcp
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: HCSCluster
    name: <cluster-name>

#Deploy Worker Nodes

The control plane alone is not a usable cluster. After the control plane is Ready, create the worker resources in the following order.

Before you prepare worker YAML, complete the HCS input checklist in Infrastructure Resources for Huawei Cloud Stack. In particular, list every worker subnet in HCSCluster.spec.network.subnets, allocate worker IPs from planned free IP ranges, and collect the provider-recognized flavorName and availabilityZone API values. If you add a new worker subnet to an existing Ready cluster, patch HCSCluster.spec.network.subnets with the full subnet object instead of adding only the subnet name.

#Step 1: Configure Machine Configuration Pool

The HCSMachineConfigPool defines the network configuration and any pool-managed persistent disks for worker node VMs. You must plan and configure the IP addresses, hostnames, persistent disk slots, and other parameters before deployment.

Use one subnet selector per networks[] entry. For new manifests, set either subnetName or subnetId, but not both. Existing manifests may keep the deprecated subenetName field; if you also add subnetName while updating that manifest, its value must exactly match subenetName. Do not supply conflicting values across subenetName, subnetName, and subnetId.

If you use subnetName for worker nodes, include the same subnet name in the parent HCSCluster.spec.network.subnets list before you create or scale the worker pool. For an existing Ready cluster, append the full subnet object, including the subnet ID, instead of adding only the subnet name.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineConfigPool
metadata:
  name: <cluster-name>-worker-pool
  namespace: cpaas-system
  labels:
    cluster.x-k8s.io/cluster-name: <cluster-name>
spec:
  configs:
    - hostname: <worker-1-hostname>
      networks:
        - subnetName: <subnet-name>
          ipAddress: <worker-1-ip>
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs
    - hostname: <worker-2-hostname>
      networks:
        - subnetName: <subnet-name>
          ipAddress: <worker-2-ip>
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs
    - hostname: <worker-3-hostname>
      networks:
        - subnetName: <subnet-name>
          ipAddress: <worker-3-ip>
      persistentDisks:
        - slot: 0
          size: 100
          type: SSD
          mountPath: /var/cpaas
          format: xfs

*For new manifests, set either subnetName or subnetId. Existing manifests may continue to use subenetName, and may add subnetName only if both fields use the same value. Do not provide conflicting subnet selector values.

Persistent disk fields are required when persistentDisks is specified.

Use persistentDisks[] for node-local state that must survive worker replacement. Do not declare the same mount path in HCSMachineTemplate.spec.template.spec.dataVolumes[].

Note: The CRD schema lists subnetName, subenetName, and subnetId as optional fields and does not express their allowed combinations. Follow the provider-level rules above when writing manifests.

Note: networks[] can contain more than one entry when a worker node needs multiple NICs. The current provider only uses each entry to attach a NIC with a subnet selector and static IP. It does not support per-NIC role declarations, default gateway selection, static routes, route metrics, or per-NIC DNS settings.

#Pool-Managed Persistent Disks for Workers

Declare worker-node disks that must survive replacement in the matching HCSMachineConfigPool.spec.configs[].persistentDisks[] entry. Use this model for /var/cpaas and for any other node-local state that must be retained during rolling replacement.

• Keep HCSMachineTemplate.spec.template.spec.dataVolumes[] for temporary disks that may be recreated with each ECS.
• Keep slots unique and contiguous from 0 for each hostname. The provider uses (hostname, slot) as the persistent-disk identity.
• Treat slot, size, type, format, and mountPath as immutable after the provider accepts the entry.
• You can update mountOptions. The change takes effect after the worker is replaced.
• You can append new persistentDisks[] entries. The provider creates or claims the EVS disk, but it does not hot-mount the disk into the running ECS. Trigger a rolling replacement with MachineDeployment.spec.strategy.rollingUpdate.maxSurge: 0 before you expect the new disk to be formatted and mounted inside the guest OS.

To inspect persistent-disk runtime state during worker operations, check the pool status:

kubectl get hcsmachineconfigpool <cluster-name>-worker-pool -n cpaas-system -o yaml

#Step 2: Configure Machine Template

The HCSMachineTemplate defines the VM specifications for worker nodes.

Configure worker nodes with a system volume and temporary data volumes for paths that may be recreated with each ECS, such as /var/lib/kubelet and /var/lib/containerd. Put /var/cpaas in HCSMachineConfigPool.spec.configs[].persistentDisks[] when platform state must survive worker replacement.

Use the provider-recognized flavorName and availabilityZone API values when you prepare the worker template. These values are not the tenant UI display names.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineTemplate
metadata:
  name: <cluster-name>-worker-template
  namespace: cpaas-system
spec:
  template:
    spec:
      imageName: <vm-image-name>
      flavorName: <instance-flavor>
      availabilityZone: <availability-zone>
      rootVolume:
        type: SSD
        size: 100
      configPoolRef:
        name: <cluster-name>-worker-pool
      dataVolumes:
        - size: 20
          type: SSD
          mountPath: /var/lib/kubelet
          format: xfs
        - size: 20
          type: SSD
          mountPath: /var/lib/containerd
          format: xfs

*Required when dataVolumes is specified.

dataVolumes[] are recreated with the ECS. Do not use them for /var/cpaas or any other path that must survive rolling replacement.

Note: Do not set runtime identity fields such as providerID or serverId in HCSMachineTemplate manifests. The provider assigns these values when it creates HCS instances.

Note: The ACP web UI loads flavors and availability zones through the selected HCS credential. When you write YAML, get the exact provider-recognized Flavor.Name and ZoneName values from the HCS administrator because the native HCS tenant console might not expose them.

#Step 3: Configure Bootstrap Template

The KubeadmConfigTemplate defines the bootstrap configuration for worker nodes.

imagePullCredentialsVerificationPolicy: NeverVerify is required only starting with Kubernetes 1.35. Omit this parameter when creating workers with Kubernetes 1.34 or earlier.

apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
  name: <cluster-name>-worker-kct
  namespace: cpaas-system
spec:
  template:
    spec:
      files:
        - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
          owner: root:root
          permissions: "0644"
          content: |
            {
              "apiVersion": "kubelet.config.k8s.io/v1beta1",
              "kind": "KubeletConfiguration",
              "imagePullCredentialsVerificationPolicy": "NeverVerify",
              "protectKernelDefaults": true,
              "staticPodPath": null,
              "tlsCertFile": "/etc/kubernetes/pki/kubelet.crt",
              "tlsPrivateKeyFile": "/etc/kubernetes/pki/kubelet.key",
              "streamingConnectionIdleTimeout": "5m",
              "clientCAFile": "/etc/kubernetes/pki/ca.crt"
            }
      postKubeadmCommands:
        - chmod 600 /var/lib/kubelet/config.yaml
      joinConfiguration:
        patches:
          directory: /etc/kubernetes/patches
        nodeRegistration:
          kubeletExtraArgs:
            volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"

The HCS controller injects /etc/kubernetes/pki/kubelet.crt and /etc/kubernetes/pki/kubelet.key while resolving worker cloud-init data. The kubelet patch above configures kubelet to use those controller-provided certificate files.

#Step 4: Configure Machine Deployment

The MachineDeployment orchestrates the creation and management of worker nodes.

apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: <cluster-name>-md-0
  namespace: cpaas-system
spec:
  clusterName: <cluster-name>
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
  template:
    spec:
      clusterName: <cluster-name>
      version: <kubernetes-version>
      nodeDrainTimeout: 1m
      nodeDeletionTimeout: 5m
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: <cluster-name>-worker-kct
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: HCSMachineTemplate
        name: <cluster-name>-worker-template

#Cluster Verification

After deploying all cluster resources, verify that the cluster has been created successfully.

#Using kubectl

# Check cluster status
kubectl get cluster -n cpaas-system <cluster-name>

# Verify control plane nodes
kubectl get kubeadmcontrolplane -n cpaas-system <cluster-name>-kcp

# Check machine status
kubectl get machines -n cpaas-system

# Verify cluster deployment status
kubectl get clustermodule <cluster-name> -o jsonpath='{.status.base.deployStatus}'

#Verify Control Plane HA

If you enabled HCSCluster.spec.controlPlaneHA, inspect the HCSCluster condition first:

kubectl get hcscluster <cluster-name> -n cpaas-system \
  -o jsonpath='{range .status.conditions[?(@.type=="ControlPlaneHAReady")]}{.status}{" "}{.reason}{" "}{.message}{"\n"}{end}'

Interpret the condition as follows:

Inspect the observed server group and membership snapshot:

kubectl get hcscluster <cluster-name> -n cpaas-system \
  -o jsonpath='serverGroup={.status.controlPlaneHA.name}{" id="}{.status.controlPlaneHA.id}{" policy="}{.status.controlPlaneHA.policy}{"\n"}{range .status.controlPlaneHA.members[*]}{.machineName}{" "}{.serverID}{"\n"}{end}'

For anti-affinity, HCS treats the server group policy as a hard scheduling constraint. If capacity is insufficient, ECS creation can fail and the condition message should be used as the first diagnostic signal. For soft-anti-affinity, HCS tries to spread members but may still place multiple ECS instances on the same host.

#Expected Results

A successfully created cluster should show:

• Cluster status: Running or Provisioned
• All control plane machines: Running
• Kubernetes nodes: Ready
• Cluster Module Status: Completed

#Managing Nodes

For day-2 node operations such as scaling, template updates, and web UI node pools, refer to Manage Nodes on Huawei Cloud Stack.

#Upgrading Clusters

For instructions on upgrading cluster components, refer to Upgrade Kubernetes.

#Troubleshooting

If the cluster reaches Provisioned but never becomes Ready — for example, workload nodes stay NotReady because the CNI is not deployed — start with the provider-agnostic Troubleshoot a Workload Cluster Stuck in Provisioned.

For HCS-specific failure patterns (for example, kubeadm init never completes because a dotted HCSMachineConfigPool hostname produced a POSIX hostname containing dots), see Troubleshoot Huawei Cloud Stack Workload Clusters.