Creating an On-Premise Cluster

TOC

Prerequisites

Node Requirements

  1. If you downloaded a single-architecture installation package from Download Installation Package, ensure your node machines have the same architecture as the package. Otherwise, nodes won't start due to missing architecture-specific images.
  2. Verify that your node operating system and kernel are supported. See Supported OS and Kernels for details.
  3. Perform availability checks on node machines. For specific check items, refer to Node Preprocessing > Node Checks.
  4. If node machine IPs cannot be directly accessed via SSH, provide a SOCKS5 proxy for the nodes. The global cluster will access nodes through this proxy service.

Load Balancing

For production environments, a load balancer is required for cluster control plane nodes to ensure high availability. You can provide your own hardware load balancer or enable Self-built VIP, which provides software load balancing using haproxy + keepalived. We recommend using a hardware load balancer because:

  • Better Performance: Hardware load balancing performs better than software load balancing.
  • Lower Complexity: If you're unfamiliar with keepalived, misconfigurations could make the cluster unavailable, leading to lengthy troubleshooting and seriously affecting cluster reliability.

When using your own hardware load balancer, you can use the load balancer's VIP as the IP Address / Domain parameter. If you have a domain name that resolves to the load balancer's VIP, you can use that domain as the IP Address / Domain parameter. Note:

  • The load balancer must correctly forward traffic to ports 6443, 11780, and 11781 on all control plane nodes in the cluster.
  • If your cluster has only one control plane node and you use that node's IP as the IP Address / Domain parameter, the cluster cannot be scaled from a single node to a highly available multi-node setup later. Therefore, we recommend providing a load balancer even for single-node clusters.

When enabling Self-built VIP, you need to prepare:

  1. An available VRID
  2. A host network that supports the VRRP protocol
  3. All control plane nodes and the VIP must be on the same subnet, and the VIP must be different from any node IP.

Connecting global Cluster and Workload Cluster

The platform requires mutual access between the global cluster and workload clusters. If they're not on the same network, you need to:

  1. Provide External Access for the workload cluster to ensure the global cluster can access it. Network requirements must ensure global can access ports 6443, 11780, and 11781 on all control plane nodes.
  2. Add an additional address to global that the workload cluster can access. When creating a workload cluster, add this address to the cluster's annotations with the key cpaas.io/platform-url and the value set to the public access address of global.

Image Registry

Cluster images support Platform Built-in, Private Repository, and Public Repository options.

  • Platform Built-in: Uses the image registry provided by the global cluster. If the cluster cannot access global, see Add External Address for Built-in Registry.
  • Private Repository: Uses your own image registry. For details on pushing required images to your registry, contact technical support.
  • Public Repository: Uses the platform's public image registry. Before using, complete Updating Public Repository Credentials.

Container Networking

If you plan to use Kube-OVN's Underlay for your cluster, refer to Preparing Kube-OVN Underlay Physical Network.

Creation Procedure

  1. Enter the Platform Management view, and click Clusters/Clusters in the left navigation bar.

  2. Click Create Cluster.

  3. Configure the following sections according to the instructions below: Basic Info, Container Network, Node Settings, and Extended Parameters.

Basic Info

ParameterDescription
Kubernetes Version

All optional versions are rigorously tested for stability and compatibility.


Recommendation: Choose the latest version for optimal features and support.

Container Runtime

Containerd is provided as the default container runtime.


If you prefer using Docker as the container runtime, please refer to Choosing a Container Runtime.

Cluster Network Protocol

Supports three modes: IPv4 single stack, IPv6 single stack, IPv4/IPv6 dual stack.


Note: If you select dual stack mode, ensure all nodes have correctly configured IPv6 addresses; the network protocol cannot be changed after setting.

Cluster Endpoint

IP Address / Domain: Enter the pre-prepared domain name or VIP if no domain name is available.


Self-Built VIP: Disabled by default. Only enable if you haven't provided a LoadBalancer. When enabled, the installer will automatically deploy keepalived for software load balancing support.


External Access: Enter the externally accessible address prepared for the cluster when it's not in the same network environment as the global cluster.

Container Network

Kube-OVN
Calico
Flannel
Custom

An enterprise-grade Cloud Native Kubernetes container network orchestration system developed by Alauda. It brings mature networking capabilities from the OpenStack domain to Kubernetes, supporting cross-cloud network management, traditional network architecture and infrastructure interconnection, and edge cluster deployment scenarios, while greatly enhancing Kubernetes container network security, management efficiency, and performance.

ParameterDescription
Subnet

Also known as Cluster CIDR, represents the default subnet segment. After cluster creation, additional subnets can be added.

Transmit Mode

Overlay: A virtual network abstracted over the infrastructure that doesn't consume physical network resources. When creating an Overlay default subnet, all Overlay subnets in the cluster use the same cluster NIC and node NIC configuration.
Underlay: This transmission method relies on physical network devices. It can directly allocate physical network addresses to Pods, ensuring better performance and connectivity with the physical network. Nodes in an Underlay subnet must have multiple NICs, and the NIC used for bridge networking must be exclusively used by Underlay and not carry other traffic like SSH. When creating an Underlay default subnet, the cluster NIC is actually a default NIC for bridge networking, and the node NIC is the node NIC configuration in the bridge network.

  • Default Gateway: The physical network gateway address, which is the gateway address for the Cluster CIDR segment (must be within the Cluster CIDR address range).
  • VLAN ID: Virtual LAN identifier (VLAN number), e.g., 0.
  • Reserved IPs: Set reserved IPs that won't be automatically allocated, such as IPs in the subnet that are already used by other devices.
Service CIDR

IP address range used by Kubernetes Services of type ClusterIP. Cannot overlap with the default subnet range.

Join CIDR

In Overlay transmission mode, this is the IP address range used for communication between nodes and pods. Cannot overlap with the default subnet or Service CIDR.

Node Settings

ParameterDescription
Network Interface Card

The name of the host network interface device used by the cluster network plugin.


Note:

  • When selecting Underlay transmission mode for the Kube-OVN default subnet, you must specify the network interface name, which will be the default NIC for bridge networking.
    - The platform's network interface traffic monitoring by default recognizes traffic on interfaces named like eth.|en.|wl.|ww.. If you use interfaces with different naming conventions, please refer to Collect Network Data from Custom-Named Network Interfaces after cluster onboarding to modify the relevant resources and ensure the platform can properly monitor network interface traffic.
Node Name

You can choose to use either the node IP or hostname as the node name on the platform.


Note: When choosing to use hostname as the node name, ensure that the hostnames of nodes added to the cluster are unique.

Nodes

Add nodes to the cluster, or Recovery from draft temporarily saved node information. See the detailed parameter descriptions for adding nodes below.

Monitoring Type

Supports Prometheus and VictoriaMetrics.
When selecting VictoriaMetrics as the monitoring component, you must configure the Deploy Type:
- Deploy VictoriaMetrics: Deploys all related components, including VMStorage, VMAlert, VMAgent, etc.


- Deploy VictoriaMetrics Agent: Only deploys the log collection component, VMAgent. When using this deployment method, you need to associate with a VictoriaMetrics instance already deployed on another cluster in the platform to provide monitoring services for the cluster.

Monitoring Nodes

Select nodes for deploying cluster monitoring components. Supports selecting compute nodes and control plane nodes that allow application deployment.


To avoid affecting cluster performance, it's recommended to prioritize compute nodes. After the cluster is successfully created, monitoring components with storage type Local Volume will be deployed on the selected nodes.

Node Addition Parameters

ParameterDescription
Type

Control Plane Node: Responsible for running components such as kube-apiserver, kube-scheduler, kube-controller-manager, etcd, container network, and some platform management components in the cluster. When Application Deployable is enabled, control plane nodes can also be used as compute nodes.


Worker Node: Responsible for hosting business pods running on the cluster.

IPv4 Address

The IPv4 address of the node. For clusters created in internal network mode, enter the node's private IP.

IPv6 Address

Valid when the cluster has IPv4/IPv6 dual stack enabled. The IPv6 address of the node.

Application Deployable

Valid when Node Type is Control Plane Node. Whether to allow business applications to be deployed on this control plane node, scheduling business-related pods to this node.

Display NameThe display name of the node.
SSH Connection IP

The IP address that can connect to the node when accessing it via SSH service.


If you can log in to the node using ssh <username>@<node's IPv4 address>, this parameter is not required; otherwise, enter the node's public IP or NAT external IP to ensure the global cluster and proxy can connect to the node via this IP.

Network Interface Card

Enter the name of the network interface used by the node. The priority of network interface configuration effectiveness is as follows (from left to right, in descending order):


Kube-OVN Underlay: Node NIC > Cluster NIC


Kube-OVN Overlay: Node NIC > Cluster NIC > NIC corresponding to the node's default route


Calico: Cluster NIC > NIC corresponding to the node's default route


Flannel: Cluster NIC > NIC corresponding to the node's default route

Associated Bridge Network

Note: When creating a cluster, bridge network configuration is not supported; this option is only available when adding nodes to a cluster that already has Underlay subnets created.


Select an existing Add Bridge Network. If you don't want to use the bridge network's default NIC, you can configure the node NIC separately.

SSH Port

SSH service port number, e.g., 22.

SSH Username

SSH username, needs to be a user with root privileges, e.g., root.

Proxy

Whether to access the node's SSH port through a proxy. When the global cluster cannot directly access the node to be added via SSH (e.g., the global cluster and workload cluster are not in the same subnet; the node IP is an internal IP that the global cluster cannot directly access), this switch needs to be turned on and proxy-related parameters configured. After configuring the proxy, node access and deployment can be achieved through the proxy.


Note: Currently, only SOCKS5 proxy is supported.


Access URL: Proxy server address, e.g., 192.168.1.1

.
Username: Username for accessing the proxy server.


Password: Password for accessing the proxy server.

SSH Authentication

Authentication method and corresponding authentication information for logging into the added node. Options include:


Password: Requires a username with root privileges and the corresponding SSH password.
Key: Requires a private key with root privileges and the private key password .

Save Draft

Saves the currently configured data in the dialog as a draft and closes the Add Node dialog.


Without leaving the Create Cluster page, you can select Restore from draft to open the Add Node dialog and restore the configuration data saved as a draft.


Note: The data restored from the draft is the most recently saved draft data.

Extended Parameters

Note:

  • Apart from required configurations, it's not recommended to set extended parameters, as incorrect settings may make the cluster unavailable and cannot be modified after cluster creation.

  • If a entered Key duplicates a default parameter Key, it will override the default configuration.

Procedure

  1. Click Extended Parameters to expand the extended parameter configuration area. You can optionally set the following extended parameters for the cluster:
ParameterDescription
Docker Parameters

dockerExtraArgs, additional configuration parameters for Docker, which will be written to /etc/sysconfig/docker. Modification is not recommended. To configure Docker through the daemon.json file, it must be configured as key-value pairs.

Kubelet Parameters

kubeletExtraArgs, additional configuration parameters for Kubelet.


Note: When the Container Network's Node IP Count parameter is entered, a default Kubelet Parameter configuration with the key max-pods and a value of Node IP Count is automatically generated. This sets the maximum number of pods that can run on any node in the cluster. This configuration is not displayed in the interface.


Adding a new max-pods: maximum number of runnable pods key-value pair in the Kubelet Parameters area will override the default value. Any positive integer is allowed, but it's recommended to use the default value (Node IP Count) or enter a value not exceeding 256.

Controller Manager Parameters

controllerManagerExtraArgs, additional configuration parameters for the Controller Manager.

Scheduler Parameters

schedulerExtraArgs, additional configuration parameters for the Scheduler.

APIServer Parameters

apiServerExtraArgs, additional configuration parameters for the APIServer.

APIServer URL

publicAlternativeNames, APIServer access addresses issued in the certificate. Only IPs or domain names can be entered, with a maximum of 253 characters.

Cluster Annotations

Cluster annotation information, marking cluster characteristics in metadata in the form of key-value pairs for platform components or business components to obtain relevant information.

  1. Click Create. You'll return to the cluster list page where the cluster will be in the Creating state.

Post-Creation Steps

Viewing Creation Progress

On the cluster list page, you can view the list of created clusters. For clusters in the Creating state, you can check the execution progress.

Procedure

  1. Click the small icon View Execution Progress to the right of the cluster status.

  2. In the execution progress dialog that appears, you can view the cluster's execution progress (status.conditions).

    Tip: When a certain type is in progress or in a failed state with a reason, hover your cursor over the corresponding reason (shown in blue text) to view detailed information about the reason (status.conditions.reason).

Associating with Projects

After the cluster is created, you can add it to projects in the project management view.