#Node Preprocessing

Before installing the global cluster, all nodes (control plane nodes and worker nodes) must complete preprocessing.

#Supported OS and Kernel Versions

The following table lists the supported operating systems, their validated versions, and the corresponding tested kernel versions.

The platform enforces the version-matching granularity declared in the support list:

• OS version: Use the distribution release listed below. A different major or minor release is not officially supported unless it is also listed.
• Kernel version: Use an official kernel supplied by the operating system vendor. When the list gives an exact x.y.z-build value, the build suffix can vary but x.y.z must match. When the list names an operating system release without a kernel value, use the official kernel shipped with that release. Do not assume that an arbitrary newer kernel is supported.

#x86

#ARM

#Operating System Support Scope Changes

4.4 introduced Kubernetes 1.35, which removed cgroup v1. Nodes in a cluster running Kubernetes 1.35 or later must use cgroup v2 and Linux kernel 5.8 or later.

Because their kernels are earlier than 5.8, the following operating systems were supported through 4.3 and are no longer in the support scope from 4.4 onward:

• Kylin Linux Advanced Server V10 and V10 SP3 (kernel 4.19)
• Ubuntu 20.04 LTS (kernel 5.4)
• RHEL 7.x and RHEL 8.x (kernel 3.10 and kernel 4.18)
• CentOS 7.x (kernel 3.10)

If your nodes run one of these operating systems, choose one of the following paths. Moving to a newer release from the same operating system vendor is usually the cheapest option, because it keeps your existing vendor relationship, tooling, and operational practices:

• Move RHEL 7.x or RHEL 8.x nodes to RHEL 9.6.
• Move Ubuntu 20.04 LTS nodes to Ubuntu 22.04 LTS.
• Move Kylin Linux Advanced Server V10 or V10 SP3 nodes to Kylin Linux Advanced Server V11 2503.
• Move the nodes to Alauda OS. See the Immutable Infrastructure documentation. Alauda OS is an immutable operating system, so this path changes how the nodes are managed and is usually the most expensive option.
• Stay on 4.3.x until you can complete the operating system change.

CentOS 7.x has no successor in the support list. Move those nodes to one of the supported operating systems above.

#Execute the Quick Configuration Script

The installation package provides a script for quickly configuring nodes.

Unzip the installation package to obtain the init.sh script file in the res directory. Copy the script file to the nodes and ensure that you have root privileges.

Execute the script:

bash init.sh

#Node Checks

The following lists all the checks that must be completed on the nodes. Depending on the node's role, the required checks will vary. For example, some checks apply only to control plane nodes.

Checks are divided into two categories:

• ✅ Indicates a check that must pass.
• ⚠️ Indicates a check that must be met in specific scenarios. Please determine whether the corresponding conditions are met according to the instructions. If they are, you must resolve them.

The following is the list of checks:

• OS and Kernel

  • ✅ The machine's grub boot configuration must have the transparent_hugepage=never parameter.
  • ✅ Check whether the kernel modules ip_vs, ip_vs_rr, ip_vs_wrr, and ip_vs_sh are enabled.
  • ⚠️ If the global cluster plans to use Kube-OVN CNI, the kernel modules geneve and openvswitch must be enabled.
  • ✅ SELinux and AppArmor must be disabled. They cannot be kept enabled on a node that runs a traditional operating system; see Why SELinux and AppArmor Must Be Disabled.
  • ✅ The host firewall must be disabled, unless your security policy requires it to stay enabled. In that case the node must be configured as described in Hardened Node Security Configuration before you install the platform.
    • init.sh disables the host firewall. To keep it enabled, pass the matching --skip- argument to init.sh instead of turning it back on afterwards.
    • Keeping the host firewall enabled is a non-default configuration. The platform does not manage the traditional operating system, so an enabled host firewall can cause unexpected failures on the node or in the cluster. You are responsible for operating and troubleshooting the nodes in this configuration.
    • When the host firewall is enabled, you configure and maintain the port rules on every node. For the ports that the platform requires, see Communication Matrix.
  • ✅ Disable swap .

• Users and Permissions

  • ✅ The node's SSH user has root privileges and can use sudo without the password.
  • ✅ The UseDNS parameter in /etc/ssh/sshd_config must be set to no.
  • ✅ Set the UsePAM parameter in /etc/ssh/sshd_config to no before adding the node, then restart sshd. PAM session policies (such as a forced password change, pam_access, faillock, or pam_limits) can otherwise block the SSH-based node join. After the node reaches the Ready state, you can restore UsePAM yes. On SELinux-enforcing systems that use password authentication, UsePAM no can itself break SSH login; use key-based authentication in that case.
  • ✅ systemctl show --property=DefaultTasksMax must return infinity; a low limit can make busy containers fail to create threads. If it is not infinity, set DefaultTasksMax=infinity in /etc/systemd/system.conf and run systemctl daemon-reexec.

• Node Network

  • ✅ hostname must comply with the following rules:
    • No more than 36 characters.
    • Starts and ends with a letter or number.
    • Contains only lowercase letters, numbers, -, and ., but cannot contain .-, .., or -..
  • ✅ localhost in /etc/hosts must resolve to 127.0.0.1.
  • ✅ The /etc/resolv.conf file must exist and contain nameserver configurations, but must not contain addresses starting with 172 (disable systemd-resolved).
  • ⚠️ The /etc/resolv.conf file should not configure search domains (if you must configure them, see Configure Search Domain).
  • ✅ The machine's IP address cannot be a loopback, multicast, link-local, all-0, or broadcast address.
  • ✅ Executing ip route must return a default route or a route pointing to 0.0.0.0.
  • ✅ The nodes must not occupy the following ports. This check confirms that the ports are free before the installation starts; it is not the list of ports that a host firewall must allow. For that list, see Communication Matrix.
    • Control plane nodes: 2379, 2380, 6443, 10249 ~ 10256
    • Node where the installer is located: 8080, 12080, 12443, 16443, 2379, 2380, 6443, 10249 ~ 10256
    • Worker nodes: 10249 ~ 10256
  • ✅ If the cluster uses Kube-OVN or Calico, ensure that the following ports are not occupied:
    • Kube-OVN: 6641, 6642
    • Calico: 179
  • ⚠️ Ensure that the IP addresses in the network segment 172.17.x.x ~ 172.18.x.x required by nerdctl are not occupied. If the IPs in this network segment are occupied and cannot be changed, please contact technical support.

• Software and Directory Requirements:

  • ✅ Must have the following installed: ip, ss, tar, swapoff, modprobe, sysctl, md5sum, and scp or sftp.
  • ⚠️ If you plan to use local storage TopoLVM or Rook, you need to install lvm2.
  • ✅ The /etc/systemd/system/kubelet.service file is not allowed to exist.
  • ✅ /tmp mount parameters must not contain noexec.
  • ✅ Remove packages that conflict with global cluster components (see Remove Conflicting Packages).
  • ✅ The following files must be deleted if they exist:
    • /var/lib/docker
    • /var/lib/nerdctl
    • /opt/nerdctl/
    • /var/lib/containerd
    • /var/log/pods
    • /var/lib/kubelet/pki

• Cross-Node Checks

  • ✅ Network communication between nodes in the global cluster must not be restricted:
    • In the default configuration, where no host firewall is enabled, there must be no network firewall restrictions between the nodes.
    • When a host firewall is enabled on the nodes, every port in Communication Matrix must be allowed between the nodes, and Pod and Service traffic inside the cluster must not be filtered by port.
  • ✅ The hostname of each node in the cluster must be unique.
  • ✅ The time zones of all nodes must be unified, and the time synchronization error must be ≤ 10 seconds.

#Appendix

#Why SELinux and AppArmor Must Be Disabled

On a node that runs a traditional operating system, SELinux and AppArmor must be disabled. This is a platform requirement for this release, not a hardening recommendation. init.sh disables both, and the node join procedure disables them again when a node joins a cluster.

SELinux. Platform components mount host paths into containers. The image registry installer bind-mounts the host /proc/mounts file into its container, and the registry and object storage components keep their data under host directories such as /cpaas and /etc/kubernetes, which carry the operating system's default file labels. On a traditional operating system the container runtime is configured without SELinux labeling, so these mounts are evaluated against the host policy, and they are denied when SELinux runs in enforcing mode. The container runtime then fails to create the container before its process starts, the image registry component never finishes installing, and the platform installation or upgrade stops.

permissive mode is not an alternative: it changes what is logged, not the requirement. A node that is switched back to enforcing after the installation is outside the supported configuration.

AppArmor. Node preparation disables AppArmor, and the platform is not validated with AppArmor profiles enforced on cluster nodes. Running a node with AppArmor enabled is outside the supported configuration for this release.

A hardened operating system profile, such as the STIG profile provided by Ubuntu Security Guide, enables these mechanisms. Applying such a profile to a cluster node is outside the supported configuration for the same reason.

If your security policy requires these mechanisms to stay enabled. Use Alauda OS, the immutable operating system delivered with the platform. On Alauda OS, SELinux and the host firewall are enabled by default, and the platform maintains the policy and the port rules for you. See the Immutable Infrastructure documentation.

Support for running Red Hat Enterprise Linux 9.6 with SELinux in enforcing mode is planned for a future release. Contact technical support for its status.

#Hardened Node Security Configuration

Use this section only when a mandatory security policy, such as a STIG baseline, requires a host firewall to stay enabled on cluster nodes. Keeping the host firewall disabled is the default and recommended node configuration, and it is what init.sh produces when you run it without arguments. Enabling it is a non-default configuration that you own, so validate it in a non-production environment before you apply it to production nodes.

SELinux and AppArmor are a different case: they must stay disabled. See Why SELinux and AppArmor Must Be Disabled.

#Keep the Host Firewall Enabled Instead of Turning It Back On

init.sh can skip individual steps. Pass the matching --skip- argument so that the script never disables the host firewall your policy requires:

For example, to keep firewalld enabled on a RHEL node:

bash init.sh --skip-disable_firewalld

This is the recommended path. The node keeps the state it was installed with, and init.sh still performs every other node preparation step.

#Order of Operations

Configure the node before the platform installation, not after. A firewall problem then surfaces while the installer runs, instead of after the cluster carries workloads.

1. Run init.sh with the --skip- argument for the host firewall implementation you use.
2. Configure the host firewall rules.
3. Verify the resulting state on the node with the status command of your firewall implementation, for example firewall-cmd --list-all or ufw status verbose.
4. Install the platform.

#If a Node Already Ran the Full init.sh

Use this path only when the host firewall was already disabled and you have to turn it back on. It is more work than skipping the step in the first place.

1. Enable and start the service, for example with systemctl enable --now firewalld, or with ufw enable on Ubuntu.
2. Configure the port rules as described below.
3. Confirm the resulting rule set with firewall-cmd --list-all or ufw status verbose.

#Host Firewall

The platform does not require a specific host firewall implementation. RHEL and Kylin Linux Advanced Server nodes typically use firewalld, and Ubuntu nodes typically use ufw. Whichever implementation you use, allow the same set of ports. For the port list, see Communication Matrix.

The node that runs the installer also uses 8080, 12080, 12443, and 16443. Allow these ports on that node before you start the installation.

#Remove Conflicting Packages

Before installation, applications may already be running in the docker/nerdctl/containerd environment on the nodes, or software conflicting with the global cluster may have been installed. Therefore, it is necessary to check and uninstall conflicting packages.

The following commands can be used for reference.

for x in \
    docker docker-client docker-common docker-latest \
    podman-docker podman \
    runc \
    containernetworking-plugins \
    apptainer \
    kubernetes kubernetes-master kubernetes-node kubernetes-client \
    ; do
    rpm -qa | grep -F "$x"
done

for x in \
    docker docker-client docker-common docker-latest \
    podman-docker podman \
    runc \
    containernetworking-plugins \
    apptainer \
    kubernetes kubernetes-master kubernetes-node kubernetes-client \
    ; do
    yum remove "$x"
done

for x in \
    docker.io \
    podman-docker \
    containerd \
    rootlesskit \
    rkt \
    containernetworking-plugins \
    kubernetes \
    ; do
    dpkg-query -l | grep -F "$x"
done

for x in \
    kubernetes-worker \
    kubectl kube-proxy kube-scheduler kube-controller-manager kube-apiserver \
    k8s microk8s \
    kubeadm kubelet \
    ; do
    snap list | grep -F "$x"
done

for x in \
    docker.io \
    podman-docker \
    containerd \
    rootlesskit \
    rkt \
    containernetworking-plugins \
    kubernetes \
    ; do
    apt-get purge "$x"
done

for x in \
    kubernetes-worker \
    kubectl kube-proxy kube-scheduler kube-controller-manager kube-apiserver \
    k8s microk8s \
    kubeadm kubelet \
    ; do
    snap remove --purge "$x"
done

for x in \
    docker docker-client docker-common \
    docker-engine docker-proxy docker-runc \
    podman-docker podman \
    containernetworking-plugins \
    apptainer \
    containerd \
    kubernetes kubernetes-master kubernetes-node kubernetes-client kubernetes-kubeadm \
    ; do
    rpm -qa | grep -F "$x"
done

for x in \
    docker docker-client docker-common \
    docker-engine docker-proxy docker-runc \
    podman-docker podman \
    containernetworking-plugins \
    apptainer \
    containerd \
    kubernetes kubernetes-master kubernetes-node kubernetes-client kubernetes-kubeadm \
    ; do
    yum remove "$x"
done

#Configure Search Domain

In Linux OS, the /etc/resolv.conf file is used to configure DNS client domain name resolution settings. The search line specifies the domain search path for DNS queries.

Configuration Requirements

• Number of Domains: The number of domains in the search line should be less than domainCountLimit - 3 (default domainCountLimit is 32).
• Length of Single Domain: Each domain name must not exceed 253 characters.
• Total Character Length: The total character count of all domain names and spaces must not exceed MaxDNSSearchListChar (default is 2048).

Example

search domain1.com domain2.com domain3.com

• The total number of domains is 3.
• The length of a single domain, such as domain1.com, is 11.
• The total character length is 35, i.e., 11 + 11 + 11 + 2 (two spaces).