#Communication Matrix

This page is the single reference for the network access that requires. Other pages link here instead of listing ports themselves. It has two parts:

• Network Flows are the connections that have to be allowed between machines and networks — through site firewalls, security groups, router ACLs, and load balancers. They are listed by the endpoint that receives the traffic, and they apply to every cluster, whatever operating system its nodes run, unless a row says otherwise. Use them to request network and security approvals before you install the global cluster or create a workload cluster.
• Node Ports are the ports that platform components listen on on each node, which is what a host firewall on the node has to allow. This part differs between Alauda OS nodes and non-Alauda OS nodes.

Every port on this page is TCP unless stated otherwise. In the tables, Needed when states the condition under which a row applies. A row marked Always applies to every deployment. If the condition of a row does not describe your deployment, you do not need to open that row. The sections that apply to a single scenario, Installation and Infrastructure Providers, state the scenario in their introduction instead.

#Decide What Applies to You

Your choices decide which rows you need. Settle them before you request network approvals.

#Overview

The diagram shows the steady-state flows between the main parties, plus the two flows of an Alauda OS installation that involve the bootstrap node. The other flows that exist only during installation are listed in Installation. The tables in Network Flows give the source, port, purpose, and condition of each flow.

Diagram: Steady-state network flows between users, the global cluster, a standby global cluster, a workload cluster, an external registry, and an IaaS platform, with the bootstrap node of an Alauda OS installation.

#Network Flows

Most tables list the traffic that arrives at one endpoint; Installation, Infrastructure Providers, and Infrastructure Services list the flows of one scenario. Traffic that a load balancer forwards does not originate from a cluster node IP address, so allow it from the load balancer as well as from the cluster networks. We recommend configuring health checks on the load balancer for each forwarded port.

#Platform Access Address and global VIP

The platform access address is the global VIP, or the domain that resolves to it. If the global cluster also uses an External IP, the rows for 443, 6443, and 11443 apply to it as well.

Configure the load balancer to forward the ports marked Yes in Forward on load balancer from the VIP to all control plane nodes of the global cluster. With Self-built VIP there are no load balancer listeners; see Self-built VIP. In the Allowed from column, Global DR marks the sources that a disaster recovery pair adds; see Global Disaster Recovery.

On Huawei Cloud Stack, the HCS provider creates the global ELB listeners for clusters whose nodes run Alauda OS. See Installing the global Cluster.

#Workload Cluster VIP and Nodes

The global cluster manages every workload cluster, so the global cluster nodes must reach the VIP of each workload cluster and, for non-Alauda OS nodes, the nodes themselves over SSH.

Workload cluster VIP. Configure the load balancer to forward the ports marked Yes in Forward on load balancer from the VIP to all control plane nodes of the workload cluster.

Workload cluster nodes.

The nodes of a workload cluster also reach the platform access address of the global cluster, listed in Platform Access Address and global VIP.

#Self-built VIP

Self-built VIP replaces the external load balancer with keepalived running on the control plane nodes. It configures no load balancer listeners. Instead of a TCP or UDP port, it uses the VRRP protocol (IP protocol number 112) and moves the VIP with gratuitous ARP.

• Allow VRRP between all control plane nodes that share the VIP, and allow gratuitous ARP updates on their subnet. firewalld and ufw block VRRP by default, so on a non-Alauda OS node with a host firewall the VIP never comes up unless you allow it.
• Place all control plane nodes and the VIP in the same subnet. The VIP must differ from every node IP address.
• No load balancer listeners are configured. The flows to the VIP listed on this page still apply.

Clusters whose nodes run a non-Alauda OS support Self-built VIP. For the control plane endpoint modes that clusters whose nodes run Alauda OS support, see Plan the Control Plane Endpoint.

#Image Registry

The platform built-in Registry is served on the global VIP; see the 11443 row in Platform Access Address and global VIP. The temporary registry on the bootstrap node of an Alauda OS installation is listed in Installation. The registries below need their own flows.

#Global Disaster Recovery

A disaster recovery pair needs every flow marked Global DR in Platform Access Address and global VIP. Allow them in both directions between the Primary and Standby cluster networks — on firewalls, security groups, router ACLs, and any other inter-site network controls — and forward them on both cluster VIPs as the Forward on load balancer column of that table says.

The rules must be bidirectional because the cluster roles are reversed after a failover: the former Primary becomes the new Standby and has to reach the new Primary on the same ports. This does not make etcd replication bidirectional — etcd Synchronizer runs only on the current Standby Cluster and writes to its local etcd.

For the procedure, see Global Disaster Recovery. For a global cluster whose nodes run Alauda OS, see Global Cluster Disaster Recovery.

#Within a Cluster

#Installation

These flows are needed only while the global cluster is being installed.

Alauda OS. setup.sh runs on a temporary bootstrap node that never joins the global cluster. In a disaster recovery deployment, each side is installed from its own bootstrap node, so allow these flows for both bootstrap nodes.

For the bootstrap node requirements, see Installing the global Cluster.

Non-Alauda OS. setup.sh runs on a node that becomes a control plane node of the global cluster.

#Infrastructure Providers

Clusters whose nodes run Alauda OS are created through a Cluster API infrastructure provider. The provider runs on the bootstrap node while the global cluster is installed and on the global cluster afterwards, so allow these flows from the bootstrap node and from the global cluster nodes.

If the global cluster nodes have several network interfaces, make sure that each destination is routable from the interface that reaches it.

Bare Metal. Hosts are registered, installed, and joined over the network, so open these flows before the first host boots the SeedImage ISO:

A Bare Metal host also needs DNS, NTP, the registry, its cluster's control plane endpoint, and cluster-internal traffic, as listed in the other sections of this page. A firewall approval that covers only 443 produces hosts that register successfully and then fail during installation or join.

#Infrastructure Services

#What This Matrix Does Not Cover

Check each of the following against your own deployment. A requirement that applies to your cluster but is not listed here still has to be allowed.

• Pod and Service traffic. See Allow Cluster-Internal Traffic.
• Secondary and non-default container networks. The 6081/UDP port in Node Ports is used by the default Kube-OVN Geneve tunnel. If you change the Kube-OVN tunnel type, or attach a secondary network with Multus on top of the default one, those networks carry Pod traffic between node IP addresses using their own encapsulation, which is not listed here. Confirm what your configuration uses and allow it between the nodes.
• Application traffic. Ports exposed by your applications through an ingress controller, a gateway, or NodePort and LoadBalancer Services are defined by those applications.
• Agnostic extensions. Ports required by Agnostic extensions are maintained by those extensions.

#Node Ports

This part lists the ports that core components and Aligned extensions listen on. Some of the components run only on global cluster nodes; a workload cluster node listens on the subset of these ports that applies to the components it runs.

#Allow Cluster-Internal Traffic

#Alauda OS Nodes

The Alauda OS image preconfigures the node host firewall (firewalld), and the platform maintains it. You do not manage these rules. To open an additional port — for example, for a workload that listens directly on the node — use a MachineConfig as described in Managing Firewall Ports.

The preconfigured rules allow the following:

• The Preconfigured on Alauda OS column of Platform Ports and Calico Ports shows which of the ports in those tables the image already allows.
• VRRP is allowed, so Self-built VIP needs no additional node rule.
• The list covers core components and Aligned extensions, so it also includes ports beyond the platform ports, such as those of service mesh, storage, and data services.

Ports required by Agnostic extensions and other third-party components are not preconfigured. If your cluster needs a port that is not preconfigured, open it with a MachineConfig.

#Non-Alauda OS Nodes

You need this section only when a host firewall, such as firewalld or ufw, is enabled on the cluster nodes. Keeping the host firewall disabled is the default and recommended node configuration. Before you enable a host firewall, read Hardened Node Security Configuration, which gives the init.sh arguments and the order in which to apply the configuration.

This section is not the same as the port occupancy check in Node Preprocessing. That check confirms that no other process has already bound the ports the platform needs before the installation starts. This section describes which ports a host firewall must allow. Both apply when a host firewall is enabled.

#Platform Ports

The last column shows whether the Alauda OS image preconfigures the port; on Alauda OS nodes you do not open it yourself.

• Unless a note below says otherwise, every port in the table has to be reachable between the cluster nodes and from the node that runs the installer.
• 443/tcp additionally carries the platform access address, so it has to be reachable by the users and systems that access the platform.
• 80/tcp is in use only when HTTP access to the platform is enabled, which we do not recommend.
• 11443/tcp is in use only when the cluster serves the platform built-in Registry.
• 30000-32767 is the port range reserved for NodePort Services created by user applications. A port in this range is opened on the nodes when an application that uses it is deployed, and it is closed again when the corresponding Service is deleted. Allow this range on the host firewall only if your applications use NodePort Services, and restrict it to the sources those applications are meant to serve.

#Installer Node Ports

The node that runs the installer also uses 8080, 12080, 12443, and 16443. Allow these ports on that node before you start the installation.

#Calico Ports

The following ports are required only when the cluster uses the Calico CNI plugin.

#VRRP for Self-built VIP

When the cluster uses Self-built VIP, allow the VRRP protocol between the control plane nodes that share the VIP. See Self-built VIP.