#Configure Subnets

#IP Allocation Rules

• Project Allocation:

  • If a project is not bound to a subnet, Pods in all namespaces under that project can only use IP addresses from the default subnet. If there are insufficient IP addresses in the default subnet, the Pods will not be able to start.
  • If a project is bound to a subnet, Pods in all namespaces under that project can only use IP addresses from that specific subnet.

• Namespace Allocation:

  • If a namespace is not bound to a subnet, Pods in that namespace can only use IP addresses from the default subnet. If there are insufficient IP addresses in the default subnet, the Pods will not be able to start.
  • If a namespace is bound to a subnet, Pods in that namespace can only use IP addresses from that specific subnet.

#Calico Network

Creating subnets in the Calico network to achieve finer granularity of network isolation for resources within the cluster.

#Constraints and Limitations

In an IPv6 cluster environment, the subnets created within the Calico network, by default, use VXLAN encapsulation. The ports required for VXLAN encapsulation differ from those of IPIP encapsulation. You need to ensure that UDP port 4789 is open.

#Example Subnet custom resource (CR) with Calico Network

# test-calico-subnet.yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: test-calico
spec:
  cidrBlock: 10.1.1.1/24
  default: false
  ipipMode: Always
  natOutgoing: true
  private: false
  protocol: Dual
  v4blockSize: 30

#Creating a Subnet in the Calico network by using the web console

1. Go to Platform Management.

2. In the left navigation bar, click Network Management > Subnets.

3. Click Create Subnet.

4. Refer to the following instructions to configure the relevant parameters.

   Parameter	Description
   CIDR	After allocating the subnet to a project or namespace, the container groups within the namespace will randomly use IP addresses within this CIDR for communication. Note: For the correspondence between CIDR and BlockSize, please refer to Reference Content.
   Encapsulation Protocol	Select the encapsulation protocol. IPIP is not supported in dual-stack mode. IPIP: Implements inter-segment communication using the IPIP protocol. VXLAN (Alpha): Implements inter-segment communication using the VXLAN protocol. No Encapsulation: Directly connected through routing forwarding.
   Encapsulation Mode	When the encapsulation protocol is IPIP or VXLAN, the encapsulation mode must be set, defaulting to Always. Always: Always enable IPIP / VXLAN tunnels. Cross Subnet: Enable IPIP / VXLAN tunnels only when the host is in different subnets; direct connection via routing forwarding when the host is in the same subnet.
   Outbound Traffic NAT	Choose whether to enable outbound traffic NAT (Network Address Translation), which is enabled by default. It is primarily used to set the access addresses exposed to the external network when the subnet container group accesses the external network. When outbound traffic NAT is enabled, the host IP will be used as the access address for the current subnet container group; when not enabled, the IPs of the container groups in the subnet will be directly exposed to the external network.

5. Click Confirm.

6. On the subnet details page, select Actions > Allocate Project / Allocate Namespace.

7. Complete the configuration and click Allocate.

#Creating a Subnet in the Calico network by using the CLI

kubectl apply -f test-calico-subnet.yaml

#Reference Content

The dynamic matching relationship between CIDR and blockSize is shown in the table below.

#Kube-OVN Network

Creating a subnet in the Kube-OVN Overlay Network to achieve more granular network isolation of resources in the cluster.

#Example Subnet custom resource (CR) with Kube-OVN Overlay Network

# test-overlay-subnet.yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: test-overlay-subnet
spec:
  default: false
  protocol: Dual
  cidrBlock: 10.1.0.0/23
  natOutgoing: true
  excludeIps:
    - 10.1.1.2
  gatewayType: distributed
  gatewayNode: ""
  private: false
  enableEcmp: false

#Creating a Subnet in the Kube-OVN Overlay Network by using the web console

1. Go to Platform Management.

2. In the left navigation bar, click on Network Management > Subnet.

3. Click on Create Subnet.

4. Refer to the following instructions to configure the related parameters.

   Parameter	Description
   Network Segment	After assigning the subnet to the project or namespace, IPs within this segment will be randomly allocated for use by Pods.
   Reserved IP	The set reserved IP will not be automatically allocated. For example, it can be used as the IP address for computing components' fixed IP.
   Gateway Type	Select the type of gateway for the subnet to control the outbound traffic. - Distributed: Each host in the cluster can act as an outbound node for Pods on the current host, enabling distributed egress. - Centralized: All Pods in the cluster use one or more specific hosts as outbound nodes, facilitating external auditing and firewall control. Setting multiple centralized gateway nodes can achieve high availability.
   ECMP (Alpha)	When choosing a Centralized gateway, the ECMP feature can be used. By default, the gateway operates in master-slave mode, with only the master gateway processing traffic. When enabling ECMP (Equal-Cost Multipath Routing), outbound traffic will be routed through multiple equal-cost paths to all available gateway nodes, thereby increasing the total throughput of the gateway. Note: Please enable ECMP-related features in advance.
   Gateway Nodes	When using a Centralized gateway, select one or more specific hosts as gateway nodes.
   Outbound Traffic NAT	Choose whether to enable outbound traffic NAT (Network Address Translation). By default, it is enabled. It is mainly used to set the access address exposed to the external network when the Pods in the subnet access the internet. When outbound traffic NAT is enabled, the host IP will be used as the access address for the Pods in the current subnet; when not enabled, the IPs of the Pods within the subnet will be directly exposed to the external network. In this case, using a centralized gateway is recommended.

5. Click Confirm.

6. On the subnet details page, select Actions > Allocate Project / Namespace.

7. Complete the configuration and click Allocate.

#Creating a Subnet in the Kube-OVN Overlay Network by using the the CLI

kubectl apply -f test-overlay-subnet.yaml

#Underlay Network

Creating subnets in the Kube-OVN Underlay network not only enables finer-grained network isolation for resources but also provides a better performance experience.

#Usage Instructions

The general process for creating subnets in the Kube-OVN Underlay network is: Add Bridge Network > Add VLAN > Create Subnet.

#Add Bridge Network by using the web console (Optional)

# test-provider-network.yaml
kind: ProviderNetwork
apiVersion: kubeovn.io/v1
metadata:
  name: test-provider-network
spec:
  defaultInterface: eth1
  customInterfaces:
    - interface: eth2
      nodes:
        - node1
  excludeNodes:
    - node2

A bridge network refers to a bridge, and after binding the network card to the bridge, it can forward container network traffic, achieving intercommunication with the physical network.

Procedure:

1. Go to Platform Management.

2. In the left navigation bar, click Network Management > Bridge Network.

3. Click Add Bridge Network.

4. Configure the relevant parameters based on the following instructions.

   Note:

   • Target Pod refers to all Pods scheduled on the current node or Pods in namespaces bound to specific subnets scheduled to the current node. This depends on the scope of the subnet under the bridge network.

   • The nodes in the Underlay subnet must have multiple network cards, and the network card used by the bridge network must be exclusively assigned to the Underlay and cannot carry other traffic, such as SSH. For example, if the bridge network has three nodes planning for eth0, eth0, eth1 for exclusive use by the Underlay, then the default network card can be set as eth0, and the network card for node three can be eth1.

   Parameter	Description
   Default Network Card Name	By default, the target Pod will use this as the bridge network card for intercommunication with the physical network.
   Configure Network Card by Node	The target Pods on the configured nodes will bridge to the specified network card instead of the default network card.
   Exclude Nodes	When nodes are excluded, all Pods scheduled to these nodes will not bridge to any network card on these nodes. Note: Pods on excluded nodes will not be able to communicate with the physical network or cross-node container networks, and care should be taken to avoid scheduling related Pods to these nodes.

5. Click Add.

#Add Bridge Network by using the CLI

kubectl apply -f test-provider-network.yaml

#Add VLAN by using the web console (Optional)

# test-vlan.yaml
kind: Vlan
apiVersion: kubeovn.io/v1
metadata:
  name: test-vlan
spec:
  id: 0
  provider: test-provider-network

The platform has a pre-configured ovn-vlan virtual LAN, which will connect to the provider bridge network. You can also configure a new VLAN to connect to other bridge networks, thereby achieving network isolation between VLANs.

Procedure:

1. Navigate to Platform Management.

2. In the left navigation bar, click Network Management > VLAN.

3. Click Add VLAN.

4. Configure the relevant parameters based on the following instructions.

   Parameter	Description
   VLAN ID	The unique identifier for this VLAN, which will be used to differentiate different virtual LANs.
   Bridge Network	The VLAN will connect to this bridge network for intercommunication with the physical network.

5. Click Add.

#Add VLAN by using the CLI

kubectl apply -f test-vlan.yaml

#Example Subnet custom resource (CR) with Kube-OVN Underlay Network

# test-underlay-network.yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: test-underlay-network
spec:
  default: false
  protocol: Dual
  cidrBlock: 11.1.0.0/23
  gateway: 11.1.0.1
  excludeIps:
    - 11.1.0.3
  private: false
  allowSubnets: []
  vlan: test-vlan
  enableEcmp: false

#Creating a Subnet in the Kube-OVN Underlay Network by using the web console

Procedure:

1. Navigate to Platform Management.

2. In the left navigation bar, click Network Management > Subnet.

3. Click Create Subnet.

4. Configure the relevant parameters based on the following instructions.

   Parameter	Description
   VLAN	The VLAN to which the subnet belongs.
   Subnet	After assigning the subnet to a project or namespace, IPs within the physical subnet will be randomly allocated for use by Pods.
   Gateway	The physical gateway within the above subnet.
   Reserved IP	The specified reserved IP will not be automatically assigned. For example, it can be used as the IP for the compute component fixed IP.

5. Click Confirm.

6. On the subnet details page, select Action > Assign Project / Namespace.

7. Complete the configuration and click Assign.

#Creating a Subnet in the Kube-OVN Underlay Network by using the CLI

kubectl apply -f test-underlay-network.yaml

#Related Operations

When both Underlay and Overlay subnets exist in a cluster, you can configure the Automatic Intercommunication Between Underlay and Overlay Subnets as needed.

#Subnet Management

#Updating Gateway by using the web console

This includes changing the outbound traffic method, gateway nodes, and NAT configuration.

1. Go to Platform Management.

2. In the left sidebar, click on Network Management > Subnets.

3. Click the name of the subnet.

4. Select Action > Update Gateway.

5. Update the parameter configurations; refer to the Parameter Description for details.

6. Click OK.

#Updating Gateway by using the CLI

kubectl patch subnet test-overlay-subnet --type=json -p='[
  {"op": "replace", "path": "/spec/gatewayType", "value": "centralized"},
  {"op": "replace", "path": "/spec/gatewayNode", "value": "192.168.66.210"},
  {"op": "replace", "path": "/spec/natOutgoing", "value": true},
  {"op": "replace", "path": "/spec/enableEcmp", "value": true}
]'

#Updating Reserved IPs by using the web console

The gateway IP cannot be removed from the reserved IPs, while other reserved IPs can be edited, deleted, or added freely.

1. Go to Platform Management.

2. In the left sidebar, click on Network Management > Subnets.

3. Click the name of the subnet.

4. Select Action > Update Reserved IP.

5. After completing the updates, click Update.

#Updating Reserved IPs by using the CLI

kubectl patch subnet test-overlay-subnet --type=json -p='[
  {
    "op": "replace",
    "path": "/spec/excludeIps",
    "value": ["10.1.0.1", "10.1.1.2", "10.1.1.4"]
  }
]'

#Assigning Projects by using the web console

Assigning subnets to specific projects helps teams better manage and isolate network traffic for different projects, ensuring that each project has sufficient network resources.

1. Navigate to Platform Management.

2. In the left sidebar, click on Network Management > Subnets.

3. Click the name of the subnet.

4. Select Action > Assign Project.

5. After adding or removing projects, click Assign.

#Assigning Projects by using the CLI

kubectl patch subnet test-overlay-subnet --type=json -p='[
  {
    "op": "replace",
    "path": "/spec/namespaceSelectors",
    "value": [
      {
        "matchLabels": {
          "cpaas.io/project": "cong"
        }
      }
    ]
  }
]'

#Assigning Namespaces by using the web console

Assigning subnets to specific namespaces allows for finer network isolation.

Note: The assignment process will rebuild the gateway, and outbound data packets will be discarded! Please ensure no business applications are currently accessing external clusters.

1. Navigate to Platform Management.

2. In the left sidebar, click on Network Management > Subnets.

3. Click the name of the subnet.

4. Select Action > Assign Namespace.

5. After adding or removing namespaces, click Assign.

#Assigning Namespaces by using the CLI

kubectl patch subnet test-overlay-subnet --type=json -p='[
  {
    "op": "replace",
    "path": "/spec/namespaces",
    "value": ["cert-manager"]
  }
]'

#Expanding Subnets by using the web console

When the reserved IP range of a subnet reaches its usage limit or is about to be exhausted, it can be expanded based on the original subnet range without affecting the normal operation of existing services.

1. Navigate to Platform Management.

2. In the left sidebar, click on Network Management > Subnets.

3. Click the name of the subnet.

4. Select Action > Expand Subnet.

5. Complete the configuration and click Update.

#Expanding Subnets by using the CLI

kubectl patch subnet test-overlay-subnet --type=json -p='[
  {
    "op": "replace",
    "path": "/spec/cidrBlock",
    "value": "10.1.0.0/22"
  }
]'

#Managing Calico Networks

Support for assigning projects and namespaces; for details, please refer to the project assignment and namespace assignment.

#Delete Subnet by using the web console

1. Go to Platform Management.

2. In the left navigation bar, click Network Management > Subnets.

3. Click ⋮ > Delete, and proceed with the deletion.

#Delete Subnet by using the CLI

kubectl delete subnet test-overlay-subnet