Add External Address for Built-in Registry

TOC

Overview

When the global cluster uses the Platform Built-in registry, workload clusters typically also use this registry to pull images. The registry not only serves components within the global cluster but must also be accessible to workload cluster nodes.

In certain scenarios, workload cluster nodes cannot directly access the global cluster's registry address - for example, when the global cluster is in a private data center while workload clusters are in public clouds or edge environments.

This guide explains how to configure an externally accessible address for the platform's default registry to allow workload clusters to pull images.

Prerequisites

Before you begin, prepare the following:

  • A domain name accessible by workload cluster nodes
  • The IP address that the domain name points to
  • A valid SSL certificate for the domain name
WARNING
  • The domain name must be different from the platform access address
  • Ensure the domain's IP address can forward traffic to all control plane nodes of the global cluster

Procedure

Configure Certificate and Routing Rules for the Platform Registry

  1. Copy the domain's valid certificate to any control plane node of the global cluster

  2. Create a TLS secret containing the domain certificate:

    kubectl create secret tls registry-address.tls --cert=<certificate-filename> --key=<key-filename> -n kube-system

    Example:

    kubectl create secret tls registry-address.tls --cert=custom.crt --key=custom.key -n kube-system

    Note: After creating the certificate, monitor the expiration date of the registry-address.tls secret in the kube-system namespace of the global cluster. Replace the certificate before it expires.

  3. Create ingress rules on any control plane node of the global cluster:

    REGISTRY_DOMAIN_NAME=<www.registry.com> # Replace with your accessible domain name
    cat << EOF | kubectl create -f -
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: HTTPS
      name: registry-address
      namespace: kube-system
      labels:
        service_name: registry
    spec:
      rules:
        - host: $REGISTRY_DOMAIN_NAME
          http:
            paths:
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/_catalog
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/.+/tags/list
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/.+/manifests/[A-Za-z0-9_+.-:]+
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/.+/blobls/[A-Za-z0-9-:]+
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /v2/.+/blobls/uploads/[A-Za-z0-9-:]+
                pathType: ImplementationSpecific
              - backend:
                  service:
                    name: registry
                    port:
                      number: 443
                path: /auth/token
                pathType: ImplementationSpecific
      tls:
        - secretName: registry-address.tls
          hosts:
            - $REGISTRY_DOMAIN_NAME
    EOF

    A response similar to ... created indicates successful ingress creation.

  4. Check if a Registry Service resource exists:

    kubectl -n kube-system get svc | grep registry

    If the Service doesn't exist, create it with:

    cat << EOF | kubectl create -f -
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        name: registry
        service_name: registry
      name: registry
      namespace: kube-system
    spec:
      ports:
        - protocol: TCP
          port: 443
          targetPort: 60080
      selector:
        component: registry
      type: ClusterIP
    EOF
  5. Test the configuration by pulling an image from the registry using the domain name:

    crictl pull <registry-domain-name>/automation/qaimages:helloworld

    Or

    docker pull <registry-domain-name>/automation/qaimages:helloworld