Release Notes

Review release notes with the Kubernetes Support Matrix, Version and Lifecycle, and feature-specific documentation to understand release changes and exact support boundaries.

4.4.0

Issued: 2026-08-18

Features and Enhancements

Kubernetes 1.35 and Upgrade Readiness

4.4 upgrades the platform baseline to Kubernetes 1.35. Before upgrading, verify every production node meets the new Kubernetes requirements: the kernel must be version 5.8 or later, and the node must use cgroup v2. The upgrade preflight check blocks the upgrade until an administrator confirms that these checks are complete.

For the required checks and acknowledgement procedure, see Prepare for an Upgrade and Upgrade a Global Cluster.

More Consistent Platform Installation and Upgrades

4.4 improves installation and upgrade reliability. On a new global cluster, the upgrade-management component is installed automatically. Platform components are also installed in a fixed dependency order, reducing installation and upgrade failures when optional platform services are installed independently.

For upgrade procedures, see Upgrade.

Log Collection Plugin Updates

The following updates are delivered by independently versioned Log Collection plugin releases. The matching plugin release can be published after the ACP 4.4 platform release; install it when the package is available. Its product documentation and release notes will provide the detailed configuration and upgrade procedure.

  • Vector log collection — The Log Collection plugin moves to Vector as the log collector. Vector provides a higher-performance collection path for large log volumes and a migration path from the previous collector.
  • OpenSearch log storage — The matching plugin release replaces its embedded Elasticsearch service with a connection to a customer-provided OpenSearch service. New log data is written to OpenSearch. Existing Elasticsearch data is not automatically migrated and can be retained as read-only historical data. The plugin release documentation will describe supported historical-data handling and the upgrade procedure.

Service Mesh v2

Alauda Service Mesh is versioned and released independently of the platform. Starting with 4.4, deploy Alauda Service Mesh v2. Alauda Service Mesh v1 receives no further releases; do not deploy it on 4.4.

If you already run Alauda Service Mesh v1, upgrading the platform to 4.4 does not migrate the existing mesh. After the platform upgrade, install Alauda Service Mesh v2 and migrate your Service Mesh v1 data to v2. Plan the mesh migration as an activity separate from the platform upgrade.

Alauda Service Mesh v2 product documentation and release notes can be published after the ACP 4.4 platform release, and they will provide the supported v1-to-v2 migration procedure. See Alauda Service Mesh v2 and Alauda Service Mesh.

Platform Image Sources

For new production deployments, 4.4 changes the recommended platform image-source strategy: use an external image registry, either your organization's existing registry or a separately deployed registry. Use it as the central image source for the platform, managed clusters, and plugins.

The platform built-in registry remains supported, but it is no longer the recommended default for new production environments.

For registry requirements, payload upload, installation, validation, upgrade preparation, and troubleshooting, see Prepare an External Platform Image Registry.

The next separately versioned component release in the ACP 4.4 delivery cycle will allow Alauda OS-based Workload Clusters to use a third-party registry independently of the Global Cluster registry. This lets you place the image source nearer to a Workload Cluster in geographically separated environments. The related product documentation and release notes will identify supported scenarios and configuration after the component release is available.

Application Image and Workload Operations

  • Operator-managed Registry v2 for application images — ACP provides an Operator-managed integrated registry for application developers and workloads to push, pull, and manage application images. Administrators install it from OperatorHub and configure storage, namespace-scoped access, and managed ServiceAccount pull secrets. Workloads use the in-cluster Registry service; administrators can expose it to developer machines and CI when required. It supports scheduled image pruning with a configured retention policy; run registry garbage collection separately when you need to reclaim unreferenced storage. The legacy Registry Cluster Plugin remains available for existing legacy deployments. For installation, configuration, access, and cleanup procedures, see Registry v2 administration.

  • Policy-based workload rebalancing — Install the Alauda Build of Descheduler Cluster Plugin when workloads need to be rebalanced after changes in utilization, node configuration, affinity, taints, or topology. The plugin evicts eligible Pods according to the configured policies; for Pods managed by a controller, the default scheduler places the replacements after an eviction. It is not installed by default, does not schedule replacement Pods itself, and respects PodDisruptionBudgets. For installation, policy, and verification guidance, see Workload Rebalancing (Descheduler).

  • In-place Pod resource resizing — ACP 4.4 provides guidance for using the Kubernetes Pod resize subresource to adjust CPU and memory requests and limits on a running Pod when the target cluster supports it. Use kubectl or an API client that supports the subresource. This is not a general web-console editing feature, and resizePolicy determines whether a container must restart; VPA InPlaceOrRecreate can still fall back to Pod recreation. For prerequisites, examples, and limitations, see Adjust Pod Resource Levels Without Pod Disruption.

  • PodDisruptionBudget operational guidance — New operational and API guidance explains how to use minAvailable or maxUnavailable to protect replicated workloads during voluntary disruptions such as node drain, maintenance, and upgrades. PodDisruptionBudgets do not protect workloads from involuntary failures such as node hardware faults. For examples and API details, see Using PodDisruptionBudgets.

Monitoring, Dashboards, and Cost Management

  • Perses Monitoring Dashboards — Create and manage metric dashboards with the Perses dashboard experience, import supported Perses or Grafana dashboard JSON, and migrate existing MonitorDashboard resources when needed. Existing Monitoring Dashboards remain available during the transition. For details, see Perses Monitoring Dashboards.

  • Fleet Monitoring — Platform administrators can see connected clusters, monitoring-data freshness, resource capacity and utilization, and project quota allocation and usage in one multi-cluster view. Fleet Monitoring complements, rather than replaces, detailed monitoring and troubleshooting of an individual cluster. For details, see Fleet Monitoring.

  • VictoriaMetrics for Cost Management — Cost Management and metering can use VictoriaMetrics as their metrics data source. Queries are scoped to the relevant cluster, helping ensure that cost and resource-usage data is returned for the correct cluster in multi-cluster environments. For details, see Cost Management.

Networking

  • OVN security group FQDN rules — SecurityGroup rules support domain-name (FQDN) matching, so administrators can define egress policies with domain names instead of only static IP or CIDR targets.

  • OVN database SSL encryption — Connections to the OVN database can be encrypted with SSL, including certificate-based access and certificate rotation, improving control-plane communication security.

  • AdminNetworkPolicy (Technology Preview) — AdminNetworkPolicy adapts to the new API version and adds web console management. The upstream API is still policy.networking.k8s.io/v1alpha1 and continues to evolve, so evaluate this feature within its documented limits instead of treating it as production-supported. For details, see Admin Network Policy.

  • IPVLAN CNI binary — The Kube-OVN delivery now ships the ipvlan CNI binary alongside macvlan, enabling secondary-network-interface scenarios such as VIP egress over an underlay BGP fabric.

  • Gateway and Ingress observability — New monitoring dashboards cover the Ingress Controller and Gateway, and MetalLB dashboards add BGP session state and IP address pool utilization. ingress-nginx adds OpenTelemetry support and authenticated access to its metrics endpoint.

Storage

  • Unified storage component management — The Alauda storage operator becomes the unified entry point for storage components, managing installation, upgrade, status, and dependencies of built-in Ceph, TopoLVM, local storage, and external Ceph.

  • Ceph encryption in transit and at rest — Ceph supports in-transit encryption and KMS-backed at-rest encryption. For details, see In-Transit Encryption and Persistent Volume Encryption.

  • Object storage usability — CephObjectStoreUser quota usage can be viewed, monitored, and alerted on. Bucket claim Secrets expose standard accessKey/secretKey fields that AWS SDKs and common tools can consume directly, and the console shows the Secret associated with each bucket claim.

Virtualization

  • CPU and memory hotplug — Add CPU and memory to a running virtual machine without shutting it down. For procedures and guest OS support details, see CPU and Memory Hotplug.

Global Cluster Disaster Recovery Enhancements

Global Cluster Disaster Recovery remains supported for both traditional operating system deployments and Alauda OS-based Immutable Infrastructure deployments. In 4.4, the synchronization path is more resilient and covers more of the global cluster lifecycle. This capability protects the global control plane; it does not protect application data.

The standby cluster now preserves the active cluster's update order, including when recovery takes long enough for the active etcd to compact its history. The DR configuration can adapt when platform components change without replacing the DR service. The standby cluster also receives global-cluster upgrade state and workload-cluster lifecycle information, so it has the information required to take over more safely.

Before a DR switchover, or before a DR-aware global cluster upgrade removes the synchronizer, continue to validate that the active and standby clusters are consistent. This prevents stale standby data from causing incorrect recovery actions.

For details, see Global Cluster Disaster Recovery and Global Cluster Disaster Recovery on Immutable Infrastructure.

Alauda OS Backup and Restore

Clusters that use Alauda OS can now use Cluster Enhancer for etcd backup and restore. You can run scheduled or on-demand backups, retain them on the control plane nodes, and optionally keep an additional copy in S3-compatible object storage. This protects cluster configuration data; an etcd restore overwrites the existing etcd data and must be performed according to the documented recovery procedure.

For configuration and recovery steps, see etcd Backup and Restore.

Access and Authorization

  • Violet API-token publishingviolet, the CLI used to package and publish plugins, now accepts a platform API token when publishing a package to the platform. Administrators and automation can publish without an interactive login. Download violet from the Customer Portal. For details, see Upload Packages.

  • In-platform CLI downloads — Users can download ACP CLI packages from the web console for Linux (amd64 and arm64), macOS (amd64 and arm64), and Windows (amd64).

  • Safer role delegation — The platform prevents users from granting permissions that exceed their own authorization boundary. The console also includes the related ClusterRole capabilities.

Immutable Infrastructure

4.4 continues to expand the documented Immutable Infrastructure path for Alauda OS, the supported operating system for immutable nodes in the documented provider scenarios. The following capabilities are available in their supported provider scenarios:

  • Bare Metal cluster lifecycle — Create and manage both global and workload clusters on supported Bare Metal environments. The global cluster path includes Global Cluster Disaster Recovery.
  • Control plane and storage resilience — Improve control-plane availability through provider placement rules or a self-built control-plane VIP where no load balancer is available, and preserve declared local disks when nodes are replaced during a rolling upgrade.
  • Alauda OS node storage on Huawei DCS — Huawei DCS Provider v1.0.16 or later supports dedicated /var/lib/kubelet and /var/lib/containerd disks. When disks are declared as persistent, the provider detaches and reattaches them during rolling node replacement, reducing Btrfs I/O pressure on the system disk and protecting declared node-local data.
  • Network and host configuration — Configure multiple network interfaces for node traffic isolation where supported. On Huawei Cloud Stack, a node can retain both its short hostname and its FQDN.
  • Huawei DCS Provider compatibility for Alauda OS — For Huawei DCS clusters that use the Alauda OS image for ACP v4.3.2 or later, including ACP v4.4.0 and later, install Huawei DCS Provider v1.0.21 or later. Select the Alauda OS image from the OS Support Matrix row for the same ACP release. Do not pair DCS Provider v1.0.21 or later with an Alauda OS image for an ACP release earlier than v4.3.2. See Alauda OS and Provider Compatibility for Huawei DCS.
  • Safer Huawei DCS virtual-machine operations — Stop a virtual machine before deleting its node, and remove the boot CD-ROM attachment after startup so it does not prevent later VM migration.

For your provider and exact supported version, see About Immutable Infrastructure and its provider release notes.

Upcoming Immutable Infrastructure Provider Releases

The following updates are planned for separately versioned provider releases in the ACP 4.4 delivery cycle. Upgrading ACP to 4.4.0 alone does not install them; use the matching provider release when it becomes available. Provider product documentation and release notes will publish the detailed support boundaries and procedures.

  • VMware vSphere fixed-address node lifecycle — The VMware vSphere Provider update improves MachineConfigPool health and bootstrap diagnostics, persistent and ephemeral disk handling, and rolling-update safeguards for fixed-address nodes. A virtual machine that an administrator intentionally powers off for maintenance can remain powered off.

Deprecations and Removals

  • System ALB removed — The platform-level system ALB is removed in 4.4.

  • MinIO Operator removed — The MinIO Operator is no longer shipped with the platform starting in 4.4. Use Ceph object storage for S3-compatible object storage.

  • Alauda Service Mesh v1 no longer released — Alauda Service Mesh v1 receives no further releases. Deploy Alauda Service Mesh v2 on 4.4. An existing v1 deployment must install Alauda Service Mesh v2 after the platform upgrade and migrate its Service Mesh v1 data to v2. For details, see Service Mesh v2.

  • Legacy monitoring and alerting APIs deprecated — The /v2/metrics, /v2/alerts, and /v1/alerthistories endpoints are deprecated in 4.4. They remain available in this release and are planned for removal in a future release. Migrate to the corresponding /platform endpoints, which apply the platform role and permission model to every request.

    Deprecated endpointReplacement
    GET /v2/metrics/{cluster}/indicatorsGET /platform/monitoring.alauda.io/v1beta1/clusters/{cluster}/indicators
    POST /v2/metrics/{cluster}/query
    POST /v2/metrics/{cluster}/query_range
    GET /platform/monitoring.alauda.io/v1beta1/clusters/{cluster}/metrics
    GET /v2/metrics/{cluster}/prometheus/query
    GET /v2/metrics/{cluster}/prometheus/query_range
    GET /platform/monitoring.alauda.io/v1beta1/clusters/{cluster}/metrics
    GET /v2/metrics/{cluster}/prometheus/label/{labelName}/valuesGET /platform/monitoring.alauda.io/v1beta1/clusters/{cluster}/variables
    GET /v2/alerts/{cluster}/alertstatusGET /platform/alerts.alauda.io/v1beta1/clusters/{cluster}/alerts
    GET /v2/alerts/{cluster}/silencestatusGET /platform/alerts.alauda.io/v1beta1/clusters/{cluster}/silences
    GET /v1/alerthistoriesGET /platform/alerts.alauda.io/v1beta1/histories
    GET /v1/alerthistories/{name}/messagesGET /platform/alerts.alauda.io/v1beta1/histories/{name}/messages

    Namespace-scoped variants of the replacement endpoints are available under projects/{project}/clusters/{cluster}/namespaces/{namespace}. Project-scoped variants are available under projects/{project} for the monitoring endpoints and for the alert history endpoints. For the monitoring endpoint reference, see Monitoring APIs.

Fixed Issues

  • Certificate expiry monitoring metrics carried a label that changed on every scrape, so the number of time series grew continuously and the monitoring components consumed increasing amounts of resources. This issue is fixed in v4.4.0.
  • When an ACP 4.1.6 workload cluster running on a traditional operating system was managed by ACP 4.4, adding a node could fail because the containerd configuration delivered to the node was incompatible with the containerd version on that node. This issue is fixed in v4.4.0.
  • After two-factor authentication was enabled, the alerting service in member clusters failed to start because the required callback credential was not delivered. This issue is fixed in v4.4.0.
  • Failed backup jobs were not cleaned up in time, so etcd backup pods accumulated continuously and could reach the node's pod limit. This issue is fixed in v4.4.0.
  • After a disaster recovery failover, nodes in a workload cluster could not be rebuilt or scaled out, and newly created nodes never joined the cluster. This issue is fixed in v4.4.0.
  • With containerd 2.x, the NVIDIA runtime configuration still referenced the legacy runtime, so GPU workloads could not be created. This issue is fixed in v4.4.0.
  • When a CAS server used different path prefixes for its login and ticket validation endpoints, the CAS connector did not validate tickets against the configured validation endpoint, and single sign-on failed. This issue is fixed in v4.4.0.
  • In large business clusters, VictoriaMetrics vmselect could generate high disk I/O while processing recording rule queries, increasing disk usage pressure and affecting monitoring query performance. This issue was fixed in ACP v4.4.0.
  • During prolonged operation, the frontend component leaked HTTP connections continuously, which could exhaust local ephemeral TCP ports and make frontend pages inaccessible. This issue is fixed in v4.4.0.
  • When the control plane was scaled out in a workload cluster with five control plane nodes, some requests intermittently returned HTTP 401 and the cluster node list failed to load. This issue is fixed in v4.4.0.
  • After a platform upgrade, running kubectl top node to query node resource usage returned an error. This issue is fixed in v4.4.0.
  • When processing of a single plugin image failed, image-loader stopped warming up the remaining images, which could prevent plugins from loading or make frontend pages unavailable. This issue is fixed in v4.4.0.
  • When monitoring was deployed on a cluster that had log storage installed but no log collector, monitoring could report no data at all. This issue is fixed in v4.4.0, and monitoring now collects and reports data normally in this scenario.
  • When VPC Egress Gateway ran with multiple replicas, some namespaces could lose all connectivity to external networks, and connectivity did not recover automatically until the affected workload containers were restarted. This issue is fixed in v4.4.0.
  • After resources were successfully imported into a native application, no new history version snapshot was created, so the change could not be rolled back. This issue is fixed in v4.4.0, and a new history version is now created after a successful import so that the change can be rolled back.
  • In environments that use an NFSv3 server where rpc.statd is not running on the node, deleting a PVC could fail, leaving backend directories and storage space unreclaimed and causing zombie processes to accumulate. This issue is fixed in v4.4.0.
  • In environments that combine MetalLB, ALB, and underlay networking, access from an in-cluster Pod to an in-cluster backend through the LoadBalancer VIP could fail when ALB ran with multiple replicas. This issue is fixed in v4.4.0.
  • The details page of a template application showed no compute components, and applications that carried user group information did not sync their workloads correctly. This issue is fixed in v4.4.0, and such applications now sync and display their workloads as expected.
  • When a VPC Egress Gateway used a dual-stack internal subnet together with an IPv4-only external subnet, the gateway init container restarted repeatedly and the gateway never became ready. This issue is fixed in v4.4.0.
  • When a VPC Egress Gateway used a dual-stack internal subnet together with an IPv4-only external subnet, the gateway init container restarted repeatedly and the gateway never became ready. This issue is fixed in v4.4.0.
  • A problematic or slow Chart could block the HelmRequest processing queue of its cluster, delaying the deployment and update of other Charts on the same cluster. v4.4.0 adds the ability to configure the number of HelmRequest workers that Captain uses per cluster, which relieves this queue blocking and improves Chart deployment and update efficiency.
  • When a Helm chart was uploaded or deleted in Application Marketplace, a backend timeout while probing Harbor could cause the operation to be incorrectly reported as failed. This issue is fixed in v4.4.0, with improved Harbor endpoint detection, protocol fallback, and timeout handling so that operation results are reported correctly.
  • The ephemeral-storage limit configured for the metis component was too small, so the metis container restarted repeatedly once its usage exceeded the limit. This issue is fixed in v4.4.0, where the limit has been raised to an appropriate value.

Known Issues

  • When deploying a global cluster in DCS mode, the installation can stall at the restartImagePullBackOffPods step and fail to complete: all pods in the cert-manager namespace remain in ImagePullBackOff, the cert-manager AppRelease stays at Synced/WaitReady, and subsequent components are never created.

    This happens because the registry username and password in the global cluster's ClusterCredential (cc-global) are cleared after the installer writes them. The component that synchronizes image pull credentials across namespaces cannot read the credentials and silently skips the synchronization. As a result, the global-registry-auth secret in the cert-manager and data-services-system namespaces keeps an outdated registry address that does not match the address the pods actually pull from, so image pulls fall back to anonymous access and are rejected by the registry with a 401 error.

    Workaround: restore the correct registry username and password in the global cluster's ClusterCredential (cc-global). The synchronization component refreshes global-registry-auth in the affected namespaces on its next reconcile, and the cert-manager pods recover. This operation requires cluster-admin command-line access to the global cluster. If the credentials are cleared again by the periodic token rotation, correct the global-registry-auth secret in the affected namespaces manually.
  • On virtual-machine nodes with high CPU steal, the management view shows a continuous 0% CPU utilization for worker nodes while their actual utilization is not zero; alerts based on this metric also do not fire. Querying the raw node CPU metric returns the real utilization.
  • The local-storage device discovery component updates its resources at a high frequency, continuously generating a large volume of audit logs on clusters with many nodes. No workaround is currently available.
  • Application creation failure triggered by the defaultMode field in YAML.
    Affected Path: Alauda Container Platform → Application Management → Application List → Create from YAML. Submitting YAML containing the defaultMode field (typically used for ConfigMap/Secret volume mount permissions) triggers validation errors and causes deployment failure.
    Workaround: Manually remove all defaultMode declarations before application creation.
  • When pre-delete post-delete hook is set in helm chart.
    When the delete template application is executed and the chart is uninstalled, the hook execution fails for some reasons, thus the application cannot be deleted. It is necessary to investigate the cause and give priority to solving the problem of hook execution failure.

The following product sites provide component-specific documentation, compatibility information, upgrade guidance, and, where published, release notes. Their versions and release schedules are independent of ACP 4.4, so product documentation and release notes can be published after the ACP 4.4 platform release.