#Fast Upgrade from ACP 4.2 or 4.3 via Quickstart Script

This page covers the one-shot quickstart script that drives the ACP 4.2 or 4.3 → 4.4 upgrade on a storage cluster. For the manual kubectl apply walkthrough with every field explained, see Upgrade from ACP 4.2 or 4.3 to 4.4.

Pick this path when:

• The cluster currently runs the ACP 4.2 or 4.3 Log Storage for ClickHouse chart (logclickhouse / logcenter), with ClickHouseInstallation/cpaas-clickhouse in cpaas-system.
• The upgrade target is ACP 4.4 with the operator-managed data path (log-storage-operator), reusing the existing ClickHouse data via adopt.
• No OpenSearch migration is involved. OpenSearch upgrade is not covered by the script.

If the ACP 4.2 or 4.3 install uses Log Storage for Elasticsearch, use Upgrade from Log Storage for Elasticsearch and Historical Data Migration instead. The script does not apply to that path.

#1. What the script does

The script runs five stages on the storage cluster, in order:

Stage E is intentionally last. Kafka and the mq-conn Secret are already in place when the operator boots, so the first auto-migration pass derives platform-default-ch-conn and creates PlatformLogForward/platform-default cleanly, without going through the suspiciousEmptyConnSecret refuse branch that applies when the Secret is missing.

#2. Prerequisites

#2.1 ACP 4.2 or 4.3 logging stack is running

kubectl -n cpaas-system get chi cpaas-clickhouse
kubectl -n cpaas-system get apprelease logclickhouse
kubectl -n cpaas-system get secret clickhouse-basic-auth

All three objects must exist. If any is missing, the ACP 4.2 or 4.3 install is incomplete; repair it before upgrading.

#2.2 Dynamic StorageClass (for the new Kafka)

kubectl get sc

At least one SC with a provisioner other than no-provisioner. Kafka broker PVCs will land on it. It does not have to match the SC the existing ClickHouse uses.

#2.3 OperatorHub registrations

The script installs two operators through OLM. Both must be registered in the ACP platform CatalogSource:

kubectl -n cpaas-system get packagemanifest strimzi-kafka-operator log-storage-operator

Both rows must report a non-empty AGE. The chop operator is resolved by OLM as a dependency and does not need an explicit entry in this check (see Section 9).

#2.4 Full upgrade pre-flight

This page covers the subset of pre-flight most relevant to the quickstart script. For the full seven-item pre-flight (SA tokens, Feature CRD, Kafka hostNetwork, OLM dependency placement, cross-namespace routing), review the manual Upgrade from ACP 4.2 or 4.3 to 4.4 guide and run a dry run against a staging environment before any production upgrade.

#3. Get the script bundle

Unpack on the jump host:

quickstart/
  upgrade.sh                              ← used for this upgrade path
  install.sh                              ← fresh install only; not used here
  install.env.example
  templates/
    00u-strimzi-subscription.yaml         ← upgrade.sh Stage A
    99u-log-storage-subscription.yaml     ← upgrade.sh Stage E
    12-rdskafka.yaml                      ← upgrade.sh Stage B
    20-kafka-user-topics.yaml             ← upgrade.sh Stage C
    30-mq-conn-secret.yaml                ← upgrade.sh Stage D
    # Other templates (10-keeper, 11-chi, 31-ch-conn, 40-plf) are for the fresh-install script only.

#4. Configure install.env

The upgrade script shares install.env with the fresh-install script. CK_* and KEEPER_* variables are present in the sample file and are safe to leave as-is for the upgrade path — upgrade.sh does not read them.

cp install.env.example install.env

Two required fields:

CK_STORAGE_SIZE keeps the sample value (for example 300Gi). upgrade.sh does not read or validate it.

#4.1 Optional — pin Kafka pods to a dedicated node pool

Use the same NODE_AFFINITY_* and TOLERATION_* fields as in Fast Install § 3.1 "Optional — pin pods to a dedicated node pool". The upgrade script applies them to the three RdsKafka pod template groups (kafka, controller, entityOperator).

#4.2 Optional — Kafka SASL password

KAFKA_SASL_PASSWORD defaults to empty, in which case the script derives a stable 40-hex value from NAMESPACE + KAFKA_NAME + KAFKA_USER via sha256sum. Reruns yield the same password. On Alauda OS / FIPS hosts this satisfies the ≥ 32-character requirement.

#5. Run

#5.1 Render, review, then apply (recommended)

Render YAML into ./rendered-upgrade/ without touching the cluster:

./upgrade.sh install.env --render

Review the four files under ./rendered-upgrade/. platform-default-mq-conn is intentionally omitted from the render output — it requires the live cpaas-kafka-cluster-ca-cert from strimzi, which only exists after Stage B's RdsKafka is Ready. The live run materializes it from the live Secret.

When ready:

./upgrade.sh install.env

#5.2 Direct live apply

./upgrade.sh install.env

#5.3 Auto-migration takes over

After Stage E, log-storage-operator starts and runs its auto-migration loop. Within one to three minutes it:

1. Detects AppRelease/logclickhouse and runs the AdoptCK branch.
2. Derives platform-default-ch-conn from secret/clickhouse-basic-auth and svc/cpaas-clickhouse.
3. Derives PlatformLogForward/platform-default's spec.output.clickhouse from AppRelease.spec.values and discovers rdskafka/cpaas-kafka for Kafka fields.
4. Creates the PLF with label log.alauda.io/auto-generated-by=log-storage-operator.

The admin does not need to kubectl apply any PLF manifest. Stage F of upgrade.sh blocks on PLF/platform-default materializing and reaching Ready=True before the script returns, so when the final ALL STAGES PASS prints, the operator-managed data path is already converged.

#6. Verify

Once PLF is Ready=True, log queries from the ACP console continue to serve historical data (adopt preserved every record) and the new ingest flows through the Kafka buffer into the same ClickHouse tables.

#7. Decommission the ACP 4.2 or 4.3 CK helm plugin

After the operator-managed data path passes Section 6, retire the ACP 4.2 or 4.3 logclickhouse plugin so that the storage cluster is managed by log-storage-operator alone. The flow is five steps, in strict order:

1. Upgrade logagent to v4.4 on every workload cluster that still writes to this storage cluster (§7.1).
2. Identify the legacy logclickhouse ModuleInfo on each storage cluster (§7.2).
3. Retarget each logagent ModuleInfo on that storage cluster from legacy CK to Standard (§7.3).
4. Uninstall logclickhouse (§7.4).
5. Verify the ClickHouse data plane is intact (§7.5).

The ClickHouse data plane (CHI, Keeper, PVs, the cpaas-clickhouse / cpaas-clickhouse-keeper Services, and secret/clickhouse-basic-auth) is already annotated helm.sh/resource-policy=keep by log-storage-operator during auto-migration, so a helm uninstall leaves it in place. No manual annotation step is needed.

Every ModuleInfo DELETE — whether triggered by the console or by kubectl — is validated by the vmoduleinfo.kb.io admission webhook (backed by cluster-transformer in cpaas-system, failurePolicy=Fail). The webhook rejects the delete when:

• The owning ModuleConfig has spec.deleteable: false and the ModuleInfo carries no annotation cpaas.io/force-delete=true.
• Another ModuleInfo on the same storage cluster carries the dependency label logclickhouse.plugins.cpaas.io/cluster=<storage-cluster> — set by any plugin whose spec.config.crossClusterDependency.logclickhouse still points at this cluster. Typically the ACP 4.2/4.3 logagent plugin.
• A ClusterPluginInstance controller on the target cluster still claims the plugin.

For logclickhouse specifically, moduleconfigs/logclickhouse-v4.2.2 ships with deleteable: true, so the dependency check is the gate. Clear it with steps §7.1–§7.3.

#7.1 Upgrade logagent to v4.4 on every workload cluster

Before touching any logagent config, upgrade each workload cluster's logagent plugin to v4.4.x through the standard Platform Management → Cluster Plugin → Upgrade flow. Do not skip this step and attempt to patch directly to Standard: the admission webhook accepts the storage.type: Standard value even on a v4.2 ModuleInfo (the radio enum is a UI hint, not server-side validation), but the v4.2 chart-alauda-log-agent has no template branch for Standard, so a patched-but-not-upgraded plugin fails to render and leaves the workload cluster without a working collector.

Verify every logagent ModuleInfo that still writes to this storage cluster is at v4.4.x:

kubectl get minfo -l cpaas.io/module-name=logagent \
  -o custom-columns=NAME:.metadata.name,HOST:.metadata.labels.cpaas\.io/cluster-name,VERSION:.spec.version
# Every row's VERSION must be v4.4.x before continuing.

#7.2 Identify the legacy logclickhouse installations

List every logclickhouse ModuleInfo and read the CLUSTER column — each row is one storage cluster whose legacy CK plugin needs retiring:

kubectl get minfo -l cpaas.io/module-name=logclickhouse \
  -o custom-columns=NAME:.metadata.name,CLUSTER:.spec.cluster,VERSION:.spec.version

Pick one storage cluster at a time and bind its name for the next two steps:

STORAGE_CLUSTER=<cluster-from-the-CLUSTER-column>
LOGCLICKHOUSE_MINFO=<name-from-the-NAME-column>

#7.3 Retarget every logagent instance away from legacy CK

Enumerate every ModuleInfo still writing to the legacy logclickhouse on this storage cluster — the ACP console does not list this as a single page, so run the label lookup directly:

kubectl get minfo -l logclickhouse.plugins.cpaas.io/cluster=${STORAGE_CLUSTER}

Most rows are logagent instances. For each one, patch the config to switch to the operator-managed path:

kubectl patch minfo <logagent-minfo-name> --type=merge -p '{
  "spec": {
    "config": {
      "storage": {"type": "Standard"},
      "crossClusterDependency": {
        "logclickhouse": null,
        "logstorage": "'"${STORAGE_CLUSTER}"'"
      }
    }
  }
}'

The underlying ClickHouse cluster is still the same, but the plugin now resolves its storage via log-storage-operator's platform-default-ch-conn on that cluster rather than via the legacy logclickhouse plugin.

cluster-transformer watches ModuleInfo changes and runs processPluginDependices whenever spec.config moves. That reconciler reads spec.config.crossClusterDependency.logclickhouse: an empty value triggers delete(labels, key), so the logclickhouse.plugins.cpaas.io/cluster=<storage-cluster> label drops off the ModuleInfo within milliseconds. Re-run the lookup until no rows return:

kubectl get minfo -l logclickhouse.plugins.cpaas.io/cluster=${STORAGE_CLUSTER}
# No resources found

Equivalent UI path for one workload cluster at a time: Platform Management → Cluster Plugin, open the logagent plugin configuration on the target workload cluster, change Log Storage Plugin (日志存储插件) from ClickHouse to Standard, enter the storage cluster name, save.

#7.4 Uninstall the logclickhouse plugin

With the dependency gate clear, the DELETE goes through the webhook unobstructed. Either path works — pick one:

• ACP console: Platform Management → Cluster Plugin, switch the cluster filter to this storage cluster, find the logclickhouse entry, click Uninstall.
• kubectl: kubectl delete minfo ${LOGCLICKHOUSE_MINFO}.

Both go through the same vmoduleinfo.kb.io validation. Once validation passes, cluster-transformer deletes the ModuleInfo server-side and the delete cascades to the workload cluster: AppRelease/logclickhouse is removed, helm uninstall runs under it, resources carrying the keep annotation stay, and the helm-shipped ConfigMaps, RBAC, and the helm chop Deployment (if Section 9's recommended pre-upgrade step did not remove it already) are deleted.

#7.5 Verify the data plane is intact

kubectl -n cpaas-system get chi cpaas-clickhouse
kubectl -n cpaas-system get secret clickhouse-basic-auth
kubectl -n cpaas-system get platformlogforward platform-default

All three must still exist, and PLF must still report Ready=True. If PLF briefly flips to Ready=False while the uninstall runs, wait one minute and recheck — the operator reconciles the derived platform-default-ch-conn Secret and re-converges on the preserved CHI.

#8. Rollback

If a problem appears after the upgrade and the cluster must return to the ACP 4.2 or 4.3 chart state, this is only safe before Section 7 has run. Once the logclickhouse AppRelease is deleted, treat the upgrade as committed and resolve any issue forward via Section 10 instead.

• ClickHouse side: no action. The ACP 4.2 or 4.3 chart's cpaas-clickhouse CHI was never modified. Data is intact.
• Operator side: delete the PLF, then the log-storage-operator Subscription and CSV, then the Kafka resources (RdsKafka + Subscription + CSV).

Delete in reverse Stage order (E → A): the log-storage-operator Subscription + CSV first (which releases the auto-generated PlatformLogForward/platform-default and platform-default-ch-conn Secret), then the Kafka resources (RdsKafka, RdsTopics, RdsKafkaUser, platform-default-mq-conn), and finally the strimzi Subscription + CSV. The auto-migration's writes are limited to the operator Deployment, the two derived Secrets, and the PLF; the ClickHouse data plane is independent.

#9. chop operator dependency (upgrade-cluster side effect)

log-storage-operator's OLM bundle declares clickhouse-operator (chop) as a dependency. When Stage E applies the Subscription, OLM resolves the dependency as follows:

• OLM checks for an existing ClusterServiceVersion of chop.
• The ACP 4.2 or 4.3 chart installs chop via Helm, not via OLM. There is no CSV for the Helm-installed chop.
• OLM therefore considers chop as "not installed" and provisions a new OLM-managed chop CSV to cpaas-logging.

The result is two chop controllers coexisting on the upgrade cluster: the Helm-installed one (in whichever namespace the ACP 4.2 or 4.3 chart placed it, typically cpaas-system or cpaas-logging) and the newly OLM-installed one in cpaas-logging. Both watch ClickHouseInstallation/cpaas-clickhouse and reconcile it concurrently.

In practice the two chop versions agree on the reconciled state (both are Altinity chop), but the duplicate watchers produce visible side effects:

• Repeated resourceVersion bumps on the CHI.
• Doubled reconcile log output in both chop pods.
• Occasional spurious cpaas-clickhouse events.

To check:

kubectl get deploy -A -l app=clickhouse-operator

Two matching Deployment rows confirm the duplicate. Handle it either way:

• Keep the OLM-installed chop (recommended, aligns with the ACP 4.4 architecture). Delete the Helm-installed Deployment, ServiceAccount, Role, and RoleBinding. The CRDs are shared and must stay.
• Keep the Helm-installed chop. Delete the OLM Subscription from cpaas-logging and then its CSV.

#10. Troubleshooting

#Stage D fails with cluster CA secret cpaas-kafka-cluster-ca-cert not found

Stage B's RdsKafka is Ready, but strimzi has not yet written the cluster CA Secret. Wait two minutes and rerun ./upgrade.sh install.env. The script is idempotent.

#After Stage E, PlatformLogForward/platform-default is not auto-created within five minutes

Inspect the operator log:

kubectl -n cpaas-system logs deploy/log-storage-operator-controller-manager | grep -iE "auto-migration|derive-plf|refuse"

• suspiciousEmptyConnSecret: platform-default-mq-conn is missing. Verify with kubectl -n cpaas-system get secret platform-default-mq-conn. Rerun ./upgrade.sh install.env if missing.
• AppRelease/logclickhouse not found: the ACP 4.2 or 4.3 install is incomplete. Return to Section 2.1.
• AdoptCK with clickhouse.nodes[] empty: the legacy AppRelease.spec.values.clickhouse.nodes is empty. The PLF enters PhaseAwaitingUserInput. See the "Troubleshooting the migration" section of Upgrade from ACP 4.2 or 4.3 to 4.4 for the manual patch command.

#PlatformLogForward reports Ready=False with reason ExternalMessageQueueSecretMissing

platform-default-mq-conn is not materialized. Stage D should have created it. Verify with:

kubectl -n cpaas-system get secret platform-default-mq-conn

Rerun ./upgrade.sh install.env if missing.

#CHI reconcile bumps resourceVersion repeatedly

Two chop versions are coexisting. See Section 9 for the resolution options.

For a wider troubleshooting index covering ingest and query failures, see Troubleshooting the Logging Stack.

#11. Related

• Upgrade from ACP 4.2 or 4.3 to 4.4 — Full manual walkthrough; the authoritative reference for every field this script sets.
• Upgrade from Log Storage for Elasticsearch — For clusters that run the ACP 4.2 or 4.3 Elasticsearch backend instead.
• Historical Data Migration — Required when migrating historical records out of the legacy Elasticsearch backend.
• Fast Install via Quickstart Script — Counterpart script for fresh installs.
• Troubleshooting the Logging Stack — Common failure index.