Alauda Build of LeaderWorkerSet
LeaderWorkerSet
Alauda Build of LeaderWorkerSet is based on the LeaderWorkerSet (LWS) Kubernetes SIG project. LeaderWorkerSet provides a Kubernetes-native workload API for deploying groups of pods in a Leader/Worker pattern, enabling multi-node distributed workloads — particularly large AI model training and inference — to run as first-class citizens on Kubernetes.
Main components and capabilities include:
- LeaderWorkerSet CRD: The core API resource that defines a group of replicated Leader/Worker pod sets. Each replica consists of one leader pod and a configurable number of worker pods, co-scheduled and managed as a unit.
- Co-scheduling & Topology Awareness: Leader and worker pods within a group are scheduled together, with support for topology spread constraints to co-locate pods on the same node, rack, or availability zone for low-latency inter-node communication (e.g., NVLink, InfiniBand).
- Multi-node LLM Inference: Enables large language models that exceed single-node GPU memory (e.g., Llama 3.1 405B) to be served across multiple nodes using tensor parallelism or pipeline parallelism. LWS is a required dependency of Alauda AI Platform Inference Management for this use case.
- Multi-node Training: Supports distributed training frameworks (PyTorch DDP, DeepSpeed, Megatron-LM) by providing stable, co-located leader/worker pod groups with predictable hostnames and network identities.
- Rolling Updates & Failure Recovery: Supports rolling restarts and automatic pod replacement at the group level, ensuring the entire Leader/Worker group is recycled consistently when a failure or update occurs.
- Startup Sequencing: The leader pod can act as the entry point and coordinator, with worker pods starting after the leader is ready — enabling frameworks that require a master process to be initialized before workers connect.
LWS is installed and managed by Alauda AI Platform through the lws component in the default AmlCluster. For the Alauda AI Platform installation and component configuration, see Install Alauda AI Platform.
Documentation
LeaderWorkerSet upstream documentation and related resources:
- LeaderWorkerSet Documentation: https://lws.sigs.k8s.io/ — Official documentation covering concepts, API reference, and usage guides.
- LeaderWorkerSet GitHub: https://github.com/kubernetes-sigs/lws — Source code, API reference, and examples for the LeaderWorkerSet Kubernetes SIG project.
- KServe (Alauda AI Platform Inference Management): ../kserve/intro — KServe uses LeaderWorkerSet as a required dependency for multi-node LLM inference workloads.