English

English
Русский

Alauda AI
Model Deployment & Inference
Inference Service
Guides

Pre-installation Configuration

Install Workbench

Install Alauda AI Essentials

Install Alauda AI

Install Alauda Build of KServe

Tools Menu Configuration

Kubeflow Chart Plugins

Upgrade Alauda AI

Alauda support for Kubeflow

how_to

Use Kubeflow Notebooks

Use Kubeflow Tensorboards

Use Kubeflow Volumes

Use Kubeflow Model Registry

Use Kubeflow Pipelines

Infrastructure Management

Device Management

About Alauda Build of Hami

About Alauda Build of NVIDIA GPU Device Plugin

Namespace Management

Create Workbench

Fine-tuning LLMs using Workbench

Kubeflow Trainer Quick Start

Model Deployment & Inference

Inference Service

Inference Service

Create Inference Service using CLI

Extend Inference Runtimes

Using KServe Modelcar for Model Storage

Configure External Access for Inference Services

Configure Scaling for Inference Services

Scheduling Inference Services based on the CUDA version

Set Up Autoscaling for Inference Services with KEDA

Troubleshooting

Experiencing Inference Service Timeouts with MLServer Runtime

Inference Service Fails to Enter Running State

Model Management

Model Repository

Upload Models Using Notebook

Monitoring & Ops

Features Overview

Logging & Tracing

Resource Monitoring

Resource Monitoring

Add a Monitoring Dashboard

Troubleshooting

Monitor Dashboard Stuck at Loading

Install Label Studio

Alauda Build of Kueue

Configuring quotas

Monitoring pending workloads

Configuring fair sharing

Gang scheduling

Integrate with Alauda DevOps Pipelines

Integrate with InferenceService

LLM Compressor with Alauda AI

Alauda Build of Llama Stack

Install Llama Stack

Alauda Build of LeaderWorkerSet

Alauda Build of TrustyAI

Install TrustyAI

AI Guardrails for LLM safety

Kubernetes APIs

Inference Service APIs

ClusterServingRuntime [serving.kserve.io/v1alpha1]

InferenceService [serving.kserve.io/v1beta1]

Workspace Kind [kubeflow.org/v1beta1]

Workspace [kubeflow.org/v1beta1]

AmlNamespace [manage.aml.dev/v1alpha1]

AmlCluster [amlclusters.aml.dev/v1alpha1]

Guides

Previous pageIntroduction Next pageInference Service