Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 80 additions & 92 deletions docs/en/installation/ai-cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ weight: 30

To begin, you will need to deploy the **Alauda AI Operator**. This is the core engine for all Alauda AI products. By default, it uses the **KServe** `Raw Deployment` mode for the inference backend, which is particularly recommended for resource-intensive generative workloads. This mode provides a straightforward way to deploy models and offers robust, customizable deployment capabilities by leveraging foundational Kubernetes functionalities.

If your use case requires `Serverless` functionality, which enables advanced features like **scaling to zero on demand** for cost optimization, you can optionally install the **Alauda AI Model Serving Operator**. This operator is not part of the default installation and can be added at any time to enable `Serverless` functionality.
If your use case requires `Serverless` functionality, which enables advanced features like **scaling to zero on demand** for cost optimization, you can optionally install the **Knative Operator**. This operator is not part of the default installation and can be added at any time to enable `Serverless` functionality.



Expand All @@ -26,19 +26,19 @@ If your use case requires `Serverless` functionality, which enables advanced fea

_Download package: aml-operator.xxx.tgz_

- **Alauda AI Model Serving Operator**
- **Knative Operator**

Alauda AI Model Serving Operator provides serverless model inference.
Knative Operator provides serverless model inference.

_Download package: kserveless-operator.xxx.tgz_
_Download package: knative-operator.ALL.v1.x.x-yymmdd.tgz_

:::info
You can download the app named 'Alauda AI' and 'Alauda AI Model Serving' from the Marketplace on the Customer Portal website.
You can download the app named 'Alauda AI' and 'Knative Operator' from the Marketplace on the Customer Portal website.
:::

## Uploading

We need to upload both `Alauda AI` and `Alauda AI Model Serving` to the cluster where Alauda AI is to be used.
We need to upload both `Alauda AI` and `Knative Operator` to the cluster where Alauda AI is to be used.

<Steps>

Expand Down Expand Up @@ -66,7 +66,7 @@ export PLATFORM_ADMIN_PASSWORD=<admin-password> # [!code callout]
export CLUSTER=<cluster-name> # [!code callout]

export AI_CLUSTER_OPERATOR_NAME=<path-to-aml-operator-tarball> # [!code callout]
export KSERVELESS_OPERATOR_PKG_NAME=<path-to-kserveless-operator-tarball> # [!code callout]
export KNATIVE_OPERATOR_PKG_NAME=<path-to-knative-operator-tarball> # [!code callout]

VIOLET_EXTRA_ARGS=()
IS_EXTERNAL_REGISTRY=
Expand Down Expand Up @@ -97,9 +97,9 @@ violet push \
${VIOLET_EXTRA_ARGS[@]}

# [!code highlight]
# Push **KServeless** operator package to destination cluster
# Push **Knative Operator** package to destination cluster
violet push \
${KSERVELESS_OPERATOR_PKG_NAME} \
${KNATIVE_OPERATOR_PKG_NAME} \
--platform-address=${PLATFORM_ADDRESS} \
--platform-username=${PLATFORM_ADMIN_USER} \
--platform-password=${PLATFORM_ADMIN_PASSWORD} \
Expand All @@ -114,14 +114,14 @@ violet push \
3. `${PLATFORM_ADMIN_PASSWORD}` is the password of the ACP platform admin.
4. `${CLUSTER}` is the name of the cluster to install the Alauda AI components into.
5. `${AI_CLUSTER_OPERATOR_NAME}` is the path to the Alauda AI Cluster Operator package tarball.
6. `${KSERVELESS_OPERATOR_PKG_NAME}` is the path to the KServeless Operator package tarball.
6. `${KNATIVE_OPERATOR_PKG_NAME}` is the path to the Knative CE Operator package tarball.
7. `${REGISTRY_ADDRESS}` is the address of the external registry.
8. `${REGISTRY_USERNAME}` is the username of the external registry.
9. `${REGISTRY_PASSWORD}` is the password of the external registry.

</Callouts>

After configuration, execute the script file using `bash ./uploading-ai-cluster-packages.sh` to upload both `Alauda AI` and `Alauda AI Model Serving` operator.
After configuration, execute the script file using `bash ./uploading-ai-cluster-packages.sh` to upload both `Alauda AI` and `Knative Operator`.

</Steps>

Expand All @@ -137,7 +137,7 @@ In **Administrator** view:
2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install Alauda AI.
3. Select **Alauda AI**, then click **Install**.

**Install Alauda AI** window will popup.
**Install Alauda AI** window will pop up.

4. Then in the **Install Alauda AI** window.
5. Leave **Channel** unchanged.
Expand Down Expand Up @@ -195,9 +195,6 @@ In **Administrator** view:

By default, the configuration uses `SelfSigned` certificate type for securing ingress traffic to your cluster, the certificate is
stored in the `knative-serving-cert` secret that is specified in the **Domain Certificate Secret** field.

To use certificate provided by your own, store the certificate secret in the `istio-system` namespace, then update the value of the
**Domain Certificate Secret** field, and change the value of the **Domain Certificate Secret** field to `Provided`.
:::

11. In the **Serverless Configuration** section, set **Knative Serving Provider** to **Operator**; leave all other parameters blank.
Expand Down Expand Up @@ -230,115 +227,106 @@ Now, the core capabilities of Alauda AI have been successfully deployed. If you

Serverless functionality is an optional capability that requires an additional operator and instance to be deployed.

### 1. Installing the Alauda AI Model Serving Operator
### 1. Installing the Knative Operator


<Steps>

### Prerequisites

The `Serverless` capability relies on the **Istio** `Gateway` for its networking. Please install the **Service Mesh** first by following the [documentation](./pre-configuration.mdx#deploy-service-mesh).
:::info
Starting from **Knative CE Operator**, the Knative networking layer switches to **Kourier**, so installing **Istio** is no longer required.
:::

### Procedure
#### Procedure

In **Administrator** view:

1. Click **Marketplace / OperatorHub**.
2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install.
3. Select **Alauda AI Model Serving**, then click **Install**.
3. Search for and select **Knative Operator**, then click **Install**.

**Install Alauda AI Model Serving** window will popup.
**Install Knative Operator** window will pop up.

4. Then in the **Install Alauda AI Model Serving** window.
4. Then in the **Install Knative Operator** window.
5. Leave **Channel** unchanged.
6. Check whether the **Version** matches the **Alauda AI Model Serving** version you want to install.
7. Leave **Installation Location** unchanged, it should be `kserveless-operator` by default.
6. Check whether the **Version** matches the **Knative Operator** version you want to install.
7. Leave **Installation Location** unchanged.
8. Select **Manual** for **Upgrade Strategy**.
9. Click **Install**.

### Verification
#### Verification

Confirm that the **Alauda AI Model Serving** tile shows one of the following states:
Confirm that the **Knative Operator** tile shows one of the following states:

- `Installing`: installation is in progress; wait for this to change to `Installed`.
- `Installed`: installation is complete.

</Steps>


### 2. Creating Alauda AI Model Serving Instance

Once **Alauda AI Model Serving Operator** is installed, you can create an instance. There are two ways to do this:
### 2. Creating Knative Serving Instance

#### **Automated Creation (Recommended)**
Once **Knative Operator** is installed, you need to create the `KnativeServing` instance manually.

You can have the instance automatically created and managed by the `AmlCluster` by editing its parameters.
<Steps>

In **Administrator** view:

1. Click **Marketplace / OperatorHub**.
2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you previously installed the `AmlCluster`.
3. Select **Alauda AI**, then **Click**.
4. In the **Alauda AI** page, click **All Instances** from the tab.
5. Click name **default**.
6. Locate **Actions** dropdown list and select update.
#### Procedure

1. Create the `knative-serving` namespace.

```bash
kubectl create ns knative-serving
```

2. In the **Administrator** view, navigate to **Operators** -> **Installed Operators**.
3. Select the **Knative CE Operator**.
4. Under **Provided APIs**, locate **KnativeServing** and click **Create Instance**.
5. Switch to **YAML view**.
6. Replace the content with the following YAML:
7. Click **Create**.

```yaml
apiVersion: operator.knative.dev/v1beta1
kind: KnativeServing
metadata:
name: knative-serving
namespace: knative-serving
spec:
config:
deployment:
registries-skipping-tag-resolving: kind.local,ko.local,dev.local,private-registry # [!code callout]
domain:
example.com: ""
features:
kubernetes.podspec-affinity: enabled
kubernetes.podspec-hostipc: enabled
kubernetes.podspec-hostnetwork: enabled
kubernetes.podspec-init-containers: enabled
kubernetes.podspec-nodeselector: enabled
kubernetes.podspec-persistent-volume-claim: enabled
kubernetes.podspec-persistent-volume-write: enabled
kubernetes.podspec-securitycontext: enabled
kubernetes.podspec-tolerations: enabled
kubernetes.podspec-volumes-emptydir: enabled
queueproxy.resource-defaults: enabled
network:
domain-template: '{{.Name}}.{{.Namespace}}.{{.Domain}}'
ingress-class: kourier.ingress.networking.knative.dev
ingress:
kourier:
enabled: true
```

**update default** form will show up.
<Callouts>

7. In the **Serverless Configuration** section:
1. Set **Knative Serving Provider** to `Legacy`.
2. Set **BuiltIn Knative Serving** to `Managed`.
1. `private-registry` is a placeholder for your private registry address. You can find this in the **Administrator** view, then click **Clusters**, select `your cluster`, and check the **Private Registry** value in the **Basic Info** section.

8. Leave all other parameters unchanged. Click **Update**.
</Callouts>

</Steps>

#### **Manual Creation and Integration**

You can manually create the `KnativeServing (knativeservings.components.aml.dev)` instance.

<Steps>

In **Administrator** view:

1. Click **Marketplace / OperatorHub**.
2. At the top of the console, from the **Cluster** dropdown list, select the destination cluster where you want to install.
3. Select **Alauda AI Model Serving**, then **Click**.
4. In the **Alauda AI Model Serving** page, click **All Instances** from the tab.
5. Click **Create**.

**Select Instance Type** window will pop up.

6. Locate the **KnativeServing** tile in **Select Instance Type** window, then click **Create**.

**Create KnativeServing** form will show up.

7. Keep `default-knative-serving` unchanged for **Name**.
8. Keep `knative-serving` unchanged for **Knative Serving Namespace**.
9. In the **Ingress Gateway** section, configure the following:
1. Set the **Ingress Gateway Istio Revision** to a value that corresponds to your Istio version (e.g., `1-22`).
2. Set a valid domain for the **Domain** field.
3. Set the appropriate **Domain Certificate Type**.

:::info
For details on configuring the domain and certificate type, refer to the [relevant section](#procedure-1).
:::

10. In the **Values** section, configure the following:
1. Select **Deploy Flavor** from dropdown:
1. `single-node` for non HA deployments.
2. `ha-cluster` for HA cluster deployments (**Recommended** for production).
2. Set **Global Registry Address** to Match Your Cluster

You can find your cluster's private registry address by following these steps:
1. In the Web Console, go to Administrator / Clusters.
2. Select your target **cluster**.
3. On the **Overview** tab, find the `Private Registry address` value in the **Basic Info** section.

</Steps>
### 3. Integrate with AmlCluster

Configure the `AmlCluster` instance to integrate with a `KnativeServing` instance.
Configure the `AmlCluster` instance to integrate with the `KnativeServing` instance.

<Steps>

Expand All @@ -348,10 +336,10 @@ In the **AmlCluster** instance update window, you will need to fill in the requi
After the initial installation, you will find that only the **Knative Serving Provider** is set to `Operator`. You will now need to provide values for the following parameters:
:::

* **APIVersion**: `components.aml.dev/v1alpha1`
* **APIVersion**: `operator.knative.dev/v1beta1`
* **Kind**: `KnativeServing`
* **Name**: `default-knative-serving`
* Leave **Namespace** blank.
* **Name**: `knative-serving`
* **Namespace**: `knative-serving`
</Steps>

## Replace GitLab Service After Installation
Expand Down
105 changes: 105 additions & 0 deletions docs/en/installation/ai-generative.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
weight: 35
---

# Install Alauda AI Generative

**Alauda AI Generative** is a cloud-native component built on **KServe** for serving generative AI models. As an extension of the Alauda AI ecosystem, it specifically optimizes for **Large Language Models (LLMs)**, offering essential features such as inference orchestration, streaming responses, and resource-based auto-scaling for generative workloads.

---

## Prerequisites

Before installing **Alauda AI Generative**, you need to ensure the following dependencies are installed:

### Required Dependencies

| Dependency | Type | Description |
|------------|------|-------------|
| Alauda build of Envoy Gateway <ExternalSiteLink name="acp" href="/networking/operators/envoy_gateway_operator.mdx" children="install_envoy_gateway_operator" /> | Operator | Provides the underlying gateway functionality for AI services |
| Envoy AI Gateway | Cluster Plugin | Provides AI-specific gateway capabilities |
| [Alauda Build of LeaderWorkerSet](../../lws/install.mdx) | Cluster Plugin | Provides leader-worker set functionality for AI workloads |

:::info
`Alauda build of Envoy Gateway` is natively integrated into ACP 4.2. For environments running earlier versions (including ACP 4.0 and 4.1), please contact Customer Support for compatibility and installation guidance.
:::

### Optional Dependencies

| Dependency | Type | Description |
|------------|------|-------------|
| GIE | Built-in | Integrated GIE (gateway-api-inference-extension) for enhanced AI capabilities. Can be enabled through the Alauda AI Generative UI. |
| Alauda AI | Operator | Required only if you need to use KServe Predictive AI functionality. Can be disabled if you only need LLM Generative AI functionality. |

### Installation Notes

1. **Required Dependencies**: All three required dependencies must be installed before installing Alauda AI Generative.
2. **GIE Integration**: If you want to use GIE, you can enable it during the installation process by selecting the "Integrated GIE" option in the Alauda AI Generative UI.
3. **Alauda AI Integration**: If you don't need KServe Predictive AI functionality and only want to use LLM Generative AI, you can disable the "Integrated With Alauda AI" option during installation.

## Downloading Cluster Plugin

:::info

`Alauda AI Generative` cluster plugin can be retrieved from Customer Portal.

Please contact Consumer Support for more information.

:::

## Uploading the Cluster Plugin

For more information on uploading the cluster plugin, please refer to <ExternalSiteLink name="acp" href="ui/cli_tools/index.html#uploading-cluster-plugins" children="Uploading Cluster Plugins" />

## Installing Alauda AI Generative

1. Go to the `Administrator` -> `Marketplace` -> `Cluster Plugin` page, switch to the target cluster, and then deploy the `Alauda AI Generative` Cluster plugin.

2. In the deployment form, configure the following parameters as needed:

### Envoy Gateway Configuration

| Parameter | Description | Default Value |
|-----------|-------------|---------------|
| **ServiceAccount Name** | The name of the service account used by Envoy Gateway. | envoy-gateway |
| **ServiceAccount Namespace** | The namespace where the service account is located. | envoy-gateway-system |
| **Create Instance** | Create an Envoy Gateway instance to manage inference traffic with bundled extensions. | Enabled |
| **Instance Name** | The name of the Envoy Gateway instance to be created. | aieg |

### Envoy AI Gateway Configuration

| Parameter | Description | Default Value |
|-----------|-------------|---------------|
| **Service Name** | The Kubernetes service name for Envoy AI Gateway. | ai-gateway-controller |
| **Port Number** | The port number used by Envoy AI Gateway. | 1063 |

### KServe Gateway Configuration

| Parameter | Description | Default Value |
|-----------|-------------|---------------|
| **Enabled** | Install a KServe Gateway Instance for inferenceservices functionality. | Enabled |
| **Gateway Name** | The name of the KServe Gateway. | kserve-ingress-gateway |
| **Gateway Namespace** | The namespace where the KServe Gateway is deployed. | kserve |
| **GatewayClass** | Optional. The custom name for the GatewayClass. If left empty, the system will automatically derive it following the "\{Namespace\}-\{Name\}" pattern. | (Empty) |
| **Port Number** | The port number used by KServe Gateway. | 80 |

### GIE(gateway-api-inference-extension) Configuration

| Parameter | Description | Default Value |
|-----------|-------------|---------------|
| **BuiltIn** | Install with the bundled gateway-api-inference-extension v0.5.1 dependencies for enhanced AI capabilities. | Enabled |

### Alauda AI Integration

| Parameter | Description | Default Value |
|-----------|-------------|---------------|
| **Integrated** | Enable integration with Alauda AI core plugin to reuse existing configurations. | Disabled |

3. Click **Install** to begin the installation process.

4. Verify result. You can see the status of "Installed" in the UI.

## Upgrading Alauda AI Generative

1. Upload the new version for package of **Alauda AI Generative** plugin to ACP.
2. Go to the `Administrator` -> `Clusters` -> `Target Cluster` -> `Functional Components` page, then click the `Upgrade` button, and you will see the `Alauda AI Generative` can be upgraded.
6 changes: 6 additions & 0 deletions docs/en/lws/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
weight: 90
---
# Alauda Build of LeaderWorkerSet

<Overview />
Loading