Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/en/connections/how_to/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
weight: 20
---

# How To

<Overview />
196 changes: 196 additions & 0 deletions docs/en/connections/how_to/using_connections.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
---
weight: 10
---

# Using Connections

Connections let you store reusable access settings for external model sources and data services at the project level.

For direct YAML examples of model deployment from S3, PVC, or OCI sources, see [Model Storage](../../model_inference/model_management/functions/model_storage.mdx).

> **Note:** For OCI-based model deployment, only connections whose `ACCESS_TYPE` includes `Pull` are available in the model deployment form.

## Prerequisites

- You have access to the target project namespace.
- A suitable connection type already exists.
- You have the credentials and endpoint information for the external storage or registry that you want to use.

## Creating a Connection

1. In the business view, open the target project.
2. Click **Connections**.
3. Click **Create connection**.
4. Select a connection type.
5. Enter the Kubernetes resource name for the connection.
6. Optional: Enter a display name.
7. Complete the fields defined by the selected connection type.
8. Click **Create**.

### Built-in Field Sets

- **URI**: Provide the full URI in the `URI` field.
- **OCI-compliant registry**: Provide `ACCESS_TYPE`, upload or paste `.dockerconfigjson`, and enter `OCI_HOST`.
- **S3-compatible object storage**: Provide `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_S3_ENDPOINT`, and optionally `AWS_DEFAULT_REGION` and `AWS_S3_BUCKET`.

### Verification

- The new item appears on the **Connections** page.
- The table shows the connection name, connection type, compatibility tags, and creator.

> **Note:** The connection resource name and selected connection type cannot be changed after creation.

## Updating a Connection

1. On the **Connections** page, open the action menu for the target connection.
2. Click **Edit**.
3. Update the display name or connection fields.
4. Click **Save**.

After you update credentials or endpoints, redeploy or restart dependent workloads so they pick up the latest values.

## Deleting a Connection

1. On the **Connections** page, open the action menu for the target connection.
2. Click **Delete**.
3. Confirm the operation.

Deleting a connection removes the project secret only. Any workload that still depends on that secret must be updated separately.

## Managing Connection Types

Connection types are managed at cluster scope and stored in the `kube-public` namespace.

### What a Connection Type Contains

- Basic metadata: name, display name, categories, and enabled state
- Ordered section headings and fields
- Optional default values and read-only settings
- Compatibility markers for URI, OCI, or S3 model serving

### Built-in Templates

The connection type editor can append predefined field sets for:

- **URI**
- **OCI compliant registry**
- **S3 compatible object storage**

### Supported Field Types

| Field type | Purpose |
| --- | --- |
| Section | Visual grouping only |
| Text - Short / Text - Long | Plain text input |
| Text - Hidden | Sensitive values such as passwords or secret keys |
| URI | URI input |
| Boolean | Toggle input |
| Numeric | Numeric input with optional min, max, and unit |
| Dropdown | Single-select or multi-select options |
| File | Upload or paste file content such as `.dockerconfigjson` |

> **Note:** File-type fields in the current UI accept pasted content or uploaded files up to 1 MiB.

### Creating a Connection Type

1. Open the target cluster and go to **Settings** > **Connection types**.
2. Click **Create connection type**.
3. Enter the connection type name. The platform stores it with the `ct-` prefix.
4. Optional: Enter a display name.
5. Select one or more categories.
6. Set **Enabled** as needed.
7. Add section headings and fields, or apply a built-in model-serving-compatible template.
8. Click **Create**.

### Duplicating, Editing, Enabling, and Deleting

- Use **Duplicate** when you want to base a new connection type on an existing type.
- Custom connection types can be edited or deleted.
- Preinstalled connection types can be duplicated, but the current UI does not allow editing or deleting them directly.
- The list page provides an **Enabled** switch to mark a connection type as active or inactive.

> **Note:** Editing a connection type does not rewrite existing connection secrets that were created earlier.

## Using Connections for Model Deployment

When you create or update an inference service, you can set **Model location** to **Connection**.

1. Open the inference service creation or update form.
2. Set **Model location** to **Connection**.
3. Select the connection.
4. Complete the model path field according to the connection kind:
- **URI**: No extra path is required; the platform reads the full URI from the connection.
- **OCI**: Enter the relative path under `oci://<OCI_HOST>/`, for example `my-model:latest`.
- **S3**: Enter the object path under `s3://<AWS_S3_BUCKET>/`, for example `llm/qwen2.5`.

### Runtime Behavior

- For **OCI** connections, the connection secret is attached to the workload as an `imagePullSecret`.
- For **S3** connections, the platform creates a dedicated `ServiceAccount` that references the connection secret.
- The inference service metadata stores the selected connection name in the `aml-model-source-connection` annotation.

## Resource Model

The current implementation uses project secrets for connections and cluster-level ConfigMaps for connection types.

### Connection Secret Example

```yaml
apiVersion: v1
kind: Secret
metadata:
name: minio-models
namespace: demo-project
labels:
cpaas.io/dashboard: "true"
annotations:
cpaas.io/display-name: MinIO models
cpaas.io/connection-type-ref: ct-s3-storage
cpaas.io/connection-type-protocol: s3
serving.kserve.io/s3-endpoint: minio.minio.svc:9000
serving.kserve.io/s3-region: us-east-1
type: Opaque
stringData:
AWS_ACCESS_KEY_ID: <access-key>
AWS_SECRET_ACCESS_KEY: <secret-key>
AWS_S3_ENDPOINT: http://minio.minio.svc:9000
AWS_DEFAULT_REGION: us-east-1
AWS_S3_BUCKET: models
```

### Connection Type ConfigMap Example

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: ct-uri-repository
namespace: kube-public
labels:
cpaas.io/dashboard: "true"
cpaas.io/connection-type: "true"
annotations:
cpaas.io/display_name: URI repository
cpaas.io/disabled: "false"
data:
category: '["URI"]'
fields: >-
[
{
"type": "section",
"name": "URI",
"description": "Fields required for URI model serving."
},
{
"type": "uri",
"name": "URI",
"envVar": "URI",
"required": true,
"properties": {
"defaultReadOnly": false
}
}
]
```

> **Note:** Compared with recent OpenShift AI documentation, the current Alauda AI frontend still binds a connection to its template with `cpaas.io/connection-type-ref`, and also writes model-serving compatibility to `cpaas.io/connection-type-protocol`.
7 changes: 7 additions & 0 deletions docs/en/connections/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
weight: 72
---

# Connections

<Overview />
7 changes: 7 additions & 0 deletions docs/en/connections/overview/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
weight: 10
---

# Overview

<Overview />
36 changes: 36 additions & 0 deletions docs/en/connections/overview/intro.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
weight: 10
---

# Introduction

Connections let you store reusable access settings for external model sources and data services at the project level. In Alauda AI, a connection is stored as a Kubernetes `Secret` in the project namespace, and a connection type is stored as a Kubernetes `ConfigMap` in the `kube-public` namespace.

## Core Concepts

### Connection

A connection stores the actual access parameters used by workloads, such as:

- S3 access key, secret key, endpoint, region, and bucket
- OCI registry host and registry authentication
- A direct remote URI

### Connection Type

A connection type defines the form that users see when they create or edit a connection. It controls:

- Which fields are displayed
- Which fields are required
- Default values and read-only settings
- Whether the resulting connection is compatible with URI, OCI, or S3 model serving

## Supported Connection Kinds

| Kind | What the connection stores | How model deployment uses it |
| --- | --- | --- |
| URI | A full remote URI in the `URI` field | The deployment form reads the URI directly from the connection |
| OCI-compliant registry | Registry host, registry credentials, and access mode | The deployment form uses the connection secret as an image pull secret and asks you for the relative model path, such as `repository:tag` |
| S3-compatible object storage | Access key, secret key, endpoint, optional region, and bucket | The deployment form uses the bucket from the connection and asks you for the object path under that bucket |

For step-by-step operations, see [Using Connections](../how_to/using_connections.mdx).
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ To deploy your model, you must first store it in a storage type supported by Ala
- **Persistent Volume Claim (PVC)**: Mounts data stored on a persistent volume before the main container starts via a Storage Initializer.
- **Open Container Initiative (OCI) containers**: Also known as modelcars in KServe. This approach achieves second-level loading using the container runtime's layered caching capability via a Sidecar.

If you want to manage reusable project-level S3, URI, or OCI endpoints in the UI, see [Using Connections](../../../connections/how_to/using_connections.mdx).

## Using S3 Object Storage for model storage
This is the most commonly used mode. It implements credential management through a Secret annotated with specific S3 configuration parameters.

Expand Down Expand Up @@ -200,4 +202,4 @@ spec:
2. `aml.cpaas.io/runtime-type: vllm` specifies the code runtime type. For more information about custom inference runtimes, see [Extend Inference Runtimes](../../inference_service/how_to/custom_inference_runtime.mdx).
3. Replace `aml-vllm-0.11.2-cpu` with the runtime name that is already installed in your platform (corresponding to a ClusterServingRuntime CRD instance).
4. `storageUri: pvc://model-pvc` specifies the PVC name where the model is stored.
</Callouts>
</Callouts>