Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 60 additions & 2 deletions docs/admin-manual/auth/authentication/ldap.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
{
"title": "LDAP",
"language": "en",
"description": "Detailed guide on Apache Doris federated authentication: unified identity verification and group authorization through LDAP integration, covering configuration steps, login methods, permission mapping rules, and common troubleshooting."
"description": "Detailed guide on Apache Doris federated authentication: unified identity verification and group authorization through LDAP/LDAPS integration, covering configuration steps, login methods, permission mapping rules, and common troubleshooting."
}
---

Expand Down Expand Up @@ -56,6 +56,8 @@ In LDAP, data is organized in a tree structure. Here's an example of a typical L
ldap_group_basedn = ou=group,o=emr
```

> To enable LDAPS (encrypted connection to the LDAP server), see the [LDAPS (Encrypted Connection)](#ldaps-encrypted-connection) section below.

3. After starting `fe`, log in to Doris with `root` or `admin` account and set the LDAP admin password:

```sql
Expand Down Expand Up @@ -233,6 +235,52 @@ If user jack belongs to LDAP groups `doris_rd`, `doris_qa`, `doris_pm`, and Dori
> - Which `group` a `user` belongs to is independent of the LDAP tree's organizational structure. `user2` in the example above doesn't necessarily belong to `group2`.
> - To make `user2` belong to `group2`, you need to explicitly add `user2` to the `member` attribute of `group2`.

## LDAPS (Encrypted Connection)

:::info Supported since version 4.0.5
:::

By default, Doris communicates with the LDAP server over plain LDAP (unencrypted). Starting from version 4.0.5, Doris supports LDAPS (LDAP over SSL/TLS) to encrypt the connection between Doris FE and the LDAP server.

### Enabling LDAPS

To enable LDAPS, add the following configuration to `fe/conf/ldap.conf`:

```
ldap_use_ssl = true
```

When `ldap_use_ssl` is set to `true`, Doris uses the `ldaps://` protocol to connect to the LDAP server. You should also update the port to the standard LDAPS port (typically `636`):

```
ldap_host = ldap-host
ldap_port = 636
ldap_use_ssl = true
```

### Configuring Certificate Trust

When using LDAPS, the LDAP server's SSL certificate must be trusted by the Doris FE JVM. If your LDAP server uses a certificate signed by a well-known public CA, no additional configuration is needed.

If using a custom or self-signed Certificate Authority (CA), you must import the CA certificate into a Java trustStore and configure the JVM to use it. Add the following parameters to `JAVA_OPTS` in `fe/conf/fe.conf`:

```
# Example for JDK 17
JAVA_OPTS_FOR_JDK_17 = "-Djavax.net.ssl.trustStore=/path/to/your/cacerts -Djavax.net.ssl.trustStorePassword=changeit ..."
```

> **Steps to import a self-signed CA certificate:**
>
> 1. Obtain the CA certificate file (e.g., `ca.crt`).
> 2. Import it into a Java trustStore using `keytool`:
>
> ```shell
> keytool -importcert -alias ldap-ca -keystore /path/to/your/cacerts -file /path/to/ca.crt -storepass changeit -noprompt
> ```
>
> 3. Configure the trustStore path in `JAVA_OPTS` as shown above.
> 4. Restart the Doris FE for the changes to take effect.

## Cache Management

To avoid frequent access to LDAP services, Doris caches LDAP information in memory.
Expand All @@ -250,7 +298,8 @@ You can refresh the cache with the `refresh ldap` statement. See [REFRESH-LDAP](

## Known Limitations

- Currently, Doris's LDAP functionality only supports cleartext password verification, meaning that when users log in, passwords are transmitted in plaintext between `client` and `fe`, and between `fe` and LDAP service.
- Doris's LDAP functionality only supports cleartext password verification for the client-to-FE segment, meaning that when users log in, passwords are transmitted in plaintext between `client` and `fe`. SSL/TLS encryption between the client and Doris FE must be configured separately (see [Client Connection](#step-2-client-connection)).
- For the FE-to-LDAP segment, the connection is unencrypted by default (`ldap_use_ssl = false`). To encrypt this segment, set `ldap_use_ssl = true` to use LDAPS (see [LDAPS (Encrypted Connection)](#ldaps-encrypted-connection)).

## FAQ

Expand All @@ -266,3 +315,12 @@ Check the following items step by step:
2. Check whether the expected `group` is located under the organizational structure corresponding to `ldap_group_basedn`.
3. Check whether the expected `group` contains the `member` attribute.
4. Check whether the `member` attribute of the expected `group` contains the current user's `dn`.

**Q: LDAPS connection fails, how to troubleshoot?**

Check the following items step by step:

1. Confirm that `ldap_use_ssl = true` is set in `fe/conf/ldap.conf`.
2. Confirm that `ldap_port` is set to the correct LDAPS port (typically `636`).
3. Verify that the LDAP server's SSL certificate is trusted by the JVM. Check the `fe.log` for SSL handshake errors such as `SSLHandshakeException` or `PKIX path building failed`.
4. If using a self-signed CA, ensure the CA certificate has been imported into the trustStore and the trustStore path is correctly configured in `JAVA_OPTS`.
54 changes: 28 additions & 26 deletions docs/lakehouse/catalogs/iceberg-catalog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@ Support for Nested Namespace needs to be explicitly enabled. For details, please
'warehouse' = 'gs://bucket/iceberg_warehouse',
'gs.access_key' = '<ak>',
'gs.secret_key' = '<sk>',
'fs.gcs.support'='true'
'fs.gcs.support' = 'true'
);
```
</TabItem>
Expand All @@ -406,7 +406,7 @@ Support for Nested Namespace needs to be explicitly enabled. For details, please
'azure.account_name' = '<account_name>',
'azure.account_key' = '<account_key>',
'azure.endpoint' = 'https://<account_name>.blob.core.windows.net',
'fs.azure.support'='true'
'fs.azure.support' = 'true'
);
```
</TabItem>
Expand Down Expand Up @@ -546,7 +546,7 @@ Support for Nested Namespace needs to be explicitly enabled. For details, please
's3.access_key' = '<ak>',
's3.secret_key' = '<sk>'
);
```
```
</TabItem>
</Tabs>
</details>
Expand Down Expand Up @@ -651,7 +651,7 @@ Support for Nested Namespace needs to be explicitly enabled. For details, please
```sql
CREATE CATALOG iceberg_dlf_catalog_catalog PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type'='dlf',
'iceberg.catalog.type' = 'dlf',
'warehouse' = 'oss://bucket/iceberg-dlf-oss-warehouse',
'dlf.uid' = '203225413946383283',
'dlf.catalog_id' = 'p2_regression_case',
Expand All @@ -671,7 +671,7 @@ Support for Nested Namespace needs to be explicitly enabled. For details, please
```sql
CREATE CATALOG iceberg_dlf_catalog_catalog PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type'='dlf',
'iceberg.catalog.type' = 'dlf',
'warehouse' = 'oss://bucket/iceberg-dlf-oss-warehouse',
'dlf.uid' = '203225413946383283',
'dlf.catalog.id' = 'catalog_id',
Expand Down Expand Up @@ -1071,7 +1071,7 @@ This is an experimental feature, supported since version 4.1.0.
'warehouse' = 'hdfs://127.0.0.1:8520/iceberg-fs-hdfs-warehouse',
'fs.defaultFS' = 'hdfs://127.0.0.1:8520',
'hadoop.security.authentication' = 'kerberos',
'hadoop.kerberos.principal'='hive/presto-master.docker.cluster@LABS.TERADATA.COM',
'hadoop.kerberos.principal' = 'hive/presto-master.docker.cluster@LABS.TERADATA.COM',
'hadoop.kerberos.keytab' = '/keytabs/hive-presto-master.keytab'
);

Expand Down Expand Up @@ -1145,7 +1145,7 @@ This is an experimental feature, supported since version 4.1.0.
'type' = 'iceberg',
'iceberg.catalog.type' = 'hadoop',
'warehouse' = 'gs://bucket/iceberg_warehouse',
'fs.gcs.support'='true',
'fs.gcs.support' = 'true',
'gs.access_key' = '<ak>',
'gs.secret_key' = '<sk>'
);
Expand All @@ -1155,9 +1155,9 @@ This is an experimental feature, supported since version 4.1.0.
```sql
CREATE CATALOG test_iceberg_fs_on_minio PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type'='hadoop',
'iceberg.catalog.type' = 'hadoop',
'warehouse' = 's3://warehouse/wh',
'fs.minio.support'='true',
'fs.minio.support' = 'true',
'minio.endpoint' = 'http://127.0.0.1:19001',
'minio.access_key' = 'admin',
'minio.secret_key' = 'password',
Expand Down Expand Up @@ -1319,20 +1319,22 @@ This is an experimental feature, supported since version 4.1.0.

<details >
<summary>3.0.6+ Version</summary>
<TabItem value='FileSystem Catalog' label='FileSystem Catalog' default>
Please refer to the [S3 Tables Integration](../best-practices/doris-aws-s3tables.md) documentation.
```sql
CREATE CATALOG test_s3tables_catalog PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type' = 's3tables',
'warehouse' = 'arn:aws:s3tables:ap-east-1:9527:bucket/doris-s3-table-bucket',
's3.region' = 'ap-east-1',
's3.endpoint' = 'https://s3.ap-east-1.amazonaws.com',
's3.access_key' = '<ak>',
's3.secret_key' = '<sk>'
);
```
</TabItem>
<Tabs>
<TabItem value='FileSystem Catalog' label='FileSystem Catalog' default>
Please refer to the [S3 Tables Integration](../best-practices/doris-aws-s3tables.md) documentation.
```sql
CREATE CATALOG test_s3tables_catalog PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type' = 's3tables',
'warehouse' = 'arn:aws:s3tables:ap-east-1:9527:bucket/doris-s3-table-bucket',
's3.region' = 'ap-east-1',
's3.endpoint' = 'https://s3.ap-east-1.amazonaws.com',
's3.access_key' = '<ak>',
's3.secret_key' = '<sk>'
);
```
</TabItem>
</Tabs>
</details>

## Query Operations
Expand Down Expand Up @@ -1395,9 +1397,9 @@ SELECT * FROM iceberg_table FOR VERSION AS OF 123456789;

### Branch and Tag

> Since 3.1.0
> Since 3.1.0.
>
> For creating, dropping and managing branch and tag, please refer to [Managing Branch & Tag]
> For creating, dropping and managing branch and tag, please refer to the [Managing Branch & Tag] section.

Reading specific branches and tags of Iceberg tables is supported.

Expand Down Expand Up @@ -2906,7 +2908,7 @@ You can use the following SQL to analyze the data distribution and delete file c
CASE
WHEN file_size_in_bytes BETWEEN 0 AND 8 * 1024 * 1024 THEN '0-8M'
WHEN file_size_in_bytes BETWEEN 8 * 1024 * 1024 + 1 AND 32 * 1024 * 1024 THEN '8-32M'
WHEN file_size_in_bytes BETWEEN 2 * 1024 * 1024 + 1 AND 128 * 1024 * 1024 THEN '32-128M'
WHEN file_size_in_bytes BETWEEN 32 * 1024 * 1024 + 1 AND 128 * 1024 * 1024 THEN '32-128M'
WHEN file_size_in_bytes BETWEEN 128 * 1024 * 1024 + 1 AND 512 * 1024 * 1024 THEN '128-512M'
WHEN file_size_in_bytes > 512 * 1024 * 1024 THEN '> 512M'
ELSE 'Unknown'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
{
"title": "LDAP",
"language": "zh-CN",
"description": "详解 Apache Doris 联邦认证:通过集成 LDAP 实现统一身份验证与组授权,涵盖配置步骤、登录方式、权限映射规则及常见问题排查。"
"description": "详解 Apache Doris 联邦认证:通过集成 LDAP/LDAPS 实现统一身份验证与组授权,涵盖配置步骤、登录方式、权限映射规则及常见问题排查。"
}
---

Expand Down Expand Up @@ -44,7 +44,7 @@ Doris 支持接入第三方 LDAP 服务,提供验证登录和组授权两大
### 第一步:配置 Doris

1. 在 `fe/conf/fe.conf` 中设置认证方式:`authentication_type=ldap`。
2. 在 `fe/conf/ldap.conf` 中配置 LDAP 服务的连接信息
2. 在 `fe/conf/ldap.conf` 中配置 LDAP 服务的连接信息

```
ldap_authentication_enabled = true
Expand All @@ -56,6 +56,8 @@ Doris 支持接入第三方 LDAP 服务,提供验证登录和组授权两大
ldap_group_basedn = ou=group,o=emr
```

> 如需启用 LDAPS(加密连接至 LDAP 服务器),请参阅下文 [LDAPS(加密连接)](#ldaps加密连接) 章节。

3. 启动 `fe` 后,使用 `root` 或 `admin` 账号登录 Doris,设置 LDAP 管理员密码:

```sql
Expand Down Expand Up @@ -233,6 +235,52 @@ member: uid=jack,ou=aidp,dc=domain,dc=com
> - `user` 属于哪个 `group` 与 LDAP 树的组织结构无关。上文示例中的 `user2` 并不一定属于 `group2`。
> - 若想让 `user2` 属于 `group2`,需要在 `group2` 的 `member` 属性中显式添加 `user2`。

## LDAPS(加密连接)

:::info 自 4.0.5 版本开始支持
:::

默认情况下,Doris 通过明文 LDAP 协议与 LDAP 服务器通信。自 4.0.5 版本起,Doris 支持 LDAPS(LDAP over SSL/TLS),用于加密 Doris FE 与 LDAP 服务器之间的连接。

### 启用 LDAPS

在 `fe/conf/ldap.conf` 中添加以下配置以启用 LDAPS:

```
ldap_use_ssl = true
```

当 `ldap_use_ssl` 设置为 `true` 时,Doris 使用 `ldaps://` 协议连接 LDAP 服务器。同时应将端口更新为标准 LDAPS 端口(通常为 `636`):

```
ldap_host = ldap-host
ldap_port = 636
ldap_use_ssl = true
```

### 配置证书信任

使用 LDAPS 时,LDAP 服务器的 SSL 证书必须被 Doris FE 的 JVM 所信任。如果 LDAP 服务器使用的证书由知名公共 CA 签发,则无需额外配置。

如果使用自定义或自签名 CA,需要将 CA 证书导入 Java trustStore,并配置 JVM 使用该 trustStore。在 `fe/conf/fe.conf` 的 `JAVA_OPTS` 中添加以下参数:

```
# JDK 17 示例
JAVA_OPTS_FOR_JDK_17 = "-Djavax.net.ssl.trustStore=/path/to/your/cacerts -Djavax.net.ssl.trustStorePassword=changeit ..."
```

> **导入自签名 CA 证书的步骤:**
>
> 1. 获取 CA 证书文件(例如 `ca.crt`)。
> 2. 使用 `keytool` 将其导入 Java trustStore:
>
> ```shell
> keytool -importcert -alias ldap-ca -keystore /path/to/your/cacerts -file /path/to/ca.crt -storepass changeit -noprompt
> ```
>
> 3. 按上述方式在 `JAVA_OPTS` 中配置 trustStore 路径。
> 4. 重启 Doris FE 使配置生效。

## 缓存管理

为了避免频繁访问 LDAP 服务,Doris 会将 LDAP 信息缓存到内存中。
Expand All @@ -250,7 +298,8 @@ member: uid=jack,ou=aidp,dc=domain,dc=com

## 已知限制

- 目前 Doris 的 LDAP 功能仅支持明文密码验证,即用户登录时,密码在 `client` 与 `fe` 之间、`fe` 与 LDAP 服务之间均以明文形式传输。
- Doris 的 LDAP 功能在客户端到 FE 的链路上仅支持明文密码验证,即用户登录时,密码在 `client` 与 `fe` 之间以明文形式传输。客户端与 Doris FE 之间的 SSL/TLS 加密需要单独配置(详见[客户端连接](#第二步客户端连接))。
- FE 到 LDAP 服务器的链路默认使用明文传输(`ldap_use_ssl = false`)。如需加密该链路,请设置 `ldap_use_ssl = true` 以启用 LDAPS(详见 [LDAPS(加密连接)](#ldaps加密连接))。

## 常见问题

Expand All @@ -266,3 +315,12 @@ member: uid=jack,ou=aidp,dc=domain,dc=com
2. 检查预期的 `group` 是否位于 `ldap_group_basedn` 对应的组织结构下。
3. 检查预期的 `group` 是否包含 `member` 属性。
4. 检查预期 `group` 的 `member` 属性中是否包含当前用户的 `dn`。

**Q:LDAPS 连接失败,如何排查?**

按以下步骤逐项检查:

1. 确认 `fe/conf/ldap.conf` 中已设置 `ldap_use_ssl = true`。
2. 确认 `ldap_port` 已设置为正确的 LDAPS 端口(通常为 `636`)。
3. 检查 LDAP 服务器的 SSL 证书是否被 JVM 信任。查看 `fe.log` 中是否有 `SSLHandshakeException` 或 `PKIX path building failed` 等 SSL 握手错误。
4. 如果使用自签名 CA,确认 CA 证书已导入 trustStore,且 `JAVA_OPTS` 中的 trustStore 路径配置正确。
Loading
Loading