From a5a9167da4b3f42960e2885bb204bd3ba542116a Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 13:14:22 +0800 Subject: [PATCH 01/23] [design] Add FUSE local path configuration design for RESTCatalog Add design documents for supporting FUSE-mounted OSS paths in RESTCatalog. This allows users to access data through local file system paths without needing OSS tokens via getTableToken API. Configuration includes: - fuse.local-path.enabled: enable/disable FUSE path mapping - fuse.local-path.root: root local path for FUSE mount - fuse.local-path.database: database-level path mapping - fuse.local-path.table: table-level path mapping Security validation: - fuse.local-path.validation-mode: strict/warn/none - Option 1: Java NIO FileStore API (cross-platform FUSE detection) - Option 2: OSS data validation (recommended) - Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read OSS - Compares file size and content hash with local file - No REST API extension required - Graceful fallback to default FileIO on validation failure Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 685 ++++++++++++++++++++++++++ designs/fuse-local-path-design.md | 689 +++++++++++++++++++++++++++ 2 files changed, 1374 insertions(+) create mode 100644 designs/fuse-local-path-design-cn.md create mode 100644 designs/fuse-local-path-design.md diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md new file mode 100644 index 000000000000..20c998d06288 --- /dev/null +++ b/designs/fuse-local-path-design-cn.md @@ -0,0 +1,685 @@ + + +# RESTCatalog FUSE 本地路径配置设计 + +## 背景 + +在使用 Paimon RESTCatalog 访问 OSS（对象存储服务）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在 OSS 路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，无需 OSS 令牌。 + +本设计引入配置参数以支持 FUSE 挂载的 OSS 路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 + +## 目标 + +1. 支持通过本地文件系统访问 FUSE 挂载的 OSS 路径 +2. 支持分层路径映射：Catalog 根路径 > Database > Table +3. 当 FUSE 本地路径适用时，跳过 `getTableToken` API 调用 +4. 保持与现有 RESTCatalog 行为的向后兼容性 + +## 配置参数 + +所有参数定义在 `RESTCatalogOptions.java` 中： + +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | +| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/oss` | +| `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | +| `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | + +## 使用示例 + +### SQL 配置（Flink/Spark） + +```sql +CREATE CATALOG paimon_rest_catalog WITH ( + 'type' = 'paimon', + 'metastore' = 'rest', + 'uri' = 'http://rest-server:8080', + 'token' = 'xxx', + + -- FUSE 本地路径配置 + 'fuse.local-path.enabled' = 'true', + 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', + 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' +); +``` + +### 路径解析优先级 + +解析路径时，系统按以下顺序检查（优先级从高到低）： + +1. **Table 级别映射**（`fuse.local-path.table`） +2. **Database 级别映射**（`fuse.local-path.database`） +3. **根路径映射**（`fuse.local-path.root`） + +示例：对于表 `db1.table1`： +- 如果 `fuse.local-path.table` 包含 `db1.table1:/mnt/special/t1`，使用 `/mnt/special/t1` +- 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` +- 否则，使用 `fuse.local-path.root`（如 `/mnt/oss/warehouse`） + +## 实现方案 + +### RESTCatalog 修改 + +修改 `RESTCatalog.java` 中的 `fileIOForData` 方法： + +```java +private FileIO fileIOForData(Path path, Identifier identifier) { + // 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // 使用本地文件 IO，无需 token + return FileIO.get(localPath, CatalogContext.create(new Options(), context.hadoopConf())); + } + } + + // 原有逻辑：data token 或 ResolvingFileIO + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); +} + +/** + * 解析 FUSE 本地路径。优先级：table > database > root。 + * @return 本地路径，如果不适用则返回 null + */ +private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { + String pathStr = originalPath.toString(); + + // 1. 检查 Table 级别映射 + Map tableMappings = context.options().get(FUSE_LOCAL_PATH_TABLE); + String tableKey = identifier.getDatabaseName() + "." + identifier.getTableName(); + if (tableMappings.containsKey(tableKey)) { + String localRoot = tableMappings.get(tableKey); + return convertToLocalPath(pathStr, localRoot); + } + + // 2. 检查 Database 级别映射 + Map dbMappings = context.options().get(FUSE_LOCAL_PATH_DATABASE); + if (dbMappings.containsKey(identifier.getDatabaseName())) { + String localRoot = dbMappings.get(identifier.getDatabaseName()); + return convertToLocalPath(pathStr, localRoot); + } + + // 3. 使用根路径映射 + String fuseRoot = context.options().get(FUSE_LOCAL_PATH_ROOT); + if (fuseRoot != null) { + return convertToLocalPath(pathStr, fuseRoot); + } + + return null; +} + +private Path convertToLocalPath(String originalPath, String localRoot) { + // 将 OSS 路径转换为本地 FUSE 路径 + // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/oss/warehouse/db1/table1 + // 具体实现取决于路径结构 +} +``` + +### 行为矩阵 + +| 配置 | 路径匹配 | 行为 | +|-----|---------|------| +| `fuse.local-path.enabled=true` | 是 | 本地 FileIO，**无需调用 `getTableToken`** | +| `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | +| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | + +## 优势 + +1. **性能提升**：本地文件系统访问通常比基于网络的 OSS 访问更快 +2. **降低成本**：无需调用 `getTableToken` API，减少 REST 服务器负载 +3. **灵活性**：支持为不同的数据库/表配置不同的本地路径 +4. **向后兼容**：默认禁用，现有行为不变 + +## 安全校验机制 + +### 问题场景 + +错误的 FUSE 本地路径配置可能导致严重的数据一致性问题： + +| 场景 | 描述 | 后果 | +|-----|------|------| +| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到 OSS，导致数据丢失 | +| **OSS 路径错误** | 本地路径指向了其他库表的 OSS 路径 | 数据写入错误的表，导致数据污染 | + +### 校验方案 + +#### 1. 路径一致性校验（强校验） + +在首次访问表时，校验本地路径与 OSS 路径的一致性： + +```java +/** + * 校验 FUSE 本地路径与 OSS 路径的一致性 + * @throws IllegalArgumentException 如果路径不一致 + */ +private void validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { + // 1. 检查本地路径是否存在且为 FUSE 挂载点 + if (!isFUSEMountPoint(localPath)) { + throw new IllegalArgumentException( + String.format("FUSE local path '%s' is not a valid FUSE mount point. " + + "Data would be written to local disk instead of OSS!", localPath)); + } + + // 2. 校验路径标识一致性：通过读取本地路径下的 .paimon 表标识文件 + Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + if (fileIO.exists(localIdentifierFile)) { + String storedIdentifier = readIdentifier(localIdentifierFile); + String expectedIdentifier = identifier.getDatabaseName() + "." + identifier.getTableName(); + + if (!expectedIdentifier.equals(storedIdentifier)) { + throw new IllegalArgumentException( + String.format("FUSE path mismatch! Local path '%s' belongs to table '%s', " + + "but current table is '%s'.", + localPath, storedIdentifier, expectedIdentifier)); + } + } +} + +/** + * 检查路径是否为 FUSE 挂载点 + * 可通过检查 /proc/mounts (Linux) 或使用 stat 系统调用判断 + */ +private boolean isFUSEMountPoint(Path path) { + // 方案1: 检查 /proc/mounts 中是否包含该路径的 FUSE 挂载 + // 方案2: 检查路径的文件系统类型是否为 fuse.* + // 方案3: 通过读取 /etc/mtab 或使用 jnr-posix 库 + return checkFUSEMount(path); +} +``` + +#### 2. 表标识文件机制 + +在创建表时，自动在表目录下生成 `.paimon-identifier` 文件： + +``` +/mnt/oss/warehouse/db1/table1/ +├── .paimon-identifier # 内容: "db1.table1" +├── data-xxx.parquet +├── manifest-xxx +└── snapshot-xxx +``` + +标识文件内容： +``` +database=db1 +table=table1 +table-uuid=xxx-xxx-xxx +created-at=2026-03-13T00:00:00Z +``` + +#### 3. 校验模式配置 + +新增配置参数控制校验行为： + +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`（严格）、`warn`（警告）、`none`（不校验） | + +**校验模式说明**： + +| 模式 | 行为 | +|-----|------| +| `strict` | 启用校验，失败时抛出异常，阻止操作 | +| `warn` | 启用校验，失败时输出警告日志，但允许操作继续 | +| `none` | 不进行校验（不推荐，可能导致数据丢失或污染） | + +### 校验流程 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 访问表（getTable） │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ fuse.local-path.enabled == true ? │ +└─────────────────────────────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌───────────────────┐ ┌───────────────────┐ + │ 解析本地路径 │ │ 使用原有逻辑 │ + │ resolveFUSELocalPath│ │ (RESTTokenFileIO) │ + └───────────────────┘ └───────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ validation-mode != none ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌───────────────────┐ ┌───────────────────┐ + │ 校验 FUSE 挂载点 │ │ 跳过校验 │ + │ 校验路径一致性 │ │ 直接使用本地路径 │ + └───────────────────┘ └───────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 校验通过 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ 使用本地路径 │ │ validation-mode: │ + │ LocalFileIO │ │ - strict: 抛异常 │ + └─────────────┘ │ - warn: 警告+回退 │ + └─────────────────────┘ +``` + +### FUSE 挂载点检测实现 + +#### 方案一：Java NIO FileStore API + +```java +import java.nio.file.FileStore; +import java.nio.file.Files; +import java.nio.file.Path; + +/** + * 检测路径是否为 FUSE 挂载点（跨平台） + * 通过检查文件系统类型名称判断 + */ +private boolean isFUSEMountPoint(Path path) throws IOException { + FileStore store = Files.getFileStore(path); + + // 获取文件系统类型名称 + String type = store.type(); + String name = store.name(); + + // FUSE 文件系统类型通常包含 "fuse" 或特定标识 + // Linux: fuse.sshfs, fuseblk, fuse + // macOS: macfuse, sshfs, osxfuse + // 通用: fuse, FUSE + return type != null && ( + type.toLowerCase().contains("fuse") || + type.equalsIgnoreCase("sshfs") || + type.equalsIgnoreCase("nfs4") || + name.toLowerCase().contains("fuse") + ); +} +``` + +**平台兼容性**： + +| 平台 | FileStore.type() 示例 | +|------|----------------------| +| Linux | `fuse.sshfs`, `fuseblk`, `fuse` | +| macOS | `macfuse`, `osxfuse`, `sshfs` | +| Windows | `NTFS`, `FAT32` (不支持 FUSE，需第三方工具) | + +#### 方案二：OSS 数据校验（推荐） + +使用现有 FileIO 读取 OSS 文件，与本地文件比对验证路径正确性。 + +**完整实现**： + +```java +/** + * RESTCatalog 中 fileIOForData 的完整实现 + * 结合 FUSE 本地路径校验与 OSS 数据校验 + */ +private FileIO fileIOForData(Path path, Identifier identifier) { + // 如果 FUSE 本地路径启用，尝试使用本地路径 + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // 根据校验模式执行校验 + ValidationMode mode = getValidationMode(); + + if (mode != ValidationMode.NONE) { + ValidationResult result = validateFUSEPath(localPath, path, identifier); + if (!result.isValid()) { + handleValidationError(result, mode); + // 校验失败，回退到原有逻辑 + return createDefaultFileIO(path, identifier); + } + } + + // 校验通过或跳过校验，使用本地 FileIO + return createLocalFileIO(localPath); + } + } + + // 原有逻辑：data token 或 ResolvingFileIO + return createDefaultFileIO(path, identifier); +} + +/** + * 校验 FUSE 本地路径 + * 结合文件系统检测和 OSS 数据校验 + */ +private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { + // 1. 检查本地路径是否存在 + if (!Files.exists(localPath)) { + return ValidationResult.fail("Local path does not exist: " + localPath); + } + + // 2. 检查是否为 FUSE 挂载点（方案一） + if (!isFUSEMountPoint(localPath)) { + return ValidationResult.fail("Local path is not a FUSE mount point: " + localPath); + } + + // 3. OSS 数据校验（方案二，推荐） + return validateByOSSData(localPath, ossPath, identifier); +} + +/** + * 通过比对 OSS 和本地文件验证 FUSE 路径正确性 + * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取 OSS 文件 + */ +private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identifier identifier) { + try { + // 1. 获取 OSS FileIO（使用现有逻辑，可访问 OSS） + FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); + + // 2. 选择一个用于校验的文件（优先级：snapshot > schema > manifest） + ChecksumFile checksumFile = findChecksumFile(ossPath, ossFileIO); + if (checksumFile == null) { + // 表可能为空（新创建的表），跳过内容校验 + LOG.info("No checksum file found for table: {}, skip content validation", identifier); + return ValidationResult.success(); + } + + // 3. 读取 OSS 文件内容（仅读取前 N 字节计算 hash） + FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile.getFullPath()); + String ossHash = computeFileHash(ossFileIO, checksumFile.getFullPath(), HASH_CHECK_LENGTH); + + // 4. 构建本地文件路径并读取 + Path localChecksumFile = new Path(localPath, checksumFile.getRelativePath()); + java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); + + if (!Files.exists(localNioPath)) { + return ValidationResult.fail( + "Local file not found: " + localChecksumFile + + ". The FUSE path may not be mounted correctly or points to wrong location."); + } + + // 5. 读取本地文件内容 + long localSize = Files.size(localNioPath); + String localHash = computeLocalFileHash(localNioPath, HASH_CHECK_LENGTH); + + // 6. 比对文件特征 + if (localSize != ossStatus.getLen()) { + return ValidationResult.fail(String.format( + "File size mismatch! Local: %d bytes, OSS: %d bytes. " + + "The local path may point to a different table.", + localSize, ossStatus.getLen())); + } + + if (!localHash.equalsIgnoreCase(ossHash)) { + return ValidationResult.fail(String.format( + "File content hash mismatch! Local: %s, OSS: %s. " + + "The local path points to a different table.", + localHash, ossHash)); + } + + return ValidationResult.success(); + + } catch (Exception e) { + LOG.warn("Failed to validate FUSE path by OSS data for: {}", identifier, e); + return ValidationResult.fail("OSS data validation failed: " + e.getMessage()); + } +} + +/** + * 查找可用于校验的文件 + */ +private ChecksumFile findChecksumFile(Path tablePath, FileIO fileIO) { + // 优先级 1: snapshot 文件 + Path snapshotDir = new Path(tablePath, "snapshot"); + if (fileIO.exists(snapshotDir)) { + FileStatus[] snapshots = fileIO.listStatus(snapshotDir); + if (snapshots != null && snapshots.length > 0) { + // 返回最新的 snapshot 文件（按文件名排序） + Arrays.sort(snapshots, (a, b) -> b.getPath().getName().compareTo(a.getPath().getName())); + return new ChecksumFile(tablePath, snapshots[0].getPath()); + } + } + + // 优先级 2: schema 文件 + Path schemaFile = new Path(tablePath, "schema/schema-0"); + if (fileIO.exists(schemaFile)) { + return new ChecksumFile(tablePath, schemaFile); + } + + // 优先级 3: manifest 文件 + Path manifestDir = new Path(tablePath, "manifest"); + if (fileIO.exists(manifestDir)) { + FileStatus[] manifests = fileIO.listStatus(manifestDir); + if (manifests != null && manifests.length > 0) { + return new ChecksumFile(tablePath, manifests[0].getPath()); + } + } + + return null; +} + +/** + * 计算 OSS 文件内容哈希（仅读取前 N 字节） + */ +private String computeFileHash(FileIO fileIO, Path file, int length) throws IOException { + try (InputStream is = fileIO.newInputStream(file); + DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { + byte[] buffer = new byte[length]; + dis.read(buffer); + byte[] hash = dis.getMessageDigest().digest(); + return Hex.encodeHexString(hash); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); + } +} + +/** + * 计算本地文件内容哈希（仅读取前 N 字节） + */ +private String computeLocalFileHash(java.nio.file.Path file, int length) throws IOException { + try (InputStream is = Files.newInputStream(file); + DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { + byte[] buffer = new byte[length]; + dis.read(buffer); + byte[] hash = dis.getMessageDigest().digest(); + return Hex.encodeHexString(hash); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); + } +} + +/** + * 处理校验错误 + */ +private void handleValidationError(ValidationResult result, ValidationMode mode) { + String errorMsg = "FUSE local path validation failed: " + result.getErrorMessage(); + + switch (mode) { + case STRICT: + throw new IllegalArgumentException(errorMsg); + case WARN: + LOG.warn(errorMsg + ". Falling back to default FileIO."); + break; + case NONE: + // 不会执行到这里 + break; + } +} + +/** + * 创建本地 FileIO + */ +private FileIO createLocalFileIO(Path localPath) { + return FileIO.get(localPath, CatalogContext.create( + new Options(), + context.hadoopConf() + )); +} + +/** + * 创建默认 FileIO（原有逻辑） + */ +private FileIO createDefaultFileIO(Path path, Identifier identifier) { + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); +} + +private static final int HASH_CHECK_LENGTH = 4096; // 校验前 4KB + +// ========== 辅助类 ========== + +enum ValidationMode { + STRICT, // 严格模式：校验失败抛异常 + WARN, // 警告模式：校验失败只警告，回退到默认逻辑 + NONE // 不校验 +} + +class ValidationResult { + private final boolean valid; + private final String errorMessage; + + private ValidationResult(boolean valid, String errorMessage) { + this.valid = valid; + this.errorMessage = errorMessage; + } + + static ValidationResult success() { + return new ValidationResult(true, null); + } + + static ValidationResult fail(String errorMessage) { + return new ValidationResult(false, errorMessage); + } + + boolean isValid() { return valid; } + String getErrorMessage() { return errorMessage; } +} + +class ChecksumFile { + private final Path tablePath; + private final Path fullPath; + + ChecksumFile(Path tablePath, Path fullPath) { + this.tablePath = tablePath; + this.fullPath = fullPath; + } + + Path getFullPath() { return fullPath; } + + String getRelativePath() { + return new Path(tablePath, fullPath.getName()).toString(); + } +} +``` + +**方案优势**： + +| 优势 | 说明 | +|------|------| +| **无需扩展 API** | 使用现有 FileIO 读取 OSS 文件 | +| **准确性最高** | 直接验证数据一致性，100% 确保路径正确 | +| **双重保障** | FUSE 挂载检测 + OSS 数据比对 | +| **防止数据污染** | 可检测路径指向错误表的情况 | +| **优雅降级** | 校验失败可回退到默认 FileIO | + +**完整校验流程**： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ OSS 数据校验流程 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 1. 通过 REST API 获取表的 OSS 路径信息 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 2. 选择校验文件（snapshot/manifest/schema） │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 3. 获取 OSS 文件元数据（大小、修改时间、hash） │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 4. 读取本地对应文件 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 本地文件存在 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → 校验失败（路径错误或未挂载） + │ + ▼ + ┌───────────────────────────────────────┐ + │ 文件大小匹配 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → 校验失败 + │ + ▼ + ┌───────────────────────────────────────┐ + │ 文件内容 hash 匹配 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → 校验失败（路径指向错误表） + │ + ▼ + ┌─────────────┐ + │ 校验通过 │ + │ 可安全使用 │ + └─────────────┘ +``` + +### 使用示例（启用安全校验） + +```sql +CREATE CATALOG paimon_rest_catalog WITH ( + 'type' = 'paimon', + 'metastore' = 'rest', + 'uri' = 'http://rest-server:8080', + 'token' = 'xxx', + + -- FUSE 本地路径配置 + 'fuse.local-path.enabled' = 'true', + 'fuse.local-path.root' = '/mnt/oss/warehouse', + + -- 安全校验配置（可选，默认 strict） + 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none +); +``` + +## 限制 + +1. FUSE 挂载必须正确配置且可访问 +2. 本地路径必须与 OSS 路径具有相同的目录结构 +3. 写操作需要本地 FUSE 挂载点具有适当的权限 +4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md new file mode 100644 index 000000000000..fad5e9899e63 --- /dev/null +++ b/designs/fuse-local-path-design.md @@ -0,0 +1,689 @@ + + +# FUSE Local Path Configuration for RESTCatalog + +## Background + +When using Paimon RESTCatalog with OSS (Object Storage Service), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where OSS paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths without needing OSS tokens. + +This design introduces configuration parameters to support FUSE-mounted OSS paths, allowing users to specify local path mappings at catalog, database, and table levels. + +## Goals + +1. Enable local file system access for FUSE-mounted OSS paths +2. Support hierarchical path mapping: catalog root > database > table +3. Skip `getTableToken` API calls when FUSE local path is applicable +4. Maintain backward compatibility with existing RESTCatalog behavior + +## Configuration Parameters + +All parameters are defined in `RESTCatalogOptions.java`: + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `fuse.local-path.enabled` | Boolean | `false` | Whether to enable FUSE local path mapping for OSS paths | +| `fuse.local-path.root` | String | (none) | The root local path for FUSE-mounted OSS, e.g., `/mnt/oss` | +| `fuse.local-path.database` | Map | `{}` | Database-level local path mapping. Format: `db1:/local/path1,db2:/local/path2` | +| `fuse.local-path.table` | Map | `{}` | Table-level local path mapping. Format: `db1.table1:/local/path1,db2.table2:/local/path2` | + +## Usage Example + +### SQL Configuration (Flink/Spark) + +```sql +CREATE CATALOG paimon_rest_catalog WITH ( + 'type' = 'paimon', + 'metastore' = 'rest', + 'uri' = 'http://rest-server:8080', + 'token' = 'xxx', + + -- FUSE local path configuration + 'fuse.local-path.enabled' = 'true', + 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', + 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' +); +``` + +### Path Resolution Priority + +When resolving a path, the system checks in the following order (higher priority wins): + +1. **Table-level mapping** (`fuse.local-path.table`) +2. **Database-level mapping** (`fuse.local-path.database`) +3. **Root mapping** (`fuse.local-path.root`) + +Example: For table `db1.table1`: +- If `fuse.local-path.table` contains `db1.table1:/mnt/special/t1`, use `/mnt/special/t1` +- Else if `fuse.local-path.database` contains `db1:/mnt/custom/db1`, use `/mnt/custom/db1` +- Else use `fuse.local-path.root` (e.g., `/mnt/oss/warehouse`) + +## Implementation + +### RESTCatalog Modification + +The `fileIOForData` method in `RESTCatalog.java` will be modified: + +```java +private FileIO fileIOForData(Path path, Identifier identifier) { + // If FUSE local path is enabled and path matches, use local FileIO + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // Use local file IO, no token needed + return FileIO.get(localPath, CatalogContext.create(new Options(), context.hadoopConf())); + } + } + + // Original logic: data token or ResolvingFileIO + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); +} + +/** + * Resolve FUSE local path. Priority: table > database > root. + * @return Local path, or null if not applicable + */ +private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { + String pathStr = originalPath.toString(); + + // 1. Check table-level mapping + Map tableMappings = context.options().get(FUSE_LOCAL_PATH_TABLE); + String tableKey = identifier.getDatabaseName() + "." + identifier.getTableName(); + if (tableMappings.containsKey(tableKey)) { + String localRoot = tableMappings.get(tableKey); + return convertToLocalPath(pathStr, localRoot); + } + + // 2. Check database-level mapping + Map dbMappings = context.options().get(FUSE_LOCAL_PATH_DATABASE); + if (dbMappings.containsKey(identifier.getDatabaseName())) { + String localRoot = dbMappings.get(identifier.getDatabaseName()); + return convertToLocalPath(pathStr, localRoot); + } + + // 3. Use root mapping + String fuseRoot = context.options().get(FUSE_LOCAL_PATH_ROOT); + if (fuseRoot != null) { + return convertToLocalPath(pathStr, fuseRoot); + } + + return null; +} + +private Path convertToLocalPath(String originalPath, String localRoot) { + // Convert OSS path to local FUSE path + // Example: oss://bucket/warehouse/db1/table1 -> /mnt/oss/warehouse/db1/table1 + // Implementation depends on path structure +} +``` + +### Behavior Matrix + +| Configuration | Path Match | Behavior | +|---------------|------------|----------| +| `fuse.local-path.enabled=true` | Yes | Local FileIO, **no `getTableToken` call** | +| `fuse.local-path.enabled=true` | No | Fallback to original logic | +| `fuse.local-path.enabled=false` | N/A | Original logic (data token or ResolvingFileIO) | + +## Benefits + +1. **Performance**: Local file system access is typically faster than network-based OSS access +2. **Cost Reduction**: No need to call `getTableToken` API, reducing REST server load +3. **Flexibility**: Supports different local paths for different databases/tables +4. **Backward Compatibility**: Disabled by default, existing behavior unchanged + +## Security Validation Mechanism + +### Problem Scenarios + +Incorrect FUSE local path configuration can lead to serious data consistency issues: + +| Scenario | Description | Consequence | +|----------|-------------|-------------| +| **Local path not mounted** | User's configured `/local/table` is not actually FUSE-mounted | Data is written only to local disk, not synced to OSS, causing data loss | +| **OSS path mismatch** | Local path points to a different table's OSS path | Data is written to the wrong table, causing data pollution | + +### Validation Scheme + +#### 1. Path Consistency Validation (Strong Validation) + +Validate consistency between local path and OSS path when first accessing a table: + +```java +/** + * Validate consistency between FUSE local path and OSS path + * @throws IllegalArgumentException if paths are inconsistent + */ +private void validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { + // 1. Check if local path exists and is a FUSE mount point + if (!isFUSEMountPoint(localPath)) { + throw new IllegalArgumentException( + String.format("FUSE local path '%s' is not a valid FUSE mount point. " + + "Data would be written to local disk instead of OSS!", localPath)); + } + + // 2. Validate path identifier consistency: read .paimon table identifier file + Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + if (fileIO.exists(localIdentifierFile)) { + String storedIdentifier = readIdentifier(localIdentifierFile); + String expectedIdentifier = identifier.getDatabaseName() + "." + identifier.getTableName(); + + if (!expectedIdentifier.equals(storedIdentifier)) { + throw new IllegalArgumentException( + String.format("FUSE path mismatch! Local path '%s' belongs to table '%s', " + + "but current table is '%s'.", + localPath, storedIdentifier, expectedIdentifier)); + } + } +} + +/** + * Check if path is a FUSE mount point + * Can be determined by checking /proc/mounts (Linux) or using stat system call + */ +private boolean isFUSEMountPoint(Path path) { + // Option 1: Check /proc/mounts for FUSE mount of this path + // Option 2: Check if filesystem type is fuse.* + // Option 3: Read /etc/mtab or use jnr-posix library + return checkFUSEMount(path); +} +``` + +#### 2. Table Identifier File Mechanism + +When creating a table, automatically generate a `.paimon-identifier` file in the table directory: + +``` +/mnt/oss/warehouse/db1/table1/ +├── .paimon-identifier # Content: "db1.table1" +├── data-xxx.parquet +├── manifest-xxx +└── snapshot-xxx +``` + +Identifier file content: +``` +database=db1 +table=table1 +table-uuid=xxx-xxx-xxx +created-at=2026-03-13T00:00:00Z +``` + +#### 3. Validation Mode Configuration + +New configuration parameter to control validation behavior: + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `fuse.local-path.validation-mode` | String | `strict` | Validation mode: `strict`, `warn`, or `none` | + +**Validation Mode Description**: + +| Mode | Behavior | +|------|----------| +| `strict` | Enable validation, throw exception on failure, block the operation | +| `warn` | Enable validation, log warning on failure, but allow operation to proceed | +| `none` | No validation (not recommended, may cause data loss or pollution) | + +### Validation Flow + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Access Table (getTable) │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ fuse.local-path.enabled == true ? │ +└─────────────────────────────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌───────────────────┐ ┌───────────────────┐ + │ Resolve local path│ │ Use original logic│ + │ resolveFUSELocalPath│ │ (RESTTokenFileIO) │ + └───────────────────┘ └───────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ validation-mode != none ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌───────────────────┐ ┌───────────────────┐ + │ Validate FUSE │ │ Skip validation │ + │ mount point │ │ Use local path │ + │ Validate path │ │ directly │ + │ consistency │ │ │ + └───────────────────┘ └───────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Validation passed ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ Use local │ │ validation-mode: │ + │ path │ │ - strict: throw │ + │ LocalFileIO │ │ - warn: warn+fallback│ + └─────────────┘ └─────────────────────┘ +``` + +### FUSE Mount Point Detection Implementation + +#### Option 1: Java NIO FileStore API + +```java +import java.nio.file.FileStore; +import java.nio.file.Files; +import java.nio.file.Path; + +/** + * Detect if path is a FUSE mount point (cross-platform) + * Check filesystem type name + */ +private boolean isFUSEMountPoint(Path path) throws IOException { + FileStore store = Files.getFileStore(path); + + // Get filesystem type name + String type = store.type(); + String name = store.name(); + + // FUSE filesystem types typically contain "fuse" or specific identifiers + // Linux: fuse.sshfs, fuseblk, fuse + // macOS: macfuse, sshfs, osxfuse + // Generic: fuse, FUSE + return type != null && ( + type.toLowerCase().contains("fuse") || + type.equalsIgnoreCase("sshfs") || + type.equalsIgnoreCase("nfs4") || + name.toLowerCase().contains("fuse") + ); +} +``` + +**Platform Compatibility**: + +| Platform | FileStore.type() Examples | +|----------|---------------------------| +| Linux | `fuse.sshfs`, `fuseblk`, `fuse` | +| macOS | `macfuse`, `osxfuse`, `sshfs` | +| Windows | `NTFS`, `FAT32` (FUSE not natively supported, requires third-party tools) | + +#### Option 2: OSS Data Validation (Recommended) + +Use existing FileIO to read OSS files and compare with local files to validate path correctness. + +**Complete Implementation**: + +```java +/** + * Complete implementation of fileIOForData in RESTCatalog + * Combining FUSE local path validation with OSS data validation + */ +private FileIO fileIOForData(Path path, Identifier identifier) { + // If FUSE local path is enabled, try using local path + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // Execute validation based on validation mode + ValidationMode mode = getValidationMode(); + + if (mode != ValidationMode.NONE) { + ValidationResult result = validateFUSEPath(localPath, path, identifier); + if (!result.isValid()) { + handleValidationError(result, mode); + // Validation failed, fallback to default logic + return createDefaultFileIO(path, identifier); + } + } + + // Validation passed or skipped, use local FileIO + return createLocalFileIO(localPath); + } + } + + // Original logic: data token or ResolvingFileIO + return createDefaultFileIO(path, identifier); +} + +/** + * Validate FUSE local path + * Combining filesystem detection and OSS data validation + */ +private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { + // 1. Check if local path exists + if (!Files.exists(localPath)) { + return ValidationResult.fail("Local path does not exist: " + localPath); + } + + // 2. Check if it's a FUSE mount point (Option 1) + if (!isFUSEMountPoint(localPath)) { + return ValidationResult.fail("Local path is not a FUSE mount point: " + localPath); + } + + // 3. OSS data validation (Option 2, recommended) + return validateByOSSData(localPath, ossPath, identifier); +} + +/** + * Validate FUSE path correctness by comparing OSS and local file data + * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read OSS files + */ +private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identifier identifier) { + try { + // 1. Get OSS FileIO (using existing logic, can access OSS) + FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); + + // 2. Select a file for validation (priority: snapshot > schema > manifest) + ChecksumFile checksumFile = findChecksumFile(ossPath, ossFileIO); + if (checksumFile == null) { + // Table may be empty (newly created), skip content validation + LOG.info("No checksum file found for table: {}, skip content validation", identifier); + return ValidationResult.success(); + } + + // 3. Read OSS file content (only read first N bytes for hash) + FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile.getFullPath()); + String ossHash = computeFileHash(ossFileIO, checksumFile.getFullPath(), HASH_CHECK_LENGTH); + + // 4. Build local file path and read + Path localChecksumFile = new Path(localPath, checksumFile.getRelativePath()); + java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); + + if (!Files.exists(localNioPath)) { + return ValidationResult.fail( + "Local file not found: " + localChecksumFile + + ". The FUSE path may not be mounted correctly or points to wrong location."); + } + + // 5. Read local file content + long localSize = Files.size(localNioPath); + String localHash = computeLocalFileHash(localNioPath, HASH_CHECK_LENGTH); + + // 6. Compare file features + if (localSize != ossStatus.getLen()) { + return ValidationResult.fail(String.format( + "File size mismatch! Local: %d bytes, OSS: %d bytes. " + + "The local path may point to a different table.", + localSize, ossStatus.getLen())); + } + + if (!localHash.equalsIgnoreCase(ossHash)) { + return ValidationResult.fail(String.format( + "File content hash mismatch! Local: %s, OSS: %s. " + + "The local path points to a different table.", + localHash, ossHash)); + } + + return ValidationResult.success(); + + } catch (Exception e) { + LOG.warn("Failed to validate FUSE path by OSS data for: {}", identifier, e); + return ValidationResult.fail("OSS data validation failed: " + e.getMessage()); + } +} + +/** + * Find a file suitable for validation + */ +private ChecksumFile findChecksumFile(Path tablePath, FileIO fileIO) { + // Priority 1: snapshot file + Path snapshotDir = new Path(tablePath, "snapshot"); + if (fileIO.exists(snapshotDir)) { + FileStatus[] snapshots = fileIO.listStatus(snapshotDir); + if (snapshots != null && snapshots.length > 0) { + // Return latest snapshot file (sorted by filename) + Arrays.sort(snapshots, (a, b) -> b.getPath().getName().compareTo(a.getPath().getName())); + return new ChecksumFile(tablePath, snapshots[0].getPath()); + } + } + + // Priority 2: schema file + Path schemaFile = new Path(tablePath, "schema/schema-0"); + if (fileIO.exists(schemaFile)) { + return new ChecksumFile(tablePath, schemaFile); + } + + // Priority 3: manifest file + Path manifestDir = new Path(tablePath, "manifest"); + if (fileIO.exists(manifestDir)) { + FileStatus[] manifests = fileIO.listStatus(manifestDir); + if (manifests != null && manifests.length > 0) { + return new ChecksumFile(tablePath, manifests[0].getPath()); + } + } + + return null; +} + +/** + * Compute OSS file content hash (only read first N bytes) + */ +private String computeFileHash(FileIO fileIO, Path file, int length) throws IOException { + try (InputStream is = fileIO.newInputStream(file); + DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { + byte[] buffer = new byte[length]; + dis.read(buffer); + byte[] hash = dis.getMessageDigest().digest(); + return Hex.encodeHexString(hash); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); + } +} + +/** + * Compute local file content hash (only read first N bytes) + */ +private String computeLocalFileHash(java.nio.file.Path file, int length) throws IOException { + try (InputStream is = Files.newInputStream(file); + DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { + byte[] buffer = new byte[length]; + dis.read(buffer); + byte[] hash = dis.getMessageDigest().digest(); + return Hex.encodeHexString(hash); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); + } +} + +/** + * Handle validation error + */ +private void handleValidationError(ValidationResult result, ValidationMode mode) { + String errorMsg = "FUSE local path validation failed: " + result.getErrorMessage(); + + switch (mode) { + case STRICT: + throw new IllegalArgumentException(errorMsg); + case WARN: + LOG.warn(errorMsg + ". Falling back to default FileIO."); + break; + case NONE: + // Won't reach here + break; + } +} + +/** + * Create local FileIO + */ +private FileIO createLocalFileIO(Path localPath) { + return FileIO.get(localPath, CatalogContext.create( + new Options(), + context.hadoopConf() + )); +} + +/** + * Create default FileIO (original logic) + */ +private FileIO createDefaultFileIO(Path path, Identifier identifier) { + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); +} + +private static final int HASH_CHECK_LENGTH = 4096; // Check first 4KB + +// ========== Helper Classes ========== + +enum ValidationMode { + STRICT, // Strict mode: throw exception on validation failure + WARN, // Warn mode: log warning on failure, fallback to default logic + NONE // No validation +} + +class ValidationResult { + private final boolean valid; + private final String errorMessage; + + private ValidationResult(boolean valid, String errorMessage) { + this.valid = valid; + this.errorMessage = errorMessage; + } + + static ValidationResult success() { + return new ValidationResult(true, null); + } + + static ValidationResult fail(String errorMessage) { + return new ValidationResult(false, errorMessage); + } + + boolean isValid() { return valid; } + String getErrorMessage() { return errorMessage; } +} + +class ChecksumFile { + private final Path tablePath; + private final Path fullPath; + + ChecksumFile(Path tablePath, Path fullPath) { + this.tablePath = tablePath; + this.fullPath = fullPath; + } + + Path getFullPath() { return fullPath; } + + String getRelativePath() { + return new Path(tablePath, fullPath.getName()).toString(); + } +} +``` + +**Advantages**: + +| Advantage | Description | +|-----------|-------------| +| **No API Extension Needed** | Uses existing FileIO to read OSS files | +| **Most Accurate** | Directly validates data consistency, 100% ensures path correctness | +| **Dual Protection** | FUSE mount detection + OSS data comparison | +| **Prevent Data Pollution** | Can detect when path points to wrong table | +| **Graceful Degradation** | Validation failure falls back to default FileIO | + +**Complete Validation Flow**: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ OSS Data Validation Flow │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 1. Get table's OSS path info via REST API │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 2. Select validation file (snapshot/manifest/schema) │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 3. Get OSS file metadata (size, mtime, hash) │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 4. Read local corresponding file │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Local file exists ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → Validation failed (wrong path or not mounted) + │ + ▼ + ┌───────────────────────────────────────┐ + │ File size matches ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → Validation failed + │ + ▼ + ┌───────────────────────────────────────┐ + │ File content hash matches ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → Validation failed (path points to wrong table) + │ + ▼ + ┌─────────────┐ + │ Validation │ + │ passed │ + │ Safe to use │ + └─────────────┘ +``` + +### Usage Example (with Security Validation) + +```sql +CREATE CATALOG paimon_rest_catalog WITH ( + 'type' = 'paimon', + 'metastore' = 'rest', + 'uri' = 'http://rest-server:8080', + 'token' = 'xxx', + + -- FUSE local path configuration + 'fuse.local-path.enabled' = 'true', + 'fuse.local-path.root' = '/mnt/oss/warehouse', + + -- Security validation configuration (optional, default: strict) + 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none +); +``` + +## Limitations + +1. FUSE mount must be properly configured and accessible +2. Local path must have the same directory structure as the OSS path +3. Write operations require proper permissions on the local FUSE mount +4. Windows platform has limited FUSE support (requires third-party tools like WinFsp) + From 69d4bcd7235a79f02165a48556658db94afe1c16 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 14:38:44 +0800 Subject: [PATCH 02/23] [design] Simplify FUSE validation: remove NIO FileStore option, use SnapshotManager/SchemaManager - Remove Java NIO FileStore API option (Option 1) - Use SnapshotManager.latestSnapshot() to get latest snapshot directly - Use SchemaManager.latest() as fallback for new tables - Remove custom file traversal logic, use existing Paimon APIs - Simplify validation code and improve maintainability Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 238 +++++++++++---------------- designs/fuse-local-path-design.md | 236 +++++++++++--------------- 2 files changed, 191 insertions(+), 283 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 20c998d06288..efbc36928b6a 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -291,50 +291,9 @@ created-at=2026-03-13T00:00:00Z └─────────────────────┘ ``` -### FUSE 挂载点检测实现 +### 安全校验实现 -#### 方案一：Java NIO FileStore API - -```java -import java.nio.file.FileStore; -import java.nio.file.Files; -import java.nio.file.Path; - -/** - * 检测路径是否为 FUSE 挂载点（跨平台） - * 通过检查文件系统类型名称判断 - */ -private boolean isFUSEMountPoint(Path path) throws IOException { - FileStore store = Files.getFileStore(path); - - // 获取文件系统类型名称 - String type = store.type(); - String name = store.name(); - - // FUSE 文件系统类型通常包含 "fuse" 或特定标识 - // Linux: fuse.sshfs, fuseblk, fuse - // macOS: macfuse, sshfs, osxfuse - // 通用: fuse, FUSE - return type != null && ( - type.toLowerCase().contains("fuse") || - type.equalsIgnoreCase("sshfs") || - type.equalsIgnoreCase("nfs4") || - name.toLowerCase().contains("fuse") - ); -} -``` - -**平台兼容性**： - -| 平台 | FileStore.type() 示例 | -|------|----------------------| -| Linux | `fuse.sshfs`, `fuseblk`, `fuse` | -| macOS | `macfuse`, `osxfuse`, `sshfs` | -| Windows | `NTFS`, `FAT32` (不支持 FUSE，需第三方工具) | - -#### 方案二：OSS 数据校验（推荐） - -使用现有 FileIO 读取 OSS 文件，与本地文件比对验证路径正确性。 +使用 OSS 数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取 OSS 文件，与本地文件比对。 **完整实现**： @@ -371,20 +330,15 @@ private FileIO fileIOForData(Path path, Identifier identifier) { /** * 校验 FUSE 本地路径 - * 结合文件系统检测和 OSS 数据校验 */ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { // 1. 检查本地路径是否存在 - if (!Files.exists(localPath)) { + java.nio.file.Path localNioPath = java.nio.file.Paths.get(localPath.toUri()); + if (!Files.exists(localNioPath)) { return ValidationResult.fail("Local path does not exist: " + localPath); } - // 2. 检查是否为 FUSE 挂载点（方案一） - if (!isFUSEMountPoint(localPath)) { - return ValidationResult.fail("Local path is not a FUSE mount point: " + localPath); - } - - // 3. OSS 数据校验（方案二，推荐） + // 2. OSS 数据校验 return validateByOSSData(localPath, ossPath, identifier); } @@ -397,44 +351,53 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif // 1. 获取 OSS FileIO（使用现有逻辑，可访问 OSS） FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); - // 2. 选择一个用于校验的文件（优先级：snapshot > schema > manifest） - ChecksumFile checksumFile = findChecksumFile(ossPath, ossFileIO); - if (checksumFile == null) { - // 表可能为空（新创建的表），跳过内容校验 - LOG.info("No checksum file found for table: {}, skip content validation", identifier); - return ValidationResult.success(); + // 2. 使用 SnapshotManager 获取最新 snapshot + SnapshotManager snapshotManager = new SnapshotManager(ossFileIO, ossPath); + Snapshot latestSnapshot = snapshotManager.latestSnapshot(); + + Path checksumFile; + if (latestSnapshot != null) { + // 有 snapshot，使用 snapshot 文件校验 + checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); + } else { + // 无 snapshot（新表），使用 schema 文件校验 + SchemaManager schemaManager = new SchemaManager(ossFileIO, ossPath); + Optional latestSchema = schemaManager.latest(); + if (!latestSchema.isPresent()) { + // 表完全为空（连 schema 都没有，理论上不应该发生） + LOG.info("No snapshot or schema found for table: {}, skip validation", identifier); + return ValidationResult.success(); + } + checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); } - // 3. 读取 OSS 文件内容（仅读取前 N 字节计算 hash） - FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile.getFullPath()); - String ossHash = computeFileHash(ossFileIO, checksumFile.getFullPath(), HASH_CHECK_LENGTH); + // 3. 读取 OSS 文件内容并计算 hash + FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); + String ossHash = computeFileHash(ossFileIO, checksumFile); - // 4. 构建本地文件路径并读取 - Path localChecksumFile = new Path(localPath, checksumFile.getRelativePath()); + // 4. 读取本地文件并计算 hash + Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); if (!Files.exists(localNioPath)) { return ValidationResult.fail( "Local file not found: " + localChecksumFile + - ". The FUSE path may not be mounted correctly or points to wrong location."); + ". The FUSE path may not be mounted correctly."); } - // 5. 读取本地文件内容 long localSize = Files.size(localNioPath); - String localHash = computeLocalFileHash(localNioPath, HASH_CHECK_LENGTH); + String localHash = computeLocalFileHash(localNioPath); - // 6. 比对文件特征 + // 5. 比对文件特征 if (localSize != ossStatus.getLen()) { return ValidationResult.fail(String.format( - "File size mismatch! Local: %d bytes, OSS: %d bytes. " + - "The local path may point to a different table.", + "File size mismatch! Local: %d bytes, OSS: %d bytes.", localSize, ossStatus.getLen())); } if (!localHash.equalsIgnoreCase(ossHash)) { return ValidationResult.fail(String.format( - "File content hash mismatch! Local: %s, OSS: %s. " + - "The local path points to a different table.", + "File content hash mismatch! Local: %s, OSS: %s.", localHash, ossHash)); } @@ -447,66 +410,45 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif } /** - * 查找可用于校验的文件 + * 计算 FileIO 文件内容哈希 */ -private ChecksumFile findChecksumFile(Path tablePath, FileIO fileIO) { - // 优先级 1: snapshot 文件 - Path snapshotDir = new Path(tablePath, "snapshot"); - if (fileIO.exists(snapshotDir)) { - FileStatus[] snapshots = fileIO.listStatus(snapshotDir); - if (snapshots != null && snapshots.length > 0) { - // 返回最新的 snapshot 文件（按文件名排序） - Arrays.sort(snapshots, (a, b) -> b.getPath().getName().compareTo(a.getPath().getName())); - return new ChecksumFile(tablePath, snapshots[0].getPath()); - } - } - - // 优先级 2: schema 文件 - Path schemaFile = new Path(tablePath, "schema/schema-0"); - if (fileIO.exists(schemaFile)) { - return new ChecksumFile(tablePath, schemaFile); +private String computeFileHash(FileIO fileIO, Path file) throws IOException { + MessageDigest md; + try { + md = MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); } - // 优先级 3: manifest 文件 - Path manifestDir = new Path(tablePath, "manifest"); - if (fileIO.exists(manifestDir)) { - FileStatus[] manifests = fileIO.listStatus(manifestDir); - if (manifests != null && manifests.length > 0) { - return new ChecksumFile(tablePath, manifests[0].getPath()); + try (InputStream is = fileIO.newInputStream(file)) { + byte[] buffer = new byte[4096]; + int bytesRead; + while ((bytesRead = is.read(buffer)) != -1) { + md.update(buffer, 0, bytesRead); } } - - return null; + return Hex.encodeHexString(md.digest()); } /** - * 计算 OSS 文件内容哈希（仅读取前 N 字节） + * 计算本地文件内容哈希 */ -private String computeFileHash(FileIO fileIO, Path file, int length) throws IOException { - try (InputStream is = fileIO.newInputStream(file); - DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { - byte[] buffer = new byte[length]; - dis.read(buffer); - byte[] hash = dis.getMessageDigest().digest(); - return Hex.encodeHexString(hash); +private String computeLocalFileHash(java.nio.file.Path file) throws IOException { + MessageDigest md; + try { + md = MessageDigest.getInstance("MD5"); } catch (NoSuchAlgorithmException e) { throw new IOException("MD5 algorithm not available", e); } -} -/** - * 计算本地文件内容哈希（仅读取前 N 字节） - */ -private String computeLocalFileHash(java.nio.file.Path file, int length) throws IOException { - try (InputStream is = Files.newInputStream(file); - DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { - byte[] buffer = new byte[length]; - dis.read(buffer); - byte[] hash = dis.getMessageDigest().digest(); - return Hex.encodeHexString(hash); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); + try (InputStream is = Files.newInputStream(file)) { + byte[] buffer = new byte[4096]; + int bytesRead; + while ((bytesRead = is.read(buffer)) != -1) { + md.update(buffer, 0, bytesRead); + } } + return Hex.encodeHexString(md.digest()); } /** @@ -546,8 +488,6 @@ private FileIO createDefaultFileIO(Path path, Identifier identifier) { : fileIOFromOptions(path); } -private static final int HASH_CHECK_LENGTH = 4096; // 校验前 4KB - // ========== 辅助类 ========== enum ValidationMode { @@ -576,34 +516,26 @@ class ValidationResult { boolean isValid() { return valid; } String getErrorMessage() { return errorMessage; } } - -class ChecksumFile { - private final Path tablePath; - private final Path fullPath; - - ChecksumFile(Path tablePath, Path fullPath) { - this.tablePath = tablePath; - this.fullPath = fullPath; - } - - Path getFullPath() { return fullPath; } - - String getRelativePath() { - return new Path(tablePath, fullPath.getName()).toString(); - } -} ``` **方案优势**： | 优势 | 说明 | |------|------| -| **无需扩展 API** | 使用现有 FileIO 读取 OSS 文件 | -| **准确性最高** | 直接验证数据一致性，100% 确保路径正确 | -| **双重保障** | FUSE 挂载检测 + OSS 数据比对 | -| **防止数据污染** | 可检测路径指向错误表的情况 | +| **无需扩展 API** | 使用现有 FileIO 和 SnapshotManager/SchemaManager | +| **使用 LATEST snapshot** | 通过 `SnapshotManager.latestSnapshot()` 直接获取，无需遍历 | +| **新表支持** | 无 snapshot 时自动回退到 schema 文件校验 | +| **准确性最高** | 直接验证数据一致性，确保路径正确 | | **优雅降级** | 校验失败可回退到默认 FileIO | +**校验文件选择逻辑**： + +| 场景 | 校验文件 | +|------|----------| +| 有 snapshot | 使用 `SnapshotManager.latestSnapshot()` 获取的最新 snapshot 文件 | +| 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | +| 无 schema（理论上不存在）| 跳过校验 | + **完整校验流程**： ``` @@ -613,22 +545,44 @@ class ChecksumFile { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 1. 通过 REST API 获取表的 OSS 路径信息 │ +│ 1. 获取 OSS FileIO（RESTTokenFileIO 或 ResolvingFileIO） │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 2. 选择校验文件（snapshot/manifest/schema） │ +│ 2. 通过 SnapshotManager 获取最新 snapshot │ +│ snapshotManager.latestSnapshot() │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ Snapshot 存在 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ 使用 snapshot 文件 │ │ 通过 SchemaManager 获取最新 schema │ +│ 进行校验 │ │ schemaManager.latest() │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Schema 存在 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → 跳过校验（空表） + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 3. 获取 OSS 文件元数据（大小、修改时间、hash） │ +│ 3. 获取 OSS 文件元数据（大小） │ +│ 计算 OSS 文件 hash │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 4. 读取本地对应文件 │ +│ 4. 读取本地对应文件 │ └─────────────────────────────────────────────────────────────┘ │ ▼ diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index fad5e9899e63..bd092835cd46 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -293,50 +293,9 @@ New configuration parameter to control validation behavior: └─────────────┘ └─────────────────────┘ ``` -### FUSE Mount Point Detection Implementation +### Security Validation Implementation -#### Option 1: Java NIO FileStore API - -```java -import java.nio.file.FileStore; -import java.nio.file.Files; -import java.nio.file.Path; - -/** - * Detect if path is a FUSE mount point (cross-platform) - * Check filesystem type name - */ -private boolean isFUSEMountPoint(Path path) throws IOException { - FileStore store = Files.getFileStore(path); - - // Get filesystem type name - String type = store.type(); - String name = store.name(); - - // FUSE filesystem types typically contain "fuse" or specific identifiers - // Linux: fuse.sshfs, fuseblk, fuse - // macOS: macfuse, sshfs, osxfuse - // Generic: fuse, FUSE - return type != null && ( - type.toLowerCase().contains("fuse") || - type.equalsIgnoreCase("sshfs") || - type.equalsIgnoreCase("nfs4") || - name.toLowerCase().contains("fuse") - ); -} -``` - -**Platform Compatibility**: - -| Platform | FileStore.type() Examples | -|----------|---------------------------| -| Linux | `fuse.sshfs`, `fuseblk`, `fuse` | -| macOS | `macfuse`, `osxfuse`, `sshfs` | -| Windows | `NTFS`, `FAT32` (FUSE not natively supported, requires third-party tools) | - -#### Option 2: OSS Data Validation (Recommended) - -Use existing FileIO to read OSS files and compare with local files to validate path correctness. +Use OSS data validation to verify FUSE path correctness: read OSS files via existing FileIO and compare with local files. **Complete Implementation**: @@ -373,20 +332,15 @@ private FileIO fileIOForData(Path path, Identifier identifier) { /** * Validate FUSE local path - * Combining filesystem detection and OSS data validation */ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { // 1. Check if local path exists - if (!Files.exists(localPath)) { + java.nio.file.Path localNioPath = java.nio.file.Paths.get(localPath.toUri()); + if (!Files.exists(localNioPath)) { return ValidationResult.fail("Local path does not exist: " + localPath); } - // 2. Check if it's a FUSE mount point (Option 1) - if (!isFUSEMountPoint(localPath)) { - return ValidationResult.fail("Local path is not a FUSE mount point: " + localPath); - } - - // 3. OSS data validation (Option 2, recommended) + // 2. OSS data validation return validateByOSSData(localPath, ossPath, identifier); } @@ -399,44 +353,53 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif // 1. Get OSS FileIO (using existing logic, can access OSS) FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); - // 2. Select a file for validation (priority: snapshot > schema > manifest) - ChecksumFile checksumFile = findChecksumFile(ossPath, ossFileIO); - if (checksumFile == null) { - // Table may be empty (newly created), skip content validation - LOG.info("No checksum file found for table: {}, skip content validation", identifier); - return ValidationResult.success(); + // 2. Get latest snapshot via SnapshotManager + SnapshotManager snapshotManager = new SnapshotManager(ossFileIO, ossPath); + Snapshot latestSnapshot = snapshotManager.latestSnapshot(); + + Path checksumFile; + if (latestSnapshot != null) { + // Has snapshot, use snapshot file for validation + checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); + } else { + // No snapshot (new table), use schema file for validation + SchemaManager schemaManager = new SchemaManager(ossFileIO, ossPath); + Optional latestSchema = schemaManager.latest(); + if (!latestSchema.isPresent()) { + // Table is completely empty (no schema, shouldn't happen theoretically) + LOG.info("No snapshot or schema found for table: {}, skip validation", identifier); + return ValidationResult.success(); + } + checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); } - // 3. Read OSS file content (only read first N bytes for hash) - FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile.getFullPath()); - String ossHash = computeFileHash(ossFileIO, checksumFile.getFullPath(), HASH_CHECK_LENGTH); + // 3. Read OSS file content and compute hash + FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); + String ossHash = computeFileHash(ossFileIO, checksumFile); - // 4. Build local file path and read - Path localChecksumFile = new Path(localPath, checksumFile.getRelativePath()); + // 4. Read local file and compute hash + Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); if (!Files.exists(localNioPath)) { return ValidationResult.fail( "Local file not found: " + localChecksumFile + - ". The FUSE path may not be mounted correctly or points to wrong location."); + ". The FUSE path may not be mounted correctly."); } - // 5. Read local file content long localSize = Files.size(localNioPath); - String localHash = computeLocalFileHash(localNioPath, HASH_CHECK_LENGTH); + String localHash = computeLocalFileHash(localNioPath); - // 6. Compare file features + // 5. Compare file features if (localSize != ossStatus.getLen()) { return ValidationResult.fail(String.format( - "File size mismatch! Local: %d bytes, OSS: %d bytes. " + - "The local path may point to a different table.", + "File size mismatch! Local: %d bytes, OSS: %d bytes.", localSize, ossStatus.getLen())); } if (!localHash.equalsIgnoreCase(ossHash)) { return ValidationResult.fail(String.format( - "File content hash mismatch! Local: %s, OSS: %s. " + - "The local path points to a different table.", + "File content hash mismatch! Local: %s, OSS: %s.", localHash, ossHash)); } @@ -449,66 +412,45 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif } /** - * Find a file suitable for validation + * Compute FileIO file content hash */ -private ChecksumFile findChecksumFile(Path tablePath, FileIO fileIO) { - // Priority 1: snapshot file - Path snapshotDir = new Path(tablePath, "snapshot"); - if (fileIO.exists(snapshotDir)) { - FileStatus[] snapshots = fileIO.listStatus(snapshotDir); - if (snapshots != null && snapshots.length > 0) { - // Return latest snapshot file (sorted by filename) - Arrays.sort(snapshots, (a, b) -> b.getPath().getName().compareTo(a.getPath().getName())); - return new ChecksumFile(tablePath, snapshots[0].getPath()); - } - } - - // Priority 2: schema file - Path schemaFile = new Path(tablePath, "schema/schema-0"); - if (fileIO.exists(schemaFile)) { - return new ChecksumFile(tablePath, schemaFile); +private String computeFileHash(FileIO fileIO, Path file) throws IOException { + MessageDigest md; + try { + md = MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + throw new IOException("MD5 algorithm not available", e); } - // Priority 3: manifest file - Path manifestDir = new Path(tablePath, "manifest"); - if (fileIO.exists(manifestDir)) { - FileStatus[] manifests = fileIO.listStatus(manifestDir); - if (manifests != null && manifests.length > 0) { - return new ChecksumFile(tablePath, manifests[0].getPath()); + try (InputStream is = fileIO.newInputStream(file)) { + byte[] buffer = new byte[4096]; + int bytesRead; + while ((bytesRead = is.read(buffer)) != -1) { + md.update(buffer, 0, bytesRead); } } - - return null; + return Hex.encodeHexString(md.digest()); } /** - * Compute OSS file content hash (only read first N bytes) + * Compute local file content hash */ -private String computeFileHash(FileIO fileIO, Path file, int length) throws IOException { - try (InputStream is = fileIO.newInputStream(file); - DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { - byte[] buffer = new byte[length]; - dis.read(buffer); - byte[] hash = dis.getMessageDigest().digest(); - return Hex.encodeHexString(hash); +private String computeLocalFileHash(java.nio.file.Path file) throws IOException { + MessageDigest md; + try { + md = MessageDigest.getInstance("MD5"); } catch (NoSuchAlgorithmException e) { throw new IOException("MD5 algorithm not available", e); } -} -/** - * Compute local file content hash (only read first N bytes) - */ -private String computeLocalFileHash(java.nio.file.Path file, int length) throws IOException { - try (InputStream is = Files.newInputStream(file); - DigestInputStream dis = new DigestInputStream(is, MessageDigest.getInstance("MD5"))) { - byte[] buffer = new byte[length]; - dis.read(buffer); - byte[] hash = dis.getMessageDigest().digest(); - return Hex.encodeHexString(hash); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); + try (InputStream is = Files.newInputStream(file)) { + byte[] buffer = new byte[4096]; + int bytesRead; + while ((bytesRead = is.read(buffer)) != -1) { + md.update(buffer, 0, bytesRead); + } } + return Hex.encodeHexString(md.digest()); } /** @@ -548,8 +490,6 @@ private FileIO createDefaultFileIO(Path path, Identifier identifier) { : fileIOFromOptions(path); } -private static final int HASH_CHECK_LENGTH = 4096; // Check first 4KB - // ========== Helper Classes ========== enum ValidationMode { @@ -578,34 +518,26 @@ class ValidationResult { boolean isValid() { return valid; } String getErrorMessage() { return errorMessage; } } - -class ChecksumFile { - private final Path tablePath; - private final Path fullPath; - - ChecksumFile(Path tablePath, Path fullPath) { - this.tablePath = tablePath; - this.fullPath = fullPath; - } - - Path getFullPath() { return fullPath; } - - String getRelativePath() { - return new Path(tablePath, fullPath.getName()).toString(); - } -} ``` **Advantages**: | Advantage | Description | |-----------|-------------| -| **No API Extension Needed** | Uses existing FileIO to read OSS files | -| **Most Accurate** | Directly validates data consistency, 100% ensures path correctness | -| **Dual Protection** | FUSE mount detection + OSS data comparison | -| **Prevent Data Pollution** | Can detect when path points to wrong table | +| **No API Extension Needed** | Uses existing FileIO and SnapshotManager/SchemaManager | +| **Uses LATEST snapshot** | Gets via `SnapshotManager.latestSnapshot()`, no traversal needed | +| **New Table Support** | Falls back to schema file validation when no snapshot | +| **Most Accurate** | Directly validates data consistency, ensures path correctness | | **Graceful Degradation** | Validation failure falls back to default FileIO | +**Validation File Selection Logic**: + +| Scenario | Validation File | +|----------|-----------------| +| Has snapshot | Latest snapshot file via `SnapshotManager.latestSnapshot()` | +| No snapshot (new table) | Latest schema file via `SchemaManager.latest()` | +| No schema (theoretically impossible) | Skip validation | + **Complete Validation Flow**: ``` @@ -615,17 +547,39 @@ class ChecksumFile { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 1. Get table's OSS path info via REST API │ +│ 1. Get OSS FileIO (RESTTokenFileIO or ResolvingFileIO) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 2. Select validation file (snapshot/manifest/schema) │ +│ 2. Get latest snapshot via SnapshotManager │ +│ snapshotManager.latestSnapshot() │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ Snapshot exists ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ Use snapshot file │ │ Get latest schema via SchemaManager │ +│ for validation │ │ schemaManager.latest() │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Schema exists ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → Skip validation (empty table) + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 3. Get OSS file metadata (size, mtime, hash) │ +│ 3. Get OSS file metadata (size) │ +│ Compute OSS file hash │ └─────────────────────────────────────────────────────────────┘ │ ▼ From f86b2beb4d69405ae2b680f9d69ce802faaac499 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 14:46:55 +0800 Subject: [PATCH 03/23] [design] Use LocalFileIO instead of Java NIO Files API for FUSE validation - Replace java.nio.file.Paths.get() and Files.* with LocalFileIO - Use unified computeFileHash(FileIO, Path) method for both OSS and local files - More consistent with Paimon coding style - Removes dependency on java.nio.file package Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 65 ++++++++++------------------ designs/fuse-local-path-design.md | 47 ++++++-------------- 2 files changed, 37 insertions(+), 75 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index efbc36928b6a..ff3a7944efff 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -332,21 +332,24 @@ private FileIO fileIOForData(Path path, Identifier identifier) { * 校验 FUSE 本地路径 */ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { - // 1. 检查本地路径是否存在 - java.nio.file.Path localNioPath = java.nio.file.Paths.get(localPath.toUri()); - if (!Files.exists(localNioPath)) { - return ValidationResult.fail("Local path does not exist: " + localPath); + // 1. 创建 LocalFileIO 用于本地路径操作 + LocalFileIO localFileIO = LocalFileIO.create(); + + // 2. 检查本地路径是否存在 + if (!localFileIO.exists(localPath)) { + return ValidationResult.fail("本地路径不存在: " + localPath); } - // 2. OSS 数据校验 - return validateByOSSData(localPath, ossPath, identifier); + // 3. OSS 数据校验 + return validateByOSSData(localFileIO, localPath, ossPath, identifier); } /** * 通过比对 OSS 和本地文件验证 FUSE 路径正确性 * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取 OSS 文件 */ -private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateByOSSData( + LocalFileIO localFileIO, Path localPath, Path ossPath, Identifier identifier) { try { // 1. 获取 OSS FileIO（使用现有逻辑，可访问 OSS） FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); @@ -365,7 +368,7 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { // 表完全为空（连 schema 都没有，理论上不应该发生） - LOG.info("No snapshot or schema found for table: {}, skip validation", identifier); + LOG.info("未找到表 {} 的 snapshot 或 schema，跳过验证", identifier); return ValidationResult.success(); } checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); @@ -375,49 +378,48 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); String ossHash = computeFileHash(ossFileIO, checksumFile); - // 4. 读取本地文件并计算 hash + // 4. 构建本地文件路径并计算 hash Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); - java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); - if (!Files.exists(localNioPath)) { + if (!localFileIO.exists(localChecksumFile)) { return ValidationResult.fail( - "Local file not found: " + localChecksumFile + - ". The FUSE path may not be mounted correctly."); + "本地文件未找到: " + localChecksumFile + + "。FUSE 路径可能未正确挂载。"); } - long localSize = Files.size(localNioPath); - String localHash = computeLocalFileHash(localNioPath); + long localSize = localFileIO.getFileSize(localChecksumFile); + String localHash = computeFileHash(localFileIO, localChecksumFile); // 5. 比对文件特征 if (localSize != ossStatus.getLen()) { return ValidationResult.fail(String.format( - "File size mismatch! Local: %d bytes, OSS: %d bytes.", + "文件大小不匹配！本地: %d 字节, OSS: %d 字节。", localSize, ossStatus.getLen())); } if (!localHash.equalsIgnoreCase(ossHash)) { return ValidationResult.fail(String.format( - "File content hash mismatch! Local: %s, OSS: %s.", + "文件内容哈希不匹配！本地: %s, OSS: %s。", localHash, ossHash)); } return ValidationResult.success(); } catch (Exception e) { - LOG.warn("Failed to validate FUSE path by OSS data for: {}", identifier, e); - return ValidationResult.fail("OSS data validation failed: " + e.getMessage()); + LOG.warn("通过 OSS 数据验证 FUSE 路径失败: {}", identifier, e); + return ValidationResult.fail("OSS 数据验证失败: " + e.getMessage()); } } /** - * 计算 FileIO 文件内容哈希 + * 使用 FileIO 计算文件内容哈希 */ private String computeFileHash(FileIO fileIO, Path file) throws IOException { MessageDigest md; try { md = MessageDigest.getInstance("MD5"); } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); + throw new IOException("MD5 算法不可用", e); } try (InputStream is = fileIO.newInputStream(file)) { @@ -430,27 +432,6 @@ private String computeFileHash(FileIO fileIO, Path file) throws IOException { return Hex.encodeHexString(md.digest()); } -/** - * 计算本地文件内容哈希 - */ -private String computeLocalFileHash(java.nio.file.Path file) throws IOException { - MessageDigest md; - try { - md = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); - } - - try (InputStream is = Files.newInputStream(file)) { - byte[] buffer = new byte[4096]; - int bytesRead; - while ((bytesRead = is.read(buffer)) != -1) { - md.update(buffer, 0, bytesRead); - } - } - return Hex.encodeHexString(md.digest()); -} - /** * 处理校验错误 */ diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index bd092835cd46..f91d2b62b4b8 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -334,21 +334,24 @@ private FileIO fileIOForData(Path path, Identifier identifier) { * Validate FUSE local path */ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { - // 1. Check if local path exists - java.nio.file.Path localNioPath = java.nio.file.Paths.get(localPath.toUri()); - if (!Files.exists(localNioPath)) { + // 1. Create LocalFileIO for local path + LocalFileIO localFileIO = LocalFileIO.create(); + + // 2. Check if local path exists + if (!localFileIO.exists(localPath)) { return ValidationResult.fail("Local path does not exist: " + localPath); } - // 2. OSS data validation - return validateByOSSData(localPath, ossPath, identifier); + // 3. OSS data validation + return validateByOSSData(localFileIO, localPath, ossPath, identifier); } /** * Validate FUSE path correctness by comparing OSS and local file data * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read OSS files */ -private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateByOSSData( + LocalFileIO localFileIO, Path localPath, Path ossPath, Identifier identifier) { try { // 1. Get OSS FileIO (using existing logic, can access OSS) FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); @@ -377,18 +380,17 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); String ossHash = computeFileHash(ossFileIO, checksumFile); - // 4. Read local file and compute hash + // 4. Build local file path and compute hash Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); - java.nio.file.Path localNioPath = java.nio.file.Paths.get(localChecksumFile.toUri()); - if (!Files.exists(localNioPath)) { + if (!localFileIO.exists(localChecksumFile)) { return ValidationResult.fail( "Local file not found: " + localChecksumFile + ". The FUSE path may not be mounted correctly."); } - long localSize = Files.size(localNioPath); - String localHash = computeLocalFileHash(localNioPath); + long localSize = localFileIO.getFileSize(localChecksumFile); + String localHash = computeFileHash(localFileIO, localChecksumFile); // 5. Compare file features if (localSize != ossStatus.getLen()) { @@ -412,7 +414,7 @@ private ValidationResult validateByOSSData(Path localPath, Path ossPath, Identif } /** - * Compute FileIO file content hash + * Compute file content hash using FileIO */ private String computeFileHash(FileIO fileIO, Path file) throws IOException { MessageDigest md; @@ -432,27 +434,6 @@ private String computeFileHash(FileIO fileIO, Path file) throws IOException { return Hex.encodeHexString(md.digest()); } -/** - * Compute local file content hash - */ -private String computeLocalFileHash(java.nio.file.Path file) throws IOException { - MessageDigest md; - try { - md = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); - } - - try (InputStream is = Files.newInputStream(file)) { - byte[] buffer = new byte[4096]; - int bytesRead; - while ((bytesRead = is.read(buffer)) != -1) { - md.update(buffer, 0, bytesRead); - } - } - return Hex.encodeHexString(md.digest()); -} - /** * Handle validation error */ From 89323f9537f5fcd40bad7f8b4936409d4f019dc6 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 14:58:49 +0800 Subject: [PATCH 04/23] [design] Replace OSS with 'remote storage' terminology Use storage-agnostic terminology throughout the design documents: - Replace 'OSS' with 'remote storage' or 'remote' in variable names and comments - Use 'remoteFileIO', 'remotePath', 'remoteHash' instead of 'ossFileIO', 'ossPath', 'ossHash' - Update method names: validateByOSSData -> validateByRemoteData - Update flowchart labels: 'OSS Data Validation' -> 'Remote Data Validation' - Keep OSS/S3/HDFS as examples of remote storage types Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 96 +++++++++++++-------------- designs/fuse-local-path-design.md | 98 ++++++++++++++-------------- 2 files changed, 97 insertions(+), 97 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index ff3a7944efff..b9bc356191a8 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -20,13 +20,13 @@ limitations under the License. ## 背景 -在使用 Paimon RESTCatalog 访问 OSS（对象存储服务）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在 OSS 路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，无需 OSS 令牌。 +在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，无需认证令牌。 -本设计引入配置参数以支持 FUSE 挂载的 OSS 路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 +本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 ## 目标 -1. 支持通过本地文件系统访问 FUSE 挂载的 OSS 路径 +1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 2. 支持分层路径映射：Catalog 根路径 > Database > Table 3. 当 FUSE 本地路径适用时，跳过 `getTableToken` API 调用 4. 保持与现有 RESTCatalog 行为的向后兼容性 @@ -38,7 +38,7 @@ limitations under the License. | 参数 | 类型 | 默认值 | 描述 | |-----|------|--------|------| | `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | -| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/oss` | +| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | | `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | | `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | @@ -52,10 +52,10 @@ CREATE CATALOG paimon_rest_catalog WITH ( 'metastore' = 'rest', 'uri' = 'http://rest-server:8080', 'token' = 'xxx', - + -- FUSE 本地路径配置 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.root' = '/mnt/fuse/warehouse', 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' ); @@ -72,7 +72,7 @@ CREATE CATALOG paimon_rest_catalog WITH ( 示例：对于表 `db1.table1`： - 如果 `fuse.local-path.table` 包含 `db1.table1:/mnt/special/t1`，使用 `/mnt/special/t1` - 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` -- 否则，使用 `fuse.local-path.root`（如 `/mnt/oss/warehouse`） +- 否则，使用 `fuse.local-path.root`（如 `/mnt/fuse/warehouse`） ## 实现方案 @@ -129,8 +129,8 @@ private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { } private Path convertToLocalPath(String originalPath, String localRoot) { - // 将 OSS 路径转换为本地 FUSE 路径 - // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/oss/warehouse/db1/table1 + // 将远端存储路径转换为本地 FUSE 路径 + // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 // 具体实现取决于路径结构 } ``` @@ -145,7 +145,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { ## 优势 -1. **性能提升**：本地文件系统访问通常比基于网络的 OSS 访问更快 +1. **性能提升**：本地文件系统访问通常比基于网络的远端存储访问更快 2. **降低成本**：无需调用 `getTableToken` API，减少 REST 服务器负载 3. **灵活性**：支持为不同的数据库/表配置不同的本地路径 4. **向后兼容**：默认禁用，现有行为不变 @@ -158,26 +158,26 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | 场景 | 描述 | 后果 | |-----|------|------| -| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到 OSS，导致数据丢失 | -| **OSS 路径错误** | 本地路径指向了其他库表的 OSS 路径 | 数据写入错误的表，导致数据污染 | +| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到远端存储，导致数据丢失 | +| **远端路径错误** | 本地路径指向了其他库表的远端存储路径 | 数据写入错误的表，导致数据污染 | ### 校验方案 #### 1. 路径一致性校验（强校验） -在首次访问表时，校验本地路径与 OSS 路径的一致性： +在首次访问表时，校验本地路径与远端存储路径的一致性： ```java /** - * 校验 FUSE 本地路径与 OSS 路径的一致性 + * 校验 FUSE 本地路径与远端存储路径的一致性 * @throws IllegalArgumentException 如果路径不一致 */ -private void validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { +private void validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { // 1. 检查本地路径是否存在且为 FUSE 挂载点 if (!isFUSEMountPoint(localPath)) { throw new IllegalArgumentException( String.format("FUSE local path '%s' is not a valid FUSE mount point. " + - "Data would be written to local disk instead of OSS!", localPath)); + "Data would be written to local disk instead of remote storage!", localPath)); } // 2. 校验路径标识一致性：通过读取本地路径下的 .paimon 表标识文件 @@ -212,7 +212,7 @@ private boolean isFUSEMountPoint(Path path) { 在创建表时，自动在表目录下生成 `.paimon-identifier` 文件： ``` -/mnt/oss/warehouse/db1/table1/ +/mnt/fuse/warehouse/db1/table1/ ├── .paimon-identifier # 内容: "db1.table1" ├── data-xxx.parquet ├── manifest-xxx @@ -293,14 +293,14 @@ created-at=2026-03-13T00:00:00Z ### 安全校验实现 -使用 OSS 数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取 OSS 文件，与本地文件比对。 +使用远端数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取远端存储文件，与本地文件比对。 **完整实现**： ```java /** * RESTCatalog 中 fileIOForData 的完整实现 - * 结合 FUSE 本地路径校验与 OSS 数据校验 + * 结合 FUSE 本地路径校验与远端数据校验 */ private FileIO fileIOForData(Path path, Identifier identifier) { // 如果 FUSE 本地路径启用，尝试使用本地路径 @@ -331,7 +331,7 @@ private FileIO fileIOForData(Path path, Identifier identifier) { /** * 校验 FUSE 本地路径 */ -private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { // 1. 创建 LocalFileIO 用于本地路径操作 LocalFileIO localFileIO = LocalFileIO.create(); @@ -340,22 +340,22 @@ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifi return ValidationResult.fail("本地路径不存在: " + localPath); } - // 3. OSS 数据校验 - return validateByOSSData(localFileIO, localPath, ossPath, identifier); + // 3. 远端数据校验 + return validateByRemoteData(localFileIO, localPath, remotePath, identifier); } /** - * 通过比对 OSS 和本地文件验证 FUSE 路径正确性 - * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取 OSS 文件 + * 通过比对远端存储和本地文件验证 FUSE 路径正确性 + * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取远端存储文件 */ -private ValidationResult validateByOSSData( - LocalFileIO localFileIO, Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateByRemoteData( + LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { try { - // 1. 获取 OSS FileIO（使用现有逻辑，可访问 OSS） - FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); + // 1. 获取远端存储 FileIO（使用现有逻辑，可访问远端存储） + FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); // 2. 使用 SnapshotManager 获取最新 snapshot - SnapshotManager snapshotManager = new SnapshotManager(ossFileIO, ossPath); + SnapshotManager snapshotManager = new SnapshotManager(remoteFileIO, remotePath); Snapshot latestSnapshot = snapshotManager.latestSnapshot(); Path checksumFile; @@ -364,7 +364,7 @@ private ValidationResult validateByOSSData( checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); } else { // 无 snapshot（新表），使用 schema 文件校验 - SchemaManager schemaManager = new SchemaManager(ossFileIO, ossPath); + SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { // 表完全为空（连 schema 都没有，理论上不应该发生） @@ -374,12 +374,12 @@ private ValidationResult validateByOSSData( checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); } - // 3. 读取 OSS 文件内容并计算 hash - FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); - String ossHash = computeFileHash(ossFileIO, checksumFile); + // 3. 读取远端文件内容并计算 hash + FileStatus remoteStatus = remoteFileIO.getFileStatus(checksumFile); + String remoteHash = computeFileHash(remoteFileIO, checksumFile); // 4. 构建本地文件路径并计算 hash - Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); + Path localChecksumFile = new Path(localPath, remotePath.toUri().getPath()); if (!localFileIO.exists(localChecksumFile)) { return ValidationResult.fail( @@ -391,23 +391,23 @@ private ValidationResult validateByOSSData( String localHash = computeFileHash(localFileIO, localChecksumFile); // 5. 比对文件特征 - if (localSize != ossStatus.getLen()) { + if (localSize != remoteStatus.getLen()) { return ValidationResult.fail(String.format( - "文件大小不匹配！本地: %d 字节, OSS: %d 字节。", - localSize, ossStatus.getLen())); + "文件大小不匹配！本地: %d 字节, 远端: %d 字节。", + localSize, remoteStatus.getLen())); } - if (!localHash.equalsIgnoreCase(ossHash)) { + if (!localHash.equalsIgnoreCase(remoteHash)) { return ValidationResult.fail(String.format( - "文件内容哈希不匹配！本地: %s, OSS: %s。", - localHash, ossHash)); + "文件内容哈希不匹配！本地: %s, 远端: %s。", + localHash, remoteHash)); } return ValidationResult.success(); } catch (Exception e) { - LOG.warn("通过 OSS 数据验证 FUSE 路径失败: {}", identifier, e); - return ValidationResult.fail("OSS 数据验证失败: " + e.getMessage()); + LOG.warn("通过远端数据验证 FUSE 路径失败: {}", identifier, e); + return ValidationResult.fail("远端数据验证失败: " + e.getMessage()); } } @@ -521,12 +521,12 @@ class ValidationResult { ``` ┌─────────────────────────────────────────────────────────────┐ -│ OSS 数据校验流程 │ +│ 远端数据校验流程 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 1. 获取 OSS FileIO（RESTTokenFileIO 或 ResolvingFileIO） │ +│ 1. 获取远端存储 FileIO（RESTTokenFileIO 或 ResolvingFileIO）│ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -557,8 +557,8 @@ class ValidationResult { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 3. 获取 OSS 文件元数据（大小） │ -│ 计算 OSS 文件 hash │ +│ 3. 获取远端文件元数据（大小） │ +│ 计算远端文件 hash │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -605,7 +605,7 @@ CREATE CATALOG paimon_rest_catalog WITH ( -- FUSE 本地路径配置 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.root' = '/mnt/fuse/warehouse', -- 安全校验配置（可选，默认 strict） 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none @@ -615,6 +615,6 @@ CREATE CATALOG paimon_rest_catalog WITH ( ## 限制 1. FUSE 挂载必须正确配置且可访问 -2. 本地路径必须与 OSS 路径具有相同的目录结构 +2. 本地路径必须与远端存储路径具有相同的目录结构 3. 写操作需要本地 FUSE 挂载点具有适当的权限 4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index f91d2b62b4b8..e82ebae25d70 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -20,13 +20,13 @@ limitations under the License. ## Background -When using Paimon RESTCatalog with OSS (Object Storage Service), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where OSS paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths without needing OSS tokens. +When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths without needing authentication tokens. -This design introduces configuration parameters to support FUSE-mounted OSS paths, allowing users to specify local path mappings at catalog, database, and table levels. +This design introduces configuration parameters to support FUSE-mounted remote storage paths, allowing users to specify local path mappings at catalog, database, and table levels. ## Goals -1. Enable local file system access for FUSE-mounted OSS paths +1. Enable local file system access for FUSE-mounted remote storage paths 2. Support hierarchical path mapping: catalog root > database > table 3. Skip `getTableToken` API calls when FUSE local path is applicable 4. Maintain backward compatibility with existing RESTCatalog behavior @@ -37,8 +37,8 @@ All parameters are defined in `RESTCatalogOptions.java`: | Parameter | Type | Default | Description | |-----------|------|---------|-------------| -| `fuse.local-path.enabled` | Boolean | `false` | Whether to enable FUSE local path mapping for OSS paths | -| `fuse.local-path.root` | String | (none) | The root local path for FUSE-mounted OSS, e.g., `/mnt/oss` | +| `fuse.local-path.enabled` | Boolean | `false` | Whether to enable FUSE local path mapping for remote storage paths | +| `fuse.local-path.root` | String | (none) | The root local path for FUSE-mounted storage, e.g., `/mnt/fuse` | | `fuse.local-path.database` | Map | `{}` | Database-level local path mapping. Format: `db1:/local/path1,db2:/local/path2` | | `fuse.local-path.table` | Map | `{}` | Table-level local path mapping. Format: `db1.table1:/local/path1,db2.table2:/local/path2` | @@ -52,10 +52,10 @@ CREATE CATALOG paimon_rest_catalog WITH ( 'metastore' = 'rest', 'uri' = 'http://rest-server:8080', 'token' = 'xxx', - + -- FUSE local path configuration 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.root' = '/mnt/fuse/warehouse', 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' ); @@ -72,7 +72,7 @@ When resolving a path, the system checks in the following order (higher priority Example: For table `db1.table1`: - If `fuse.local-path.table` contains `db1.table1:/mnt/special/t1`, use `/mnt/special/t1` - Else if `fuse.local-path.database` contains `db1:/mnt/custom/db1`, use `/mnt/custom/db1` -- Else use `fuse.local-path.root` (e.g., `/mnt/oss/warehouse`) +- Else use `fuse.local-path.root` (e.g., `/mnt/fuse/warehouse`) ## Implementation @@ -129,8 +129,8 @@ private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { } private Path convertToLocalPath(String originalPath, String localRoot) { - // Convert OSS path to local FUSE path - // Example: oss://bucket/warehouse/db1/table1 -> /mnt/oss/warehouse/db1/table1 + // Convert remote storage path to local FUSE path + // Example: oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 // Implementation depends on path structure } ``` @@ -145,7 +145,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { ## Benefits -1. **Performance**: Local file system access is typically faster than network-based OSS access +1. **Performance**: Local file system access is typically faster than network-based remote storage access 2. **Cost Reduction**: No need to call `getTableToken` API, reducing REST server load 3. **Flexibility**: Supports different local paths for different databases/tables 4. **Backward Compatibility**: Disabled by default, existing behavior unchanged @@ -158,26 +158,26 @@ Incorrect FUSE local path configuration can lead to serious data consistency iss | Scenario | Description | Consequence | |----------|-------------|-------------| -| **Local path not mounted** | User's configured `/local/table` is not actually FUSE-mounted | Data is written only to local disk, not synced to OSS, causing data loss | -| **OSS path mismatch** | Local path points to a different table's OSS path | Data is written to the wrong table, causing data pollution | +| **Local path not mounted** | User's configured `/local/table` is not actually FUSE-mounted | Data is written only to local disk, not synced to remote storage, causing data loss | +| **Remote path mismatch** | Local path points to a different table's remote storage path | Data is written to the wrong table, causing data pollution | ### Validation Scheme #### 1. Path Consistency Validation (Strong Validation) -Validate consistency between local path and OSS path when first accessing a table: +Validate consistency between local path and remote storage path when first accessing a table: ```java /** - * Validate consistency between FUSE local path and OSS path + * Validate consistency between FUSE local path and remote storage path * @throws IllegalArgumentException if paths are inconsistent */ -private void validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { +private void validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { // 1. Check if local path exists and is a FUSE mount point if (!isFUSEMountPoint(localPath)) { throw new IllegalArgumentException( String.format("FUSE local path '%s' is not a valid FUSE mount point. " + - "Data would be written to local disk instead of OSS!", localPath)); + "Data would be written to local disk instead of remote storage!", localPath)); } // 2. Validate path identifier consistency: read .paimon table identifier file @@ -212,7 +212,7 @@ private boolean isFUSEMountPoint(Path path) { When creating a table, automatically generate a `.paimon-identifier` file in the table directory: ``` -/mnt/oss/warehouse/db1/table1/ +/mnt/fuse/warehouse/db1/table1/ ├── .paimon-identifier # Content: "db1.table1" ├── data-xxx.parquet ├── manifest-xxx @@ -295,14 +295,14 @@ New configuration parameter to control validation behavior: ### Security Validation Implementation -Use OSS data validation to verify FUSE path correctness: read OSS files via existing FileIO and compare with local files. +Use remote data validation to verify FUSE path correctness: read remote storage files via existing FileIO and compare with local files. **Complete Implementation**: ```java /** * Complete implementation of fileIOForData in RESTCatalog - * Combining FUSE local path validation with OSS data validation + * Combining FUSE local path validation with remote data validation */ private FileIO fileIOForData(Path path, Identifier identifier) { // If FUSE local path is enabled, try using local path @@ -333,7 +333,7 @@ private FileIO fileIOForData(Path path, Identifier identifier) { /** * Validate FUSE local path */ -private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { // 1. Create LocalFileIO for local path LocalFileIO localFileIO = LocalFileIO.create(); @@ -342,22 +342,22 @@ private ValidationResult validateFUSEPath(Path localPath, Path ossPath, Identifi return ValidationResult.fail("Local path does not exist: " + localPath); } - // 3. OSS data validation - return validateByOSSData(localFileIO, localPath, ossPath, identifier); + // 3. Remote data validation + return validateByRemoteData(localFileIO, localPath, remotePath, identifier); } /** - * Validate FUSE path correctness by comparing OSS and local file data - * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read OSS files + * Validate FUSE path correctness by comparing remote and local file data + * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read remote files */ -private ValidationResult validateByOSSData( - LocalFileIO localFileIO, Path localPath, Path ossPath, Identifier identifier) { +private ValidationResult validateByRemoteData( + LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { try { - // 1. Get OSS FileIO (using existing logic, can access OSS) - FileIO ossFileIO = createDefaultFileIO(ossPath, identifier); + // 1. Get remote FileIO (using existing logic, can access remote storage) + FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); // 2. Get latest snapshot via SnapshotManager - SnapshotManager snapshotManager = new SnapshotManager(ossFileIO, ossPath); + SnapshotManager snapshotManager = new SnapshotManager(remoteFileIO, remotePath); Snapshot latestSnapshot = snapshotManager.latestSnapshot(); Path checksumFile; @@ -366,7 +366,7 @@ private ValidationResult validateByOSSData( checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); } else { // No snapshot (new table), use schema file for validation - SchemaManager schemaManager = new SchemaManager(ossFileIO, ossPath); + SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { // Table is completely empty (no schema, shouldn't happen theoretically) @@ -376,12 +376,12 @@ private ValidationResult validateByOSSData( checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); } - // 3. Read OSS file content and compute hash - FileStatus ossStatus = ossFileIO.getFileStatus(checksumFile); - String ossHash = computeFileHash(ossFileIO, checksumFile); + // 3. Read remote file content and compute hash + FileStatus remoteStatus = remoteFileIO.getFileStatus(checksumFile); + String remoteHash = computeFileHash(remoteFileIO, checksumFile); // 4. Build local file path and compute hash - Path localChecksumFile = new Path(localPath, ossPath.toUri().getPath()); + Path localChecksumFile = new Path(localPath, remotePath.toUri().getPath()); if (!localFileIO.exists(localChecksumFile)) { return ValidationResult.fail( @@ -393,23 +393,23 @@ private ValidationResult validateByOSSData( String localHash = computeFileHash(localFileIO, localChecksumFile); // 5. Compare file features - if (localSize != ossStatus.getLen()) { + if (localSize != remoteStatus.getLen()) { return ValidationResult.fail(String.format( - "File size mismatch! Local: %d bytes, OSS: %d bytes.", - localSize, ossStatus.getLen())); + "File size mismatch! Local: %d bytes, Remote: %d bytes.", + localSize, remoteStatus.getLen())); } - if (!localHash.equalsIgnoreCase(ossHash)) { + if (!localHash.equalsIgnoreCase(remoteHash)) { return ValidationResult.fail(String.format( - "File content hash mismatch! Local: %s, OSS: %s.", - localHash, ossHash)); + "File content hash mismatch! Local: %s, Remote: %s.", + localHash, remoteHash)); } return ValidationResult.success(); } catch (Exception e) { - LOG.warn("Failed to validate FUSE path by OSS data for: {}", identifier, e); - return ValidationResult.fail("OSS data validation failed: " + e.getMessage()); + LOG.warn("Failed to validate FUSE path by remote data for: {}", identifier, e); + return ValidationResult.fail("Remote data validation failed: " + e.getMessage()); } } @@ -523,12 +523,12 @@ class ValidationResult { ``` ┌─────────────────────────────────────────────────────────────┐ -│ OSS Data Validation Flow │ +│ Remote Data Validation Flow │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 1. Get OSS FileIO (RESTTokenFileIO or ResolvingFileIO) │ +│ 1. Get remote FileIO (RESTTokenFileIO or ResolvingFileIO) │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -559,8 +559,8 @@ class ValidationResult { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 3. Get OSS file metadata (size) │ -│ Compute OSS file hash │ +│ 3. Get remote file metadata (size) │ +│ Compute remote file hash │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -608,7 +608,7 @@ CREATE CATALOG paimon_rest_catalog WITH ( -- FUSE local path configuration 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/oss/warehouse', + 'fuse.local-path.root' = '/mnt/fuse/warehouse', -- Security validation configuration (optional, default: strict) 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none @@ -618,7 +618,7 @@ CREATE CATALOG paimon_rest_catalog WITH ( ## Limitations 1. FUSE mount must be properly configured and accessible -2. Local path must have the same directory structure as the OSS path +2. Local path must have the same directory structure as the remote storage path 3. Write operations require proper permissions on the local FUSE mount 4. Windows platform has limited FUSE support (requires third-party tools like WinFsp) From fa8af10d50c6b1317b4dc90128c5fc997412910c Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:01:22 +0800 Subject: [PATCH 05/23] [design] Clarify that getTableToken is still needed for validation - Update background: FUSE allows bypassing remote storage SDKs, not authentication - Update goal #3: Use local FileIO for data read/write, but still need getTableToken for validation - Update behavior matrix: clarify getTableToken is still used for validation - Remove 'Cost Reduction' from benefits since getTableToken is still called Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 11 +++++------ designs/fuse-local-path-design.md | 11 +++++------ 2 files changed, 10 insertions(+), 12 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index b9bc356191a8..1bc9fad60a17 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -20,7 +20,7 @@ limitations under the License. ## 背景 -在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，无需认证令牌。 +在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，绕过远端存储 SDK，获得更好的性能。 本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 @@ -28,7 +28,7 @@ limitations under the License. 1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 2. 支持分层路径映射：Catalog 根路径 > Database > Table -3. 当 FUSE 本地路径适用时，跳过 `getTableToken` API 调用 +3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写（仍需 `getTableToken` 用于校验） 4. 保持与现有 RESTCatalog 行为的向后兼容性 ## 配置参数 @@ -139,16 +139,15 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | 配置 | 路径匹配 | 行为 | |-----|---------|------| -| `fuse.local-path.enabled=true` | 是 | 本地 FileIO，**无需调用 `getTableToken`** | +| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写，`getTableToken` 仍用于校验 | | `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | | `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | ## 优势 1. **性能提升**：本地文件系统访问通常比基于网络的远端存储访问更快 -2. **降低成本**：无需调用 `getTableToken` API，减少 REST 服务器负载 -3. **灵活性**：支持为不同的数据库/表配置不同的本地路径 -4. **向后兼容**：默认禁用，现有行为不变 +2. **灵活性**：支持为不同的数据库/表配置不同的本地路径 +3. **向后兼容**：默认禁用，现有行为不变 ## 安全校验机制 diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index e82ebae25d70..223d65c0cd66 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -20,7 +20,7 @@ limitations under the License. ## Background -When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths without needing authentication tokens. +When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths directly, bypassing remote storage SDKs and achieving better performance. This design introduces configuration parameters to support FUSE-mounted remote storage paths, allowing users to specify local path mappings at catalog, database, and table levels. @@ -28,7 +28,7 @@ This design introduces configuration parameters to support FUSE-mounted remote s 1. Enable local file system access for FUSE-mounted remote storage paths 2. Support hierarchical path mapping: catalog root > database > table -3. Skip `getTableToken` API calls when FUSE local path is applicable +3. Use local FileIO for data read/write when FUSE local path is applicable (still need `getTableToken` for validation) 4. Maintain backward compatibility with existing RESTCatalog behavior ## Configuration Parameters @@ -139,16 +139,15 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | Configuration | Path Match | Behavior | |---------------|------------|----------| -| `fuse.local-path.enabled=true` | Yes | Local FileIO, **no `getTableToken` call** | +| `fuse.local-path.enabled=true` | Yes | Local FileIO for data read/write, `getTableToken` still used for validation | | `fuse.local-path.enabled=true` | No | Fallback to original logic | | `fuse.local-path.enabled=false` | N/A | Original logic (data token or ResolvingFileIO) | ## Benefits 1. **Performance**: Local file system access is typically faster than network-based remote storage access -2. **Cost Reduction**: No need to call `getTableToken` API, reducing REST server load -3. **Flexibility**: Supports different local paths for different databases/tables -4. **Backward Compatibility**: Disabled by default, existing behavior unchanged +2. **Flexibility**: Supports different local paths for different databases/tables +3. **Backward Compatibility**: Disabled by default, existing behavior unchanged ## Security Validation Mechanism From 8ab6366bc816d18378d67a2e7d953af42f5a6240 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:03:37 +0800 Subject: [PATCH 06/23] [design] Simplify background description, remove getTableToken references - Background: focus on SDK vs local filesystem access, not authentication - Goals: remove getTableToken validation mention - Behavior matrix: simplify description Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 6 +++--- designs/fuse-local-path-design.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 1bc9fad60a17..a59fc85b13ba 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -20,7 +20,7 @@ limitations under the License. ## 背景 -在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常需要通过 `getTableToken` API 从 REST 服务器获取认证令牌。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，绕过远端存储 SDK，获得更好的性能。 +在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 @@ -28,7 +28,7 @@ limitations under the License. 1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 2. 支持分层路径映射：Catalog 根路径 > Database > Table -3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写（仍需 `getTableToken` 用于校验） +3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写 4. 保持与现有 RESTCatalog 行为的向后兼容性 ## 配置参数 @@ -139,7 +139,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | 配置 | 路径匹配 | 行为 | |-----|---------|------| -| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写，`getTableToken` 仍用于校验 | +| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写 | | `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | | `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 223d65c0cd66..922e43f28810 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -20,7 +20,7 @@ limitations under the License. ## Background -When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically requires authentication tokens obtained from the REST server via `getTableToken` API. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths directly, bypassing remote storage SDKs and achieving better performance. +When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically goes through remote storage SDKs. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths directly, achieving better performance. This design introduces configuration parameters to support FUSE-mounted remote storage paths, allowing users to specify local path mappings at catalog, database, and table levels. @@ -28,7 +28,7 @@ This design introduces configuration parameters to support FUSE-mounted remote s 1. Enable local file system access for FUSE-mounted remote storage paths 2. Support hierarchical path mapping: catalog root > database > table -3. Use local FileIO for data read/write when FUSE local path is applicable (still need `getTableToken` for validation) +3. Use local FileIO for data read/write when FUSE local path is applicable 4. Maintain backward compatibility with existing RESTCatalog behavior ## Configuration Parameters @@ -139,7 +139,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | Configuration | Path Match | Behavior | |---------------|------------|----------| -| `fuse.local-path.enabled=true` | Yes | Local FileIO for data read/write, `getTableToken` still used for validation | +| `fuse.local-path.enabled=true` | Yes | Local FileIO for data read/write | | `fuse.local-path.enabled=true` | No | Fallback to original logic | | `fuse.local-path.enabled=false` | N/A | Original logic (data token or ResolvingFileIO) | From 846a2fd56d22b56727ea72f2733260423e590532 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:08:09 +0800 Subject: [PATCH 07/23] [design] Remove obsolete validation schemes - Remove 'Path Consistency Validation' (isFUSEMountPoint, etc.) - Remove 'Table Identifier File Mechanism' - Update validation flow diagrams to reflect remote data validation - Keep only the remote data validation approach Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 72 ++------------------------ designs/fuse-local-path-design.md | 76 ++-------------------------- 2 files changed, 8 insertions(+), 140 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index a59fc85b13ba..a2712f74d1b1 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -160,73 +160,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { | **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到远端存储，导致数据丢失 | | **远端路径错误** | 本地路径指向了其他库表的远端存储路径 | 数据写入错误的表，导致数据污染 | -### 校验方案 - -#### 1. 路径一致性校验（强校验） - -在首次访问表时，校验本地路径与远端存储路径的一致性： - -```java -/** - * 校验 FUSE 本地路径与远端存储路径的一致性 - * @throws IllegalArgumentException 如果路径不一致 - */ -private void validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { - // 1. 检查本地路径是否存在且为 FUSE 挂载点 - if (!isFUSEMountPoint(localPath)) { - throw new IllegalArgumentException( - String.format("FUSE local path '%s' is not a valid FUSE mount point. " + - "Data would be written to local disk instead of remote storage!", localPath)); - } - - // 2. 校验路径标识一致性：通过读取本地路径下的 .paimon 表标识文件 - Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); - if (fileIO.exists(localIdentifierFile)) { - String storedIdentifier = readIdentifier(localIdentifierFile); - String expectedIdentifier = identifier.getDatabaseName() + "." + identifier.getTableName(); - - if (!expectedIdentifier.equals(storedIdentifier)) { - throw new IllegalArgumentException( - String.format("FUSE path mismatch! Local path '%s' belongs to table '%s', " + - "but current table is '%s'.", - localPath, storedIdentifier, expectedIdentifier)); - } - } -} - -/** - * 检查路径是否为 FUSE 挂载点 - * 可通过检查 /proc/mounts (Linux) 或使用 stat 系统调用判断 - */ -private boolean isFUSEMountPoint(Path path) { - // 方案1: 检查 /proc/mounts 中是否包含该路径的 FUSE 挂载 - // 方案2: 检查路径的文件系统类型是否为 fuse.* - // 方案3: 通过读取 /etc/mtab 或使用 jnr-posix 库 - return checkFUSEMount(path); -} -``` - -#### 2. 表标识文件机制 - -在创建表时，自动在表目录下生成 `.paimon-identifier` 文件： - -``` -/mnt/fuse/warehouse/db1/table1/ -├── .paimon-identifier # 内容: "db1.table1" -├── data-xxx.parquet -├── manifest-xxx -└── snapshot-xxx -``` - -标识文件内容： -``` -database=db1 -table=table1 -table-uuid=xxx-xxx-xxx -created-at=2026-03-13T00:00:00Z -``` - -#### 3. 校验模式配置 +### 校验模式配置 新增配置参数控制校验行为： @@ -271,8 +205,8 @@ created-at=2026-03-13T00:00:00Z │ │ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ - │ 校验 FUSE 挂载点 │ │ 跳过校验 │ - │ 校验路径一致性 │ │ 直接使用本地路径 │ + │ 校验本地路径存在 │ │ 跳过校验 │ + │ 与远端数据比对 │ │ 直接使用本地路径 │ └───────────────────┘ └───────────────────┘ │ ▼ diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 922e43f28810..f87af3b6bc06 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -160,73 +160,7 @@ Incorrect FUSE local path configuration can lead to serious data consistency iss | **Local path not mounted** | User's configured `/local/table` is not actually FUSE-mounted | Data is written only to local disk, not synced to remote storage, causing data loss | | **Remote path mismatch** | Local path points to a different table's remote storage path | Data is written to the wrong table, causing data pollution | -### Validation Scheme - -#### 1. Path Consistency Validation (Strong Validation) - -Validate consistency between local path and remote storage path when first accessing a table: - -```java -/** - * Validate consistency between FUSE local path and remote storage path - * @throws IllegalArgumentException if paths are inconsistent - */ -private void validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { - // 1. Check if local path exists and is a FUSE mount point - if (!isFUSEMountPoint(localPath)) { - throw new IllegalArgumentException( - String.format("FUSE local path '%s' is not a valid FUSE mount point. " + - "Data would be written to local disk instead of remote storage!", localPath)); - } - - // 2. Validate path identifier consistency: read .paimon table identifier file - Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); - if (fileIO.exists(localIdentifierFile)) { - String storedIdentifier = readIdentifier(localIdentifierFile); - String expectedIdentifier = identifier.getDatabaseName() + "." + identifier.getTableName(); - - if (!expectedIdentifier.equals(storedIdentifier)) { - throw new IllegalArgumentException( - String.format("FUSE path mismatch! Local path '%s' belongs to table '%s', " + - "but current table is '%s'.", - localPath, storedIdentifier, expectedIdentifier)); - } - } -} - -/** - * Check if path is a FUSE mount point - * Can be determined by checking /proc/mounts (Linux) or using stat system call - */ -private boolean isFUSEMountPoint(Path path) { - // Option 1: Check /proc/mounts for FUSE mount of this path - // Option 2: Check if filesystem type is fuse.* - // Option 3: Read /etc/mtab or use jnr-posix library - return checkFUSEMount(path); -} -``` - -#### 2. Table Identifier File Mechanism - -When creating a table, automatically generate a `.paimon-identifier` file in the table directory: - -``` -/mnt/fuse/warehouse/db1/table1/ -├── .paimon-identifier # Content: "db1.table1" -├── data-xxx.parquet -├── manifest-xxx -└── snapshot-xxx -``` - -Identifier file content: -``` -database=db1 -table=table1 -table-uuid=xxx-xxx-xxx -created-at=2026-03-13T00:00:00Z -``` - -#### 3. Validation Mode Configuration +### Validation Mode Configuration New configuration parameter to control validation behavior: @@ -271,10 +205,10 @@ New configuration parameter to control validation behavior: │ │ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ - │ Validate FUSE │ │ Skip validation │ - │ mount point │ │ Use local path │ - │ Validate path │ │ directly │ - │ consistency │ │ │ + │ Validate local │ │ Skip validation │ + │ path exists │ │ Use local path │ + │ Compare with │ │ directly │ + │ remote data │ │ │ └───────────────────┘ └───────────────────┘ │ ▼ From 110d39653a2b4d99b28cadef3b88cc1726499ec2 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:11:03 +0800 Subject: [PATCH 08/23] [design] Fix comment about tables without schema Format tables and object tables have no schema, this is expected behavior Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 2 +- designs/fuse-local-path-design.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index a2712f74d1b1..fa27f616bbe6 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -300,7 +300,7 @@ private ValidationResult validateByRemoteData( SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { - // 表完全为空（连 schema 都没有，理论上不应该发生） + // 无 schema（如 format 表、object 表），跳过验证 LOG.info("未找到表 {} 的 snapshot 或 schema，跳过验证", identifier); return ValidationResult.success(); } diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index f87af3b6bc06..b39dee6077bd 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -302,7 +302,7 @@ private ValidationResult validateByRemoteData( SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { - // Table is completely empty (no schema, shouldn't happen theoretically) + // No schema (e.g., format table, object table), skip validation LOG.info("No snapshot or schema found for table: {}, skip validation", identifier); return ValidationResult.success(); } From 6ffefc58be34faebe48cdee5689b90c444ba1638 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:13:57 +0800 Subject: [PATCH 09/23] [design] Update validation table for tables without schema Format tables and object tables have no schema, this is expected behavior Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 2 +- designs/fuse-local-path-design.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index fa27f616bbe6..a70089d3592c 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -448,7 +448,7 @@ class ValidationResult { |------|----------| | 有 snapshot | 使用 `SnapshotManager.latestSnapshot()` 获取的最新 snapshot 文件 | | 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | -| 无 schema（理论上不存在）| 跳过校验 | +| 无 schema（如 format 表、object 表）| 跳过校验 | **完整校验流程**： diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index b39dee6077bd..5ebfa0fc2727 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -450,7 +450,7 @@ class ValidationResult { |----------|-----------------| | Has snapshot | Latest snapshot file via `SnapshotManager.latestSnapshot()` | | No snapshot (new table) | Latest schema file via `SchemaManager.latest()` | -| No schema (theoretically impossible) | Skip validation | +| No schema (e.g., format table, object table) | Skip validation | **Complete Validation Flow**: From 82c0aef6fe6dc182d190a0a0ca34389b314dafd8 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:19:02 +0800 Subject: [PATCH 10/23] [design] Add .paimon-identifier as first validation step - Add validateByIdentifierFile() method for UUID comparison - Add readIdentifierFile() helper for parsing identifier file - Update validation flow diagrams with two-step validation - Add .paimon-identifier file format documentation Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 131 ++++++++++++++++++++++++++- designs/fuse-local-path-design.md | 131 ++++++++++++++++++++++++++- 2 files changed, 254 insertions(+), 8 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index a70089d3592c..cb55ed6129cf 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -224,6 +224,25 @@ private Path convertToLocalPath(String originalPath, String localRoot) { └─────────────────────┘ ``` +### .paimon-identifier 文件 + +每个表目录下都包含一个 `.paimon-identifier` 文件用于快速校验： + +**文件位置**：`<表路径>/.paimon-identifier` + +**文件格式**： +``` +database=db1 +table=table1 +table-uuid=xxx-xxx-xxx-xxx +created-at=2026-03-13T00:00:00Z +``` + +**用途**： +- 比对本地和远端路径的表 UUID +- 在昂贵的文件内容比对前进行快速校验 +- 创建表时自动生成 + ### 安全校验实现 使用远端数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取远端存储文件，与本地文件比对。 @@ -273,12 +292,79 @@ private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Ident return ValidationResult.fail("本地路径不存在: " + localPath); } - // 3. 远端数据校验 + // 3. 第一次校验：表标识文件 + ValidationResult identifierResult = validateByIdentifierFile(localFileIO, localPath, remotePath, identifier); + if (!identifierResult.isSuccess()) { + return identifierResult; + } + + // 4. 第二次校验：远端数据校验 return validateByRemoteData(localFileIO, localPath, remotePath, identifier); } /** - * 通过比对远端存储和本地文件验证 FUSE 路径正确性 + * 第一次校验：检查 .paimon-identifier 文件 + * 比对本地和远端的表 UUID 确保路径正确性 + */ +private ValidationResult validateByIdentifierFile( + LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { + try { + // 1. 获取远端存储 FileIO + FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); + + // 2. 读取远端标识文件 + Path remoteIdentifierFile = new Path(remotePath, ".paimon-identifier"); + if (!remoteFileIO.exists(remoteIdentifierFile)) { + // 无标识文件，跳过此次校验 + LOG.debug("未找到表 {} 的 .paimon-identifier 文件，跳过标识校验", identifier); + return ValidationResult.success(); + } + + String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); + + // 3. 读取本地标识文件 + Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + if (!localFileIO.exists(localIdentifierFile)) { + return ValidationResult.fail( + "本地 .paimon-identifier 文件未找到: " + localIdentifierFile + + "。FUSE 路径可能未正确挂载。"); + } + + String localIdentifier = readIdentifierFile(localFileIO, localIdentifierFile); + + // 4. 比对标识符 + if (!remoteIdentifier.equals(localIdentifier)) { + return ValidationResult.fail(String.format( + "表标识不匹配！本地: %s，远端: %s。" + + "本地路径可能指向了其他表。", + localIdentifier, remoteIdentifier)); + } + + return ValidationResult.success(); + + } catch (Exception e) { + LOG.warn("标识文件校验失败: {}", identifier, e); + return ValidationResult.fail("标识文件校验失败: " + e.getMessage()); + } +} + +/** + * 读取 .paimon-identifier 文件内容 + * 格式：database=db1\ntable=table1\ntable-uuid=xxx-xxx-xxx + */ +private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { + try (InputStream in = fileIO.newInputStream(identifierFile)) { + Properties props = new Properties(); + props.load(in); + String database = props.getProperty("database"); + String table = props.getProperty("table"); + String uuid = props.getProperty("table-uuid"); + return database + "." + table + "@" + uuid; + } +} + +/** + * 第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性 * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取远端存储文件 */ private ValidationResult validateByRemoteData( @@ -450,11 +536,48 @@ class ValidationResult { | 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | | 无 schema（如 format 表、object 表）| 跳过校验 | +**两步校验**： + +| 步骤 | 校验方式 | 描述 | +|------|----------|------| +| 1 | `.paimon-identifier` 文件 | 比对本地和远端的表 UUID | +| 2 | 远端数据校验 | 比对 snapshot/schema 文件内容 | + **完整校验流程**： ``` ┌─────────────────────────────────────────────────────────────┐ -│ 远端数据校验流程 │ +│ 校验流程 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 第一步：.paimon-identifier 校验 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 远端存在 .paimon-identifier ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ 比对 UUID │ │ 跳过第一步，进入第二步 │ +│ 本地 vs 远端 │ │ （远端数据校验） │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ UUID 匹配 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → 失败：表标识不匹配 + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 第二步：远端数据校验 │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -486,7 +609,7 @@ class ValidationResult { │ Schema 存在 ? │ └───────────────────────────────────────┘ │ │ - Yes No → 跳过校验（空表） + Yes No → 跳过校验（format/object 表） │ ▼ ┌─────────────────────────────────────────────────────────────┐ diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 5ebfa0fc2727..b4bcb80af76e 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -226,6 +226,25 @@ New configuration parameter to control validation behavior: └─────────────┘ └─────────────────────┘ ``` +### .paimon-identifier File + +Each table directory contains a `.paimon-identifier` file for quick validation: + +**File Location**: `/.paimon-identifier` + +**File Format**: +``` +database=db1 +table=table1 +table-uuid=xxx-xxx-xxx-xxx +created-at=2026-03-13T00:00:00Z +``` + +**Usage**: +- Compare table UUID between local and remote paths +- Quick validation before expensive data comparison +- Automatically generated when table is created + ### Security Validation Implementation Use remote data validation to verify FUSE path correctness: read remote storage files via existing FileIO and compare with local files. @@ -275,12 +294,79 @@ private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Ident return ValidationResult.fail("Local path does not exist: " + localPath); } - // 3. Remote data validation + // 3. First validation: Table identifier file + ValidationResult identifierResult = validateByIdentifierFile(localFileIO, localPath, remotePath, identifier); + if (!identifierResult.isSuccess()) { + return identifierResult; + } + + // 4. Second validation: Remote data validation return validateByRemoteData(localFileIO, localPath, remotePath, identifier); } /** - * Validate FUSE path correctness by comparing remote and local file data + * First validation: Check .paimon-identifier file + * Compare table UUID between local and remote to ensure path correctness + */ +private ValidationResult validateByIdentifierFile( + LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { + try { + // 1. Get remote FileIO + FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); + + // 2. Read remote identifier file + Path remoteIdentifierFile = new Path(remotePath, ".paimon-identifier"); + if (!remoteFileIO.exists(remoteIdentifierFile)) { + // No identifier file, skip this validation + LOG.debug("No .paimon-identifier file found for table: {}, skip identifier validation", identifier); + return ValidationResult.success(); + } + + String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); + + // 3. Read local identifier file + Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + if (!localFileIO.exists(localIdentifierFile)) { + return ValidationResult.fail( + "Local .paimon-identifier file not found: " + localIdentifierFile + + ". The FUSE path may not be mounted correctly."); + } + + String localIdentifier = readIdentifierFile(localFileIO, localIdentifierFile); + + // 4. Compare identifiers + if (!remoteIdentifier.equals(localIdentifier)) { + return ValidationResult.fail(String.format( + "Table identifier mismatch! Local: %s, Remote: %s. " + + "The local path may point to a different table.", + localIdentifier, remoteIdentifier)); + } + + return ValidationResult.success(); + + } catch (Exception e) { + LOG.warn("Failed to validate by identifier file for: {}", identifier, e); + return ValidationResult.fail("Identifier validation failed: " + e.getMessage()); + } +} + +/** + * Read .paimon-identifier file content + * Format: database=db1\ntable=table1\ntable-uuid=xxx-xxx-xxx + */ +private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { + try (InputStream in = fileIO.newInputStream(identifierFile)) { + Properties props = new Properties(); + props.load(in); + String database = props.getProperty("database"); + String table = props.getProperty("table"); + String uuid = props.getProperty("table-uuid"); + return database + "." + table + "@" + uuid; + } +} + +/** + * Second validation: Validate FUSE path correctness by comparing remote and local file data * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read remote files */ private ValidationResult validateByRemoteData( @@ -452,11 +538,48 @@ class ValidationResult { | No snapshot (new table) | Latest schema file via `SchemaManager.latest()` | | No schema (e.g., format table, object table) | Skip validation | +**Two-Step Validation**: + +| Step | Validation | Description | +|------|------------|-------------| +| 1 | `.paimon-identifier` file | Compare table UUID between local and remote | +| 2 | Remote data validation | Compare snapshot/schema file content | + **Complete Validation Flow**: ``` ┌─────────────────────────────────────────────────────────────┐ -│ Remote Data Validation Flow │ +│ Validation Flow │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Step 1: .paimon-identifier Validation │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ .paimon-identifier exists remotely ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ Compare UUID │ │ Skip to Step 2 (remote data │ +│ local vs remote │ │ validation) │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ UUID matches ? │ + └───────────────────────────────────────┘ + │ │ + Yes No → FAIL: Table identifier mismatch + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Step 2: Remote Data Validation │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -488,7 +611,7 @@ class ValidationResult { │ Schema exists ? │ └───────────────────────────────────────┘ │ │ - Yes No → Skip validation (empty table) + Yes No → Skip validation (format/object table) │ ▼ ┌─────────────────────────────────────────────────────────────┐ From ba44dc07b5b88f671b472e3849db72724694c92c Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:21:41 +0800 Subject: [PATCH 11/23] [design] Simplify .paimon-identifier to only contain UUID - Remove database/table fields (they can change via rename) - Only keep table-uuid for identification - Update readIdentifierFile() to return UUID only Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 10 +++------- designs/fuse-local-path-design.md | 10 +++------- 2 files changed, 6 insertions(+), 14 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index cb55ed6129cf..5d9ec4d87474 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -232,8 +232,6 @@ private Path convertToLocalPath(String originalPath, String localRoot) { **文件格式**： ``` -database=db1 -table=table1 table-uuid=xxx-xxx-xxx-xxx created-at=2026-03-13T00:00:00Z ``` @@ -242,6 +240,7 @@ created-at=2026-03-13T00:00:00Z - 比对本地和远端路径的表 UUID - 在昂贵的文件内容比对前进行快速校验 - 创建表时自动生成 +- 仅需 UUID（database/table 名称可能因重命名而变化） ### 安全校验实现 @@ -350,16 +349,13 @@ private ValidationResult validateByIdentifierFile( /** * 读取 .paimon-identifier 文件内容 - * 格式：database=db1\ntable=table1\ntable-uuid=xxx-xxx-xxx + * 格式：table-uuid=xxx-xxx-xxx-xxx */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { Properties props = new Properties(); props.load(in); - String database = props.getProperty("database"); - String table = props.getProperty("table"); - String uuid = props.getProperty("table-uuid"); - return database + "." + table + "@" + uuid; + return props.getProperty("table-uuid"); } } diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index b4bcb80af76e..afdeaee5eda5 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -234,8 +234,6 @@ Each table directory contains a `.paimon-identifier` file for quick validation: **File Format**: ``` -database=db1 -table=table1 table-uuid=xxx-xxx-xxx-xxx created-at=2026-03-13T00:00:00Z ``` @@ -244,6 +242,7 @@ created-at=2026-03-13T00:00:00Z - Compare table UUID between local and remote paths - Quick validation before expensive data comparison - Automatically generated when table is created +- Only UUID is needed (database/table names can change via rename) ### Security Validation Implementation @@ -352,16 +351,13 @@ private ValidationResult validateByIdentifierFile( /** * Read .paimon-identifier file content - * Format: database=db1\ntable=table1\ntable-uuid=xxx-xxx-xxx + * Format: table-uuid=xxx-xxx-xxx-xxx */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { Properties props = new Properties(); props.load(in); - String database = props.getProperty("database"); - String table = props.getProperty("table"); - String uuid = props.getProperty("table-uuid"); - return database + "." + table + "@" + uuid; + return props.getProperty("table-uuid"); } } From 4eece094f701ea2a414fd25f959a071109caf91a Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:23:29 +0800 Subject: [PATCH 12/23] [design] Simplify createLocalFileIO to use existing context Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 7 ++----- designs/fuse-local-path-design.md | 7 ++----- 2 files changed, 4 insertions(+), 10 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 5d9ec4d87474..0a7af29d13aa 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -466,13 +466,10 @@ private void handleValidationError(ValidationResult result, ValidationMode mode) } /** - * 创建本地 FileIO + * 使用现有 context 创建本地 FileIO */ private FileIO createLocalFileIO(Path localPath) { - return FileIO.get(localPath, CatalogContext.create( - new Options(), - context.hadoopConf() - )); + return FileIO.get(localPath, context); } /** diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index afdeaee5eda5..90571c91d68b 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -468,13 +468,10 @@ private void handleValidationError(ValidationResult result, ValidationMode mode) } /** - * Create local FileIO + * Create local FileIO using existing context */ private FileIO createLocalFileIO(Path localPath) { - return FileIO.get(localPath, CatalogContext.create( - new Options(), - context.hadoopConf() - )); + return FileIO.get(localPath, context); } /** From 6e44797b234e50d618cd47984dcebc7bfc4393fa Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:24:36 +0800 Subject: [PATCH 13/23] [design] Rename .paimon-identifier to .identifier Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 24 ++++++++++++------------ designs/fuse-local-path-design.md | 24 ++++++++++++------------ 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 0a7af29d13aa..ce8050c8b072 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -224,11 +224,11 @@ private Path convertToLocalPath(String originalPath, String localRoot) { └─────────────────────┘ ``` -### .paimon-identifier 文件 +### .identifier 文件 -每个表目录下都包含一个 `.paimon-identifier` 文件用于快速校验： +每个表目录下都包含一个 `.identifier` 文件用于快速校验： -**文件位置**：`<表路径>/.paimon-identifier` +**文件位置**：`<表路径>/.identifier` **文件格式**： ``` @@ -302,7 +302,7 @@ private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Ident } /** - * 第一次校验：检查 .paimon-identifier 文件 + * 第一次校验：检查 .identifier 文件 * 比对本地和远端的表 UUID 确保路径正确性 */ private ValidationResult validateByIdentifierFile( @@ -312,20 +312,20 @@ private ValidationResult validateByIdentifierFile( FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); // 2. 读取远端标识文件 - Path remoteIdentifierFile = new Path(remotePath, ".paimon-identifier"); + Path remoteIdentifierFile = new Path(remotePath, ".identifier"); if (!remoteFileIO.exists(remoteIdentifierFile)) { // 无标识文件，跳过此次校验 - LOG.debug("未找到表 {} 的 .paimon-identifier 文件，跳过标识校验", identifier); + LOG.debug("未找到表 {} 的 .identifier 文件，跳过标识校验", identifier); return ValidationResult.success(); } String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); // 3. 读取本地标识文件 - Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + Path localIdentifierFile = new Path(localPath, ".identifier"); if (!localFileIO.exists(localIdentifierFile)) { return ValidationResult.fail( - "本地 .paimon-identifier 文件未找到: " + localIdentifierFile + + "本地 .identifier 文件未找到: " + localIdentifierFile + "。FUSE 路径可能未正确挂载。"); } @@ -348,7 +348,7 @@ private ValidationResult validateByIdentifierFile( } /** - * 读取 .paimon-identifier 文件内容 + * 读取 .identifier 文件内容 * 格式：table-uuid=xxx-xxx-xxx-xxx */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { @@ -533,7 +533,7 @@ class ValidationResult { | 步骤 | 校验方式 | 描述 | |------|----------|------| -| 1 | `.paimon-identifier` 文件 | 比对本地和远端的表 UUID | +| 1 | `.identifier` 文件 | 比对本地和远端的表 UUID | | 2 | 远端数据校验 | 比对 snapshot/schema 文件内容 | **完整校验流程**： @@ -545,12 +545,12 @@ class ValidationResult { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 第一步：.paimon-identifier 校验 │ +│ 第一步：.identifier 校验 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ 远端存在 .paimon-identifier ? │ + │ 远端存在 .identifier ? │ └───────────────────────────────────────┘ │ │ Yes No diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 90571c91d68b..11aa6f5b47b2 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -226,11 +226,11 @@ New configuration parameter to control validation behavior: └─────────────┘ └─────────────────────┘ ``` -### .paimon-identifier File +### .identifier File -Each table directory contains a `.paimon-identifier` file for quick validation: +Each table directory contains a `.identifier` file for quick validation: -**File Location**: `/.paimon-identifier` +**File Location**: `/.identifier` **File Format**: ``` @@ -304,7 +304,7 @@ private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Ident } /** - * First validation: Check .paimon-identifier file + * First validation: Check .identifier file * Compare table UUID between local and remote to ensure path correctness */ private ValidationResult validateByIdentifierFile( @@ -314,20 +314,20 @@ private ValidationResult validateByIdentifierFile( FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); // 2. Read remote identifier file - Path remoteIdentifierFile = new Path(remotePath, ".paimon-identifier"); + Path remoteIdentifierFile = new Path(remotePath, ".identifier"); if (!remoteFileIO.exists(remoteIdentifierFile)) { // No identifier file, skip this validation - LOG.debug("No .paimon-identifier file found for table: {}, skip identifier validation", identifier); + LOG.debug("No .identifier file found for table: {}, skip identifier validation", identifier); return ValidationResult.success(); } String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); // 3. Read local identifier file - Path localIdentifierFile = new Path(localPath, ".paimon-identifier"); + Path localIdentifierFile = new Path(localPath, ".identifier"); if (!localFileIO.exists(localIdentifierFile)) { return ValidationResult.fail( - "Local .paimon-identifier file not found: " + localIdentifierFile + + "Local .identifier file not found: " + localIdentifierFile + ". The FUSE path may not be mounted correctly."); } @@ -350,7 +350,7 @@ private ValidationResult validateByIdentifierFile( } /** - * Read .paimon-identifier file content + * Read .identifier file content * Format: table-uuid=xxx-xxx-xxx-xxx */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { @@ -535,7 +535,7 @@ class ValidationResult { | Step | Validation | Description | |------|------------|-------------| -| 1 | `.paimon-identifier` file | Compare table UUID between local and remote | +| 1 | `.identifier` file | Compare table UUID between local and remote | | 2 | Remote data validation | Compare snapshot/schema file content | **Complete Validation Flow**: @@ -547,12 +547,12 @@ class ValidationResult { │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ Step 1: .paimon-identifier Validation │ +│ Step 1: .identifier Validation │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ .paimon-identifier exists remotely ? │ + │ .identifier exists remotely ? │ └───────────────────────────────────────┘ │ │ Yes No From 45a13e9e656e08a2773328aec33622f65271e022 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:27:37 +0800 Subject: [PATCH 14/23] [design] Remove created-at from .identifier file format Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 1 - designs/fuse-local-path-design.md | 1 - 2 files changed, 2 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index ce8050c8b072..efef1298d577 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -233,7 +233,6 @@ private Path convertToLocalPath(String originalPath, String localRoot) { **文件格式**： ``` table-uuid=xxx-xxx-xxx-xxx -created-at=2026-03-13T00:00:00Z ``` **用途**： diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 11aa6f5b47b2..cce34a2758d3 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -235,7 +235,6 @@ Each table directory contains a `.identifier` file for quick validation: **File Format**: ``` table-uuid=xxx-xxx-xxx-xxx -created-at=2026-03-13T00:00:00Z ``` **Usage**: From bab21e595bcc4e9fa729dfad915e4e12a9617664 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:32:00 +0800 Subject: [PATCH 15/23] [design] Change .identifier format to JSON, use JsonSerdeUtil for parsing Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 12 ++++++------ designs/fuse-local-path-design.md | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index efef1298d577..d1d9b730a893 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -231,8 +231,8 @@ private Path convertToLocalPath(String originalPath, String localRoot) { **文件位置**：`<表路径>/.identifier` **文件格式**： -``` -table-uuid=xxx-xxx-xxx-xxx +```json +{"table-uuid":"xxx-xxx-xxx-xxx"} ``` **用途**： @@ -348,13 +348,13 @@ private ValidationResult validateByIdentifierFile( /** * 读取 .identifier 文件内容 - * 格式：table-uuid=xxx-xxx-xxx-xxx + * 格式：{"table-uuid":"xxx-xxx-xxx-xxx"} */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { - Properties props = new Properties(); - props.load(in); - return props.getProperty("table-uuid"); + String json = IOUtils.readUTF8Fully(in); + JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); + return node.get("table-uuid").asText(); } } diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index cce34a2758d3..2fcac0caad80 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -233,8 +233,8 @@ Each table directory contains a `.identifier` file for quick validation: **File Location**: `/.identifier` **File Format**: -``` -table-uuid=xxx-xxx-xxx-xxx +```json +{"table-uuid":"xxx-xxx-xxx-xxx"} ``` **Usage**: @@ -350,13 +350,13 @@ private ValidationResult validateByIdentifierFile( /** * Read .identifier file content - * Format: table-uuid=xxx-xxx-xxx-xxx + * Format: {"table-uuid":"xxx-xxx-xxx-xxx"} */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { - Properties props = new Properties(); - props.load(in); - return props.getProperty("table-uuid"); + String json = IOUtils.readUTF8Fully(in); + JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); + return node.get("table-uuid").asText(); } } From 2397e2954f6438db3fc413b841c91824301d0a17 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:33:52 +0800 Subject: [PATCH 16/23] [design] Change field name from table-uuid to uuid for consistency with Paimon internal naming Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 6 +++--- designs/fuse-local-path-design.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index d1d9b730a893..6d3604253284 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -232,7 +232,7 @@ private Path convertToLocalPath(String originalPath, String localRoot) { **文件格式**： ```json -{"table-uuid":"xxx-xxx-xxx-xxx"} +{"uuid":"xxx-xxx-xxx-xxx"} ``` **用途**： @@ -348,13 +348,13 @@ private ValidationResult validateByIdentifierFile( /** * 读取 .identifier 文件内容 - * 格式：{"table-uuid":"xxx-xxx-xxx-xxx"} + * 格式：{"uuid":"xxx-xxx-xxx-xxx"} */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { String json = IOUtils.readUTF8Fully(in); JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); - return node.get("table-uuid").asText(); + return node.get("uuid").asText(); } } diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 2fcac0caad80..093d56082081 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -234,7 +234,7 @@ Each table directory contains a `.identifier` file for quick validation: **File Format**: ```json -{"table-uuid":"xxx-xxx-xxx-xxx"} +{"uuid":"xxx-xxx-xxx-xxx"} ``` **Usage**: @@ -350,13 +350,13 @@ private ValidationResult validateByIdentifierFile( /** * Read .identifier file content - * Format: {"table-uuid":"xxx-xxx-xxx-xxx"} + * Format: {"uuid":"xxx-xxx-xxx-xxx"} */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { String json = IOUtils.readUTF8Fully(in); JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); - return node.get("table-uuid").asText(); + return node.get("uuid").asText(); } } From b792bff5ce9e9c63e57730211582217204edf2cf Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:42:09 +0800 Subject: [PATCH 17/23] Add FUSE error handling design Add comprehensive error handling section for FUSE local path operations: - Error categories: permission/auth, network, service, FUSE-specific - Read/write operation error handling flows with retry logic - New retry configuration options with exponential backoff - Implementation example with isRetryableError classification - Logging guidelines and optional metrics Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 246 +++++++++++++++++++++++++++ designs/fuse-local-path-design.md | 246 +++++++++++++++++++++++++++ 2 files changed, 492 insertions(+) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 6d3604253284..6783ff14ca8b 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -666,3 +666,249 @@ CREATE CATALOG paimon_rest_catalog WITH ( 2. 本地路径必须与远端存储路径具有相同的目录结构 3. 写操作需要本地 FUSE 挂载点具有适当的权限 4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） + +## FUSE 错误处理 + +### 错误分类 + +使用 FUSE 本地路径时，不同场景可能发生不同类型的错误。以下是常见错误类型及处理策略。 + +#### 1. 权限/认证错误 + +| 错误类型 | 场景 | 原因 | 处理策略 | +|----------|------|------|----------| +| `NotAuthorizedException` (HTTP 401) | REST API 调用 | Token 过期或无效 | 刷新 Token 重试，或失败 | +| `ForbiddenException` (HTTP 403) | REST API 调用 | 无资源访问权限 | 记录错误，操作失败 | +| `AccessDeniedException` | 本地文件访问 | FUSE 挂载点无读写权限 | 记录错误，检查挂载权限 | +| `FileNotFoundException` | 本地文件访问 | FUSE 未挂载或路径错误 | 记录错误，检查挂载状态 | + +#### 2. 网络错误 + +| 错误类型 | 场景 | 原因 | 处理策略 | +|----------|------|------|----------| +| `SocketTimeoutException` | 远程读写 | 网络超时 | 指数退避重试 | +| `ConnectException` | 连接尝试 | 连接被拒绝 | 重试或达到最大次数后失败 | +| `ConnectionClosedException` | 数据传输 | 连接意外关闭 | 重试一次，然后失败 | +| `NoRouteToHostException` | 连接尝试 | 网络不可达 | 记录错误，立即失败 | +| `UnknownHostException` | DNS 解析 | DNS 解析失败 | 记录错误，立即失败 | +| `InterruptedIOException` | I/O 操作 | 线程被中断 | 传播中断状态 | + +#### 3. 服务错误 + +| 错误类型 | 场景 | 原因 | 处理策略 | +|----------|------|------|----------| +| `ServiceUnavailableException` (HTTP 503) | REST API 调用 | 服务暂时不可用 | 退避重试，遵循 `Retry-After` 响应头 | +| HTTP 429 Too Many Requests | REST API 调用 | 限流 | 根据 `Retry-After` 延迟重试 | + +#### 4. FUSE 特有错误 + +| 错误类型 | 场景 | 原因 | 处理策略 | +|----------|------|------|----------| +| `IOException` (transport failed) | 本地文件读写 | FUSE 挂载断开 | 重试一次，回退到远程 | +| `IOException` (stale file handle) | 文件操作 | 文件被其他进程删除/修改 | 重新打开文件或失败 | + +### 错误处理策略 + +#### 读操作错误处理 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 读操作流程 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 尝试从 FUSE 本地路径读取 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 成功 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ 返回数据 │ │ 分类错误类型 │ + └─────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 可重试错误 ? │ + │ (网络超时、临时性错误) │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ 退避重试 │ │ 记录错误并失败 │ +│ (最多 3 次) │ │ 抛出 IOException 及详细信息 │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 重试成功 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ 返回数据 │ │ 记录错误并失败 │ + └─────────────┘ └─────────────────────────────────────┘ +``` + +#### 写操作错误处理 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 写操作流程 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 尝试写入 FUSE 本地路径 │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 成功 ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ 提交写入 │ │ 分类错误类型 │ + └─────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ 权限错误 ? │ + │ (AccessDenied, Forbidden) │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ 立即失败 │ │ 临时性错误 ? │ +│ 抛出异常 │ │ (超时、连接重置) │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────────────┐ ┌─────────────────────┐ + │ 退避重试 │ │ 立即失败 │ + │ (最多 3 次) │ │ 抛出异常 │ + └─────────────────────┘ └─────────────────────┘ +``` + +### 错误处理配置 + +新增以下配置选项： + +| 参数 | 类型 | 默认值 | 说明 | +|------|------|--------|------| +| `fuse.local-path.retry-enabled` | Boolean | `true` | 是否启用临时性错误重试 | +| `fuse.local-path.max-retries` | Integer | `3` | 最大重试次数 | +| `fuse.local-path.retry-interval` | Duration | `1s` | 初始重试间隔（指数退避） | +| `fuse.local-path.retry-max-interval` | Duration | `10s` | 最大重试间隔 | + +### 实现示例 + +```java +/** + * 执行文件操作，支持临时性错误重试 + */ +private T executeWithRetry(SupplierWithIOException operation, String operationName) throws IOException { + if (!fuseConfig.retryEnabled()) { + return operation.get(); + } + + int attempt = 0; + IOException lastException = null; + long intervalMs = fuseConfig.retryIntervalMs(); + + while (attempt <= fuseConfig.maxRetries()) { + try { + return operation.get(); + } catch (IOException e) { + lastException = e; + attempt++; + + if (!isRetryableError(e) || attempt > fuseConfig.maxRetries()) { + throw e; + } + + LOG.warn("{} 失败 (第 {} 次尝试), {} ms 后重试: {}", + operationName, attempt, intervalMs, e.getMessage()); + + try { + Thread.sleep(intervalMs); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + throw new IOException("重试期间操作被中断", ie); + } + + // 指数退避 + intervalMs = Math.min(intervalMs * 2, fuseConfig.retryMaxIntervalMs()); + } + } + + throw lastException; +} + +/** + * 判断错误是否可重试 + */ +private boolean isRetryableError(IOException e) { + // 临时性网络错误 - 重试 + if (e instanceof SocketTimeoutException) return true; + if (e instanceof ConnectException) return true; + if (e instanceof ConnectionClosedException) return true; + + // 权限错误 - 不重试 + if (e instanceof AccessDeniedException) return false; + if (e instanceof FileNotFoundException) return false; + + // 检查 FUSE 特有传输错误 + String message = e.getMessage(); + if (message != null) { + // FUSE 传输失败（挂载断开） + if (message.contains("Transport endpoint is not connected")) return false; + if (message.contains("Stale file handle")) return true; + } + + // 默认：重试未知 I/O 错误（可能是临时性的） + return true; +} + +@FunctionalInterface +interface SupplierWithIOException { + T get() throws IOException; +} +``` + +### 错误日志与指标 + +#### 日志规范 + +| 日志级别 | 场景 | 示例 | +|----------|------|------| +| ERROR | 所有重试后操作失败 | `FUSE 读取在 3 次重试后失败: /mnt/fuse/db/table/snapshot-1` | +| WARN | 临时性错误，将重试 | `FUSE 读取超时，重试中 (第 1/3 次): /mnt/fuse/...` | +| INFO | 回退到远程 | `FUSE 路径不可用，回退到远程 FileIO` | +| DEBUG | 重试详情 | `重试间隔: 2000ms, 下次尝试: 2` | + +#### 指标（可选） + +| 指标 | 类型 | 说明 | +|------|------|------| +| `fuse.read.errors` | Counter | 按类型统计的读取错误总数 | +| `fuse.write.errors` | Counter | 按类型统计的写入错误总数 | +| `fuse.retry.count` | Counter | 重试尝试总数 | +| `fuse.fallback.count` | Counter | 回退到远程 FileIO 的次数 | diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 093d56082081..66236e8ccc75 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -670,3 +670,249 @@ CREATE CATALOG paimon_rest_catalog WITH ( 3. Write operations require proper permissions on the local FUSE mount 4. Windows platform has limited FUSE support (requires third-party tools like WinFsp) +## FUSE Error Handling + +### Error Categories + +When using FUSE local paths, errors may occur in different scenarios. Below are common error types and handling strategies. + +#### 1. Permission/Authentication Errors + +| Error Type | Scenario | Cause | Handling Strategy | +|------------|----------|-------|-------------------| +| `NotAuthorizedException` (HTTP 401) | REST API call | Token expired or invalid | Refresh token and retry, or fail | +| `ForbiddenException` (HTTP 403) | REST API call | No permission for resource | Log error, fail operation | +| `AccessDeniedException` | Local file access | No read/write permission on FUSE mount | Log error, check mount permissions | +| `FileNotFoundException` | Local file access | FUSE not mounted or path incorrect | Log error, check mount status | + +#### 2. Network Errors + +| Error Type | Scenario | Cause | Handling Strategy | +|------------|----------|-------|-------------------| +| `SocketTimeoutException` | Remote read/write | Network timeout | Retry with exponential backoff | +| `ConnectException` | Connection attempt | Connection refused | Retry or fail after max attempts | +| `ConnectionClosedException` | Data transfer | Connection closed unexpectedly | Retry once, then fail | +| `NoRouteToHostException` | Connection attempt | Network unreachable | Log error, fail immediately | +| `UnknownHostException` | DNS resolution | DNS resolution failure | Log error, fail immediately | +| `InterruptedIOException` | I/O operation | Thread interrupted | Propagate interruption | + +#### 3. Service Errors + +| Error Type | Scenario | Cause | Handling Strategy | +|------------|----------|-------|-------------------| +| `ServiceUnavailableException` (HTTP 503) | REST API call | Service temporarily unavailable | Retry with backoff, respect `Retry-After` header | +| HTTP 429 Too Many Requests | REST API call | Rate limiting | Retry after delay from `Retry-After` header | + +#### 4. FUSE-Specific Errors + +| Error Type | Scenario | Cause | Handling Strategy | +|------------|----------|-------|-------------------| +| `IOException` (transport failed) | Local file read/write | FUSE mount disconnected | Retry once, fallback to remote | +| `IOException` (stale file handle) | File operation | File deleted/modified by another process | Reopen file or fail | + +### Error Handling Strategy + +#### Read Operation Error Handling + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Read Operation Flow │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Attempt to read from FUSE local path │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Success ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ Return data │ │ Classify error type │ + └─────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Retryable error ? │ + │ (Network timeout, transient errors) │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ Retry with backoff │ │ Log error and fail │ +│ (max 3 retries) │ │ Throw IOException with details │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Retry succeeded ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ Return data │ │ Log error and fail │ + └─────────────┘ └─────────────────────────────────────┘ +``` + +#### Write Operation Error Handling + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Write Operation Flow │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Attempt to write to FUSE local path │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Success ? │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────────────┐ + │ Commit write│ │ Classify error type │ + └─────────────┘ └─────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ Permission error ? │ + │ (AccessDenied, Forbidden) │ + └───────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ +┌─────────────────────┐ ┌─────────────────────────────────────┐ +│ Fail immediately │ │ Transient error ? │ +│ Throw exception │ │ (Timeout, connection reset) │ +└─────────────────────┘ └─────────────────────────────────────┘ + │ │ + Yes No + │ │ + ▼ ▼ + ┌─────────────────────┐ ┌─────────────────────┐ + │ Retry with backoff │ │ Fail immediately │ + │ (max 3 retries) │ │ Throw exception │ + └─────────────────────┘ └─────────────────────┘ +``` + +### Error Handling Configuration + +Add the following configuration options: + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `fuse.local-path.retry-enabled` | Boolean | `true` | Enable retry on transient errors | +| `fuse.local-path.max-retries` | Integer | `3` | Maximum number of retries | +| `fuse.local-path.retry-interval` | Duration | `1s` | Initial retry interval (exponential backoff) | +| `fuse.local-path.retry-max-interval` | Duration | `10s` | Maximum retry interval | + +### Implementation Example + +```java +/** + * Execute file operation with retry support for transient errors + */ +private T executeWithRetry(SupplierWithIOException operation, String operationName) throws IOException { + if (!fuseConfig.retryEnabled()) { + return operation.get(); + } + + int attempt = 0; + IOException lastException = null; + long intervalMs = fuseConfig.retryIntervalMs(); + + while (attempt <= fuseConfig.maxRetries()) { + try { + return operation.get(); + } catch (IOException e) { + lastException = e; + attempt++; + + if (!isRetryableError(e) || attempt > fuseConfig.maxRetries()) { + throw e; + } + + LOG.warn("{} failed (attempt {}), retrying in {} ms: {}", + operationName, attempt, intervalMs, e.getMessage()); + + try { + Thread.sleep(intervalMs); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + throw new IOException("Operation interrupted during retry", ie); + } + + // Exponential backoff + intervalMs = Math.min(intervalMs * 2, fuseConfig.retryMaxIntervalMs()); + } + } + + throw lastException; +} + +/** + * Determine if error is retryable + */ +private boolean isRetryableError(IOException e) { + // Transient network errors - retry + if (e instanceof SocketTimeoutException) return true; + if (e instanceof ConnectException) return true; + if (e instanceof ConnectionClosedException) return true; + + // Permission errors - do not retry + if (e instanceof AccessDeniedException) return false; + if (e instanceof FileNotFoundException) return false; + + // Check for FUSE-specific transport errors + String message = e.getMessage(); + if (message != null) { + // FUSE transport failure (mount disconnected) + if (message.contains("Transport endpoint is not connected")) return false; + if (message.contains("Stale file handle")) return true; + } + + // Default: retry unknown I/O errors (may be transient) + return true; +} + +@FunctionalInterface +interface SupplierWithIOException { + T get() throws IOException; +} +``` + +### Error Logging and Metrics + +#### Logging Guidelines + +| Log Level | Scenario | Example | +|-----------|----------|---------| +| ERROR | Operation failed after all retries | `FUSE read failed after 3 retries: /mnt/fuse/db/table/snapshot-1` | +| WARN | Transient error, will retry | `FUSE read timeout, retrying (attempt 1/3): /mnt/fuse/...` | +| INFO | Fallback to remote | `FUSE path unavailable, falling back to remote FileIO` | +| DEBUG | Retry details | `Retry interval: 2000ms, next attempt: 2` | + +#### Metrics (Optional) + +| Metric | Type | Description | +|--------|------|-------------| +| `fuse.read.errors` | Counter | Total read errors by type | +| `fuse.write.errors` | Counter | Total write errors by type | +| `fuse.retry.count` | Counter | Total retry attempts | +| `fuse.fallback.count` | Counter | Times fallback to remote FileIO | + From 9995d2a43a80a8fd3041370a6549e03f55944a4a Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 15:49:18 +0800 Subject: [PATCH 18/23] Simplify FUSE error handling to focus on FUSE-specific errors only Remove network, REST API, and permission error handling since they are already handled by existing Paimon mechanisms. Focus only on: - Transport endpoint is not connected (mount disconnection) - Stale file handle - Device or resource busy - Input/output error from FUSE backend Add best practices section for FUSE users. Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 290 ++++++++++----------------- designs/fuse-local-path-design.md | 290 ++++++++++----------------- 2 files changed, 210 insertions(+), 370 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 6783ff14ca8b..3e4958dc22cf 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -669,222 +669,146 @@ CREATE CATALOG paimon_rest_catalog WITH ( ## FUSE 错误处理 -### 错误分类 +本节仅涵盖 FUSE 特有的错误。其他错误（网络、REST API、权限）已由 Paimon 现有机制处理。 -使用 FUSE 本地路径时，不同场景可能发生不同类型的错误。以下是常见错误类型及处理策略。 - -#### 1. 权限/认证错误 - -| 错误类型 | 场景 | 原因 | 处理策略 | -|----------|------|------|----------| -| `NotAuthorizedException` (HTTP 401) | REST API 调用 | Token 过期或无效 | 刷新 Token 重试，或失败 | -| `ForbiddenException` (HTTP 403) | REST API 调用 | 无资源访问权限 | 记录错误，操作失败 | -| `AccessDeniedException` | 本地文件访问 | FUSE 挂载点无读写权限 | 记录错误，检查挂载权限 | -| `FileNotFoundException` | 本地文件访问 | FUSE 未挂载或路径错误 | 记录错误，检查挂载状态 | - -#### 2. 网络错误 - -| 错误类型 | 场景 | 原因 | 处理策略 | -|----------|------|------|----------| -| `SocketTimeoutException` | 远程读写 | 网络超时 | 指数退避重试 | -| `ConnectException` | 连接尝试 | 连接被拒绝 | 重试或达到最大次数后失败 | -| `ConnectionClosedException` | 数据传输 | 连接意外关闭 | 重试一次，然后失败 | -| `NoRouteToHostException` | 连接尝试 | 网络不可达 | 记录错误，立即失败 | -| `UnknownHostException` | DNS 解析 | DNS 解析失败 | 记录错误，立即失败 | -| `InterruptedIOException` | I/O 操作 | 线程被中断 | 传播中断状态 | - -#### 3. 服务错误 +### FUSE 特有错误 | 错误类型 | 场景 | 原因 | 处理策略 | |----------|------|------|----------| -| `ServiceUnavailableException` (HTTP 503) | REST API 调用 | 服务暂时不可用 | 退避重试，遵循 `Retry-After` 响应头 | -| HTTP 429 Too Many Requests | REST API 调用 | 限流 | 根据 `Retry-After` 延迟重试 | +| `Transport endpoint is not connected` | 本地文件读写 | FUSE 挂载断开或崩溃 | 立即失败，记录错误并提示检查挂载状态 | +| `Stale file handle` | 文件操作 | 文件被其他进程删除/修改 | 重试一次（重新打开文件） | +| `Device or resource busy` | 删除/重命名操作 | 文件仍被其他进程占用 | 退避重试 | +| `Input/output error` | 任意文件操作 | FUSE 后端故障（远端存储问题） | 失败并给出明确错误信息 | +| `No such file or directory`（意外） | 文件操作 | FUSE 挂载点未就绪 | 检查挂载状态，失败 | -#### 4. FUSE 特有错误 +### 错误处理策略 -| 错误类型 | 场景 | 原因 | 处理策略 | -|----------|------|------|----------| -| `IOException` (transport failed) | 本地文件读写 | FUSE 挂载断开 | 重试一次，回退到远程 | -| `IOException` (stale file handle) | 文件操作 | 文件被其他进程删除/修改 | 重新打开文件或失败 | +#### FUSE 挂载断开（最关键） -### 错误处理策略 +最关键的 FUSE 特有错误是挂载断开（`Transport endpoint is not connected`）。该错误表示： +- FUSE 进程崩溃 +- 网络问题导致 FUSE 与远端存储断开连接 +- FUSE 挂载被手动卸载 -#### 读操作错误处理 +**处理方式**：立即失败并给出明确错误信息。不要重试，因为必须先恢复挂载。 ``` ┌─────────────────────────────────────────────────────────────┐ -│ 读操作流程 │ +│ FUSE 挂载断开处理 │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ IOException: "Transport endpoint is │ + │ not connected" ? │ + └───────────────────────────────────────┘ + │ + Yes + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 尝试从 FUSE 本地路径读取 │ +│ LOG.error("FUSE 挂载断开，路径: {}。请检查: │ +│ 1. FUSE 进程是否运行 │ +│ 2. 挂载点是否存在: ls -la /mnt/fuse/... │ +│ 3. 如需重新挂载: fusermount -u /mnt/fuse && ...") │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ 成功 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ 返回数据 │ │ 分类错误类型 │ - └─────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 可重试错误 ? │ - │ (网络超时、临时性错误) │ + │ 抛出 IOException 并附带明确信息 │ └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 退避重试 │ │ 记录错误并失败 │ -│ (最多 3 次) │ │ 抛出 IOException 及详细信息 │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 重试成功 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ 返回数据 │ │ 记录错误并失败 │ - └─────────────┘ └─────────────────────────────────────┘ ``` -#### 写操作错误处理 +#### Stale File Handle（过期文件句柄） + +当文件在我们持有打开句柄时被其他进程删除或修改，会触发此错误。 + +**处理方式**：重试一次，重新打开文件。 ``` ┌─────────────────────────────────────────────────────────────┐ -│ 写操作流程 │ +│ Stale File Handle 处理 │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ IOException: "Stale file handle" ? │ + └───────────────────────────────────────┘ + │ + Yes + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 尝试写入 FUSE 本地路径 │ +│ LOG.warn("Stale file handle: {}, 重试中...", path) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ 成功 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ 提交写入 │ │ 分类错误类型 │ - └─────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 权限错误 ? │ - │ (AccessDenied, Forbidden) │ + │ 重试一次: 重新打开文件 │ └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 立即失败 │ │ 临时性错误 ? │ -│ 抛出异常 │ │ (超时、连接重置) │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────────────┐ ┌─────────────────────┐ - │ 退避重试 │ │ 立即失败 │ - │ (最多 3 次) │ │ 抛出异常 │ - └─────────────────────┘ └─────────────────────┘ + │ + ┌─────────┴─────────┐ + 成功 失败 + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ 继续操作 │ │ 抛出 IOException │ + │ │ │ 并附带详细信息 │ + └─────────────┘ └─────────────────────┘ ``` -### 错误处理配置 - -新增以下配置选项： - -| 参数 | 类型 | 默认值 | 说明 | -|------|------|--------|------| -| `fuse.local-path.retry-enabled` | Boolean | `true` | 是否启用临时性错误重试 | -| `fuse.local-path.max-retries` | Integer | `3` | 最大重试次数 | -| `fuse.local-path.retry-interval` | Duration | `1s` | 初始重试间隔（指数退避） | -| `fuse.local-path.retry-max-interval` | Duration | `10s` | 最大重试间隔 | - ### 实现示例 ```java /** - * 执行文件操作，支持临时性错误重试 + * 检查是否为 FUSE 挂载断开错误 */ -private T executeWithRetry(SupplierWithIOException operation, String operationName) throws IOException { - if (!fuseConfig.retryEnabled()) { - return operation.get(); - } - - int attempt = 0; - IOException lastException = null; - long intervalMs = fuseConfig.retryIntervalMs(); - - while (attempt <= fuseConfig.maxRetries()) { - try { - return operation.get(); - } catch (IOException e) { - lastException = e; - attempt++; - - if (!isRetryableError(e) || attempt > fuseConfig.maxRetries()) { - throw e; - } - - LOG.warn("{} 失败 (第 {} 次尝试), {} ms 后重试: {}", - operationName, attempt, intervalMs, e.getMessage()); - - try { - Thread.sleep(intervalMs); - } catch (InterruptedException ie) { - Thread.currentThread().interrupt(); - throw new IOException("重试期间操作被中断", ie); - } - - // 指数退避 - intervalMs = Math.min(intervalMs * 2, fuseConfig.retryMaxIntervalMs()); - } - } - - throw lastException; +private boolean isFuseMountDisconnected(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Transport endpoint is not connected"); } /** - * 判断错误是否可重试 + * 检查是否为 Stale file handle 错误 */ -private boolean isRetryableError(IOException e) { - // 临时性网络错误 - 重试 - if (e instanceof SocketTimeoutException) return true; - if (e instanceof ConnectException) return true; - if (e instanceof ConnectionClosedException) return true; - - // 权限错误 - 不重试 - if (e instanceof AccessDeniedException) return false; - if (e instanceof FileNotFoundException) return false; - - // 检查 FUSE 特有传输错误 +private boolean isStaleFileHandle(IOException e) { String message = e.getMessage(); - if (message != null) { - // FUSE 传输失败（挂载断开） - if (message.contains("Transport endpoint is not connected")) return false; - if (message.contains("Stale file handle")) return true; - } + return message != null && + message.contains("Stale file handle"); +} - // 默认：重试未知 I/O 错误（可能是临时性的） - return true; +/** + * 执行文件操作，处理 FUSE 特有错误 + */ +private T executeWithFuseErrorHandling( + SupplierWithIOException operation, + Path path, + String operationName) throws IOException { + + try { + return operation.get(); + } catch (IOException e) { + // FUSE 挂载断开 - 立即失败 + if (isFuseMountDisconnected(e)) { + LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + + "2) 挂载点是否存在, 3) 如需重新挂载", path); + throw new IOException("FUSE 挂载断开: " + path, e); + } + + // Stale file handle - 重试一次 + if (isStaleFileHandle(e)) { + LOG.warn("Stale file handle: {}, 重试一次...", path); + try { + return operation.get(); + } catch (IOException retryError) { + throw new IOException("Stale file handle (重试失败): " + path, retryError); + } + } + + // 其他错误 - 直接抛出（由现有机制处理） + throw e; + } } @FunctionalInterface @@ -893,22 +817,18 @@ interface SupplierWithIOException { } ``` -### 错误日志与指标 - -#### 日志规范 +### 日志规范 | 日志级别 | 场景 | 示例 | |----------|------|------| -| ERROR | 所有重试后操作失败 | `FUSE 读取在 3 次重试后失败: /mnt/fuse/db/table/snapshot-1` | -| WARN | 临时性错误，将重试 | `FUSE 读取超时，重试中 (第 1/3 次): /mnt/fuse/...` | -| INFO | 回退到远程 | `FUSE 路径不可用，回退到远程 FileIO` | -| DEBUG | 重试详情 | `重试间隔: 2000ms, 下次尝试: 2` | - -#### 指标（可选） - -| 指标 | 类型 | 说明 | -|------|------|------| -| `fuse.read.errors` | Counter | 按类型统计的读取错误总数 | -| `fuse.write.errors` | Counter | 按类型统计的写入错误总数 | -| `fuse.retry.count` | Counter | 重试尝试总数 | -| `fuse.fallback.count` | Counter | 回退到远程 FileIO 的次数 | +| ERROR | FUSE 挂载断开 | `FUSE 挂载断开，路径: /mnt/fuse/db/table/snapshot-1` | +| WARN | Stale file handle | `Stale file handle: /mnt/fuse/..., 重试一次...` | +| INFO | 正常 FUSE 操作 | （可选，用于调试） | + +### FUSE 用户最佳实践 + +1. **监控 FUSE 进程**：使用 `ps aux | grep fusermount` 或 FUSE 工具的监控功能 +2. **健康检查**：定期使用 `ls` 或 `stat` 检查挂载点 +3. **自动重启**：考虑使用 systemd 或 supervisor 在崩溃时自动重启 FUSE +4. **日志分析**：查看 `dmesg` 或 FUSE 日志进行根因分析 + diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 66236e8ccc75..3a3a322ebfaa 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -672,222 +672,147 @@ CREATE CATALOG paimon_rest_catalog WITH ( ## FUSE Error Handling -### Error Categories +This section covers FUSE-specific errors only. Other errors (network, REST API, permission) are already handled by existing mechanisms in Paimon. -When using FUSE local paths, errors may occur in different scenarios. Below are common error types and handling strategies. - -#### 1. Permission/Authentication Errors - -| Error Type | Scenario | Cause | Handling Strategy | -|------------|----------|-------|-------------------| -| `NotAuthorizedException` (HTTP 401) | REST API call | Token expired or invalid | Refresh token and retry, or fail | -| `ForbiddenException` (HTTP 403) | REST API call | No permission for resource | Log error, fail operation | -| `AccessDeniedException` | Local file access | No read/write permission on FUSE mount | Log error, check mount permissions | -| `FileNotFoundException` | Local file access | FUSE not mounted or path incorrect | Log error, check mount status | - -#### 2. Network Errors - -| Error Type | Scenario | Cause | Handling Strategy | -|------------|----------|-------|-------------------| -| `SocketTimeoutException` | Remote read/write | Network timeout | Retry with exponential backoff | -| `ConnectException` | Connection attempt | Connection refused | Retry or fail after max attempts | -| `ConnectionClosedException` | Data transfer | Connection closed unexpectedly | Retry once, then fail | -| `NoRouteToHostException` | Connection attempt | Network unreachable | Log error, fail immediately | -| `UnknownHostException` | DNS resolution | DNS resolution failure | Log error, fail immediately | -| `InterruptedIOException` | I/O operation | Thread interrupted | Propagate interruption | - -#### 3. Service Errors +### FUSE-Specific Errors | Error Type | Scenario | Cause | Handling Strategy | |------------|----------|-------|-------------------| -| `ServiceUnavailableException` (HTTP 503) | REST API call | Service temporarily unavailable | Retry with backoff, respect `Retry-After` header | -| HTTP 429 Too Many Requests | REST API call | Rate limiting | Retry after delay from `Retry-After` header | +| `Transport endpoint is not connected` | Local file read/write | FUSE mount disconnected or crashed | Fail immediately, log error with mount check suggestion | +| `Stale file handle` | File operation | File deleted/modified by another process | Retry once (reopen file) | +| `Device or resource busy` | Delete/rename operation | File still open by another process | Retry with backoff | +| `Input/output error` | Any file operation | FUSE backend failure (remote storage issue) | Fail with clear error message | +| `No such file or directory` (unexpected) | File operation | FUSE mount point not ready | Check mount status, fail | -#### 4. FUSE-Specific Errors +### Error Handling Strategy -| Error Type | Scenario | Cause | Handling Strategy | -|------------|----------|-------|-------------------| -| `IOException` (transport failed) | Local file read/write | FUSE mount disconnected | Retry once, fallback to remote | -| `IOException` (stale file handle) | File operation | File deleted/modified by another process | Reopen file or fail | +#### FUSE Mount Disconnection (Most Critical) -### Error Handling Strategy +The most critical FUSE-specific error is mount disconnection (`Transport endpoint is not connected`). This error indicates: +- FUSE process crashed +- Network issue caused FUSE to disconnect from remote storage +- FUSE mount was manually unmounted -#### Read Operation Error Handling +**Handling**: Fail immediately with clear error message. Do NOT retry as the mount must be restored first. ``` ┌─────────────────────────────────────────────────────────────┐ -│ Read Operation Flow │ +│ FUSE Mount Disconnection Handling │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ IOException: "Transport endpoint is │ + │ not connected" ? │ + └───────────────────────────────────────┘ + │ + Yes + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ Attempt to read from FUSE local path │ +│ LOG.error("FUSE mount disconnected. Please check: │ +│ 1. FUSE process is running │ +│ 2. Mount point exists: ls -la /mnt/fuse/... │ +│ 3. Remount if needed: fusermount -u /mnt/fuse && ...") │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Success ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ Return data │ │ Classify error type │ - └─────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Retryable error ? │ - │ (Network timeout, transient errors) │ + │ Throw IOException with clear message │ └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ Retry with backoff │ │ Log error and fail │ -│ (max 3 retries) │ │ Throw IOException with details │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Retry succeeded ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ Return data │ │ Log error and fail │ - └─────────────┘ └─────────────────────────────────────┘ ``` -#### Write Operation Error Handling +#### Stale File Handle + +This error occurs when a file is deleted or modified by another process while we have an open handle. + +**Handling**: Retry once by reopening the file. ``` ┌─────────────────────────────────────────────────────────────┐ -│ Write Operation Flow │ +│ Stale File Handle Handling │ └─────────────────────────────────────────────────────────────┘ │ ▼ + ┌───────────────────────────────────────┐ + │ IOException: "Stale file handle" ? │ + └───────────────────────────────────────┘ + │ + Yes + │ + ▼ ┌─────────────────────────────────────────────────────────────┐ -│ Attempt to write to FUSE local path │ +│ LOG.warn("Stale file handle for {}, retrying...", path) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Success ? │ + │ Retry once: reopen file │ └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────────────┐ - │ Commit write│ │ Classify error type │ - └─────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Permission error ? │ - │ (AccessDenied, Forbidden) │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ Fail immediately │ │ Transient error ? │ -│ Throw exception │ │ (Timeout, connection reset) │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────────────┐ ┌─────────────────────┐ - │ Retry with backoff │ │ Fail immediately │ - │ (max 3 retries) │ │ Throw exception │ - └─────────────────────┘ └─────────────────────┘ + │ + ┌─────────┴─────────┐ + Success Fail + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ Continue │ │ Throw IOException │ + │ operation │ │ with details │ + └─────────────┘ └─────────────────────┘ ``` -### Error Handling Configuration - -Add the following configuration options: - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `fuse.local-path.retry-enabled` | Boolean | `true` | Enable retry on transient errors | -| `fuse.local-path.max-retries` | Integer | `3` | Maximum number of retries | -| `fuse.local-path.retry-interval` | Duration | `1s` | Initial retry interval (exponential backoff) | -| `fuse.local-path.retry-max-interval` | Duration | `10s` | Maximum retry interval | - ### Implementation Example ```java /** - * Execute file operation with retry support for transient errors + * Check if error is FUSE mount disconnection */ -private T executeWithRetry(SupplierWithIOException operation, String operationName) throws IOException { - if (!fuseConfig.retryEnabled()) { - return operation.get(); - } - - int attempt = 0; - IOException lastException = null; - long intervalMs = fuseConfig.retryIntervalMs(); - - while (attempt <= fuseConfig.maxRetries()) { - try { - return operation.get(); - } catch (IOException e) { - lastException = e; - attempt++; - - if (!isRetryableError(e) || attempt > fuseConfig.maxRetries()) { - throw e; - } - - LOG.warn("{} failed (attempt {}), retrying in {} ms: {}", - operationName, attempt, intervalMs, e.getMessage()); - - try { - Thread.sleep(intervalMs); - } catch (InterruptedException ie) { - Thread.currentThread().interrupt(); - throw new IOException("Operation interrupted during retry", ie); - } - - // Exponential backoff - intervalMs = Math.min(intervalMs * 2, fuseConfig.retryMaxIntervalMs()); - } - } - - throw lastException; +private boolean isFuseMountDisconnected(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Transport endpoint is not connected"); } /** - * Determine if error is retryable + * Check if error is stale file handle */ -private boolean isRetryableError(IOException e) { - // Transient network errors - retry - if (e instanceof SocketTimeoutException) return true; - if (e instanceof ConnectException) return true; - if (e instanceof ConnectionClosedException) return true; - - // Permission errors - do not retry - if (e instanceof AccessDeniedException) return false; - if (e instanceof FileNotFoundException) return false; - - // Check for FUSE-specific transport errors +private boolean isStaleFileHandle(IOException e) { String message = e.getMessage(); - if (message != null) { - // FUSE transport failure (mount disconnected) - if (message.contains("Transport endpoint is not connected")) return false; - if (message.contains("Stale file handle")) return true; - } + return message != null && + message.contains("Stale file handle"); +} - // Default: retry unknown I/O errors (may be transient) - return true; +/** + * Execute file operation with FUSE-specific error handling + */ +private T executeWithFuseErrorHandling( + SupplierWithIOException operation, + Path path, + String operationName) throws IOException { + + try { + return operation.get(); + } catch (IOException e) { + // FUSE mount disconnected - fail immediately + if (isFuseMountDisconnected(e)) { + LOG.error("FUSE mount disconnected for path: {}. " + + "Please check: 1) FUSE process is running, " + + "2) Mount point exists, 3) Remount if needed.", path); + throw new IOException("FUSE mount disconnected: " + path, e); + } + + // Stale file handle - retry once + if (isStaleFileHandle(e)) { + LOG.warn("Stale file handle for {}, retrying once...", path); + try { + return operation.get(); + } catch (IOException retryError) { + throw new IOException("Stale file handle (retry failed): " + path, retryError); + } + } + + // Other errors - propagate as-is (handled by existing mechanisms) + throw e; + } } @FunctionalInterface @@ -896,23 +821,18 @@ interface SupplierWithIOException { } ``` -### Error Logging and Metrics - -#### Logging Guidelines +### Logging Guidelines | Log Level | Scenario | Example | |-----------|----------|---------| -| ERROR | Operation failed after all retries | `FUSE read failed after 3 retries: /mnt/fuse/db/table/snapshot-1` | -| WARN | Transient error, will retry | `FUSE read timeout, retrying (attempt 1/3): /mnt/fuse/...` | -| INFO | Fallback to remote | `FUSE path unavailable, falling back to remote FileIO` | -| DEBUG | Retry details | `Retry interval: 2000ms, next attempt: 2` | - -#### Metrics (Optional) - -| Metric | Type | Description | -|--------|------|-------------| -| `fuse.read.errors` | Counter | Total read errors by type | -| `fuse.write.errors` | Counter | Total write errors by type | -| `fuse.retry.count` | Counter | Total retry attempts | -| `fuse.fallback.count` | Counter | Times fallback to remote FileIO | +| ERROR | FUSE mount disconnected | `FUSE mount disconnected for path: /mnt/fuse/db/table/snapshot-1` | +| WARN | Stale file handle | `Stale file handle for /mnt/fuse/..., retrying once...` | +| INFO | Normal FUSE operations | (Optional, for debugging) | + +### Best Practices for FUSE Users + +1. **Monitor FUSE process**: Use `ps aux | grep fusermount` or your FUSE tool's monitoring +2. **Health check**: Periodically check mount point with `ls` or `stat` +3. **Auto-restart**: Consider using systemd or supervisor to auto-restart FUSE on crash +4. **Log FUSE errors**: Check `dmesg` or FUSE logs for root cause analysis From abec0689429ecccbc7d0f7bf07ff5c294d2b307b Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Fri, 13 Mar 2026 16:06:55 +0800 Subject: [PATCH 19/23] Add configurable retry with exponential backoff for FUSE error handling - Add retry configuration parameters: max-attempts, initial-delay-ms, max-delay-ms - Implement FuseErrorHandler with exponential backoff algorithm - Add FuseAwareFileIO wrapper to delegate LocalFileIO with error handling - Support retry for stale file handle and device busy errors - Fail immediately for FUSE mount disconnection (no retry) Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 220 +++++++++++++++++++++----- designs/fuse-local-path-design.md | 224 ++++++++++++++++++++++----- 2 files changed, 359 insertions(+), 85 deletions(-) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 3e4958dc22cf..889c5888ff77 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -41,6 +41,10 @@ limitations under the License. | `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | | `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | | `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | +| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | +| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | +| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | ## 使用示例 @@ -761,62 +765,194 @@ CREATE CATALOG paimon_rest_catalog WITH ( ```java /** - * 检查是否为 FUSE 挂载断开错误 + * FUSE 错误处理器，支持可配置的重试和指数退避。 */ -private boolean isFuseMountDisconnected(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Transport endpoint is not connected"); +public class FuseErrorHandler { + private final int maxAttempts; + private final long initialDelayMs; + private final long maxDelayMs; + + public FuseErrorHandler(int maxAttempts, long initialDelayMs, long maxDelayMs) { + this.maxAttempts = maxAttempts; + this.initialDelayMs = initialDelayMs; + this.maxDelayMs = maxDelayMs; + } + + /** + * 检查是否为 FUSE 挂载断开错误 + */ + public boolean isFuseMountDisconnected(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Transport endpoint is not connected"); + } + + /** + * 检查是否为 Stale file handle 错误 + */ + public boolean isStaleFileHandle(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Stale file handle"); + } + + /** + * 检查是否为 Device or resource busy 错误 + */ + public boolean isDeviceBusy(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Device or resource busy"); + } + + /** + * 计算指数退避延迟。 + * 公式: min(initialDelay * 2^attempt, maxDelay) + */ + private long calculateDelay(int attempt) { + long delay = initialDelayMs * (1L << attempt); // 2^attempt + return Math.min(delay, maxDelayMs); + } + + /** + * 执行文件操作，处理 FUSE 特有错误。 + * 对可重试错误使用指数退避策略。 + */ + public T executeWithFuseErrorHandling( + SupplierWithIOException operation, + Path path, + String operationName) throws IOException { + + IOException lastException = null; + + for (int attempt = 0; attempt < maxAttempts; attempt++) { + try { + return operation.get(); + } catch (IOException e) { + lastException = e; + + // FUSE 挂载断开 - 立即失败，不重试 + if (isFuseMountDisconnected(e)) { + LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + + "2) 挂载点是否存在, 3) 如需重新挂载", path); + throw new IOException("FUSE 挂载断开: " + path, e); + } + + // 可重试错误: stale file handle, device busy + if (isStaleFileHandle(e) || isDeviceBusy(e)) { + if (attempt < maxAttempts - 1) { + long delay = calculateDelay(attempt); + LOG.warn("FUSE 错误 ({}) 路径: {}, {}ms 后重试 (第 {}/{} 次)", + e.getMessage(), path, delay, attempt + 1, maxAttempts); + try { + Thread.sleep(delay); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + throw new IOException("重试被中断", ie); + } + continue; + } + } + + // 不可重试错误或达到最大重试次数 + throw e; + } + } + + // 不应到达这里，但以防万一 + throw new IOException("FUSE 操作失败，已重试 " + maxAttempts + " 次: " + path, + lastException); + } + + @FunctionalInterface + interface SupplierWithIOException { + T get() throws IOException; + } } +``` +### FuseAwareFileIO 包装器 + +```java /** - * 检查是否为 Stale file handle 错误 + * FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 + * 委托给 LocalFileIO 执行实际文件操作。 */ -private boolean isStaleFileHandle(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Stale file handle"); +public class FuseAwareFileIO implements FileIO { + private final FileIO delegate; + private final FuseErrorHandler errorHandler; + + public FuseAwareFileIO(Path fusePath, CatalogContext context) { + this.delegate = FileIO.get(fusePath, context); // LocalFileIO + + Options options = context.options(); + this.errorHandler = new FuseErrorHandler( + options.getInteger(FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), + options.getLong(FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), + options.getLong(FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) + ); + } + + @Override + public SeekableInputStream newInputStream(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.newInputStream(path), path, "newInputStream"); + } + + @Override + public FileStatus getFileStatus(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.getFileStatus(path), path, "getFileStatus"); + } + + @Override + public boolean exists(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.exists(path), path, "exists"); + } + + // ... 其他方法类似包装 } +``` -/** - * 执行文件操作，处理 FUSE 特有错误 - */ -private T executeWithFuseErrorHandling( - SupplierWithIOException operation, - Path path, - String operationName) throws IOException { - - try { - return operation.get(); - } catch (IOException e) { - // FUSE 挂载断开 - 立即失败 - if (isFuseMountDisconnected(e)) { - LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + - "2) 挂载点是否存在, 3) 如需重新挂载", path); - throw new IOException("FUSE 挂载断开: " + path, e); - } - - // Stale file handle - 重试一次 - if (isStaleFileHandle(e)) { - LOG.warn("Stale file handle: {}, 重试一次...", path); - try { - return operation.get(); - } catch (IOException retryError) { - throw new IOException("Stale file handle (重试失败): " + path, retryError); +### RESTCatalog 集成 + +```java +private FileIO fileIOForData(Path path, Identifier identifier) { + // 1. 尝试解析 FUSE 本地路径 + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // 2. 如果需要，执行校验（参见校验章节） + if (validationMode != ValidationMode.NONE) { + ValidationResult result = validateFUSEPath(localPath, path, identifier); + if (!result.isValid()) { + handleValidationError(result, validationMode); + return createDefaultFileIO(path, identifier); + } } + + // 3. 返回带错误处理的 FuseAwareFileIO + return new FuseAwareFileIO(localPath, context); } - - // 其他错误 - 直接抛出（由现有机制处理） - throw e; } -} -@FunctionalInterface -interface SupplierWithIOException { - T get() throws IOException; + // 4. 回退到原有逻辑 + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); } ``` +### 重试行为示例 + +| 错误类型 | 是否重试 | 延迟模式（默认设置） | +|----------|----------|---------------------| +| `Transport endpoint is not connected` | ❌ 否 | 立即失败 | +| `Stale file handle` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | +| `Device or resource busy` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | +| 其他 IOException | ❌ 否 | 直接抛出 | + ### 日志规范 | 日志级别 | 场景 | 示例 | diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design.md index 3a3a322ebfaa..f38991d9dd21 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design.md @@ -41,6 +41,10 @@ All parameters are defined in `RESTCatalogOptions.java`: | `fuse.local-path.root` | String | (none) | The root local path for FUSE-mounted storage, e.g., `/mnt/fuse` | | `fuse.local-path.database` | Map | `{}` | Database-level local path mapping. Format: `db1:/local/path1,db2:/local/path2` | | `fuse.local-path.table` | Map | `{}` | Table-level local path mapping. Format: `db1.table1:/local/path1,db2.table2:/local/path2` | +| `fuse.local-path.validation-mode` | String | `strict` | Validation mode: `strict`, `warn`, or `none` | +| `fuse.local-path.retry.max-attempts` | Integer | `3` | Maximum retry attempts for FUSE-specific errors (e.g., stale file handle) | +| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | Initial retry delay in milliseconds | +| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | Maximum retry delay in milliseconds | ## Usage Example @@ -763,64 +767,198 @@ This error occurs when a file is deleted or modified by another process while we ### Implementation Example ```java +import org.apache.paimon.utils.RetryUtils; + /** - * Check if error is FUSE mount disconnection + * FUSE error handler with configurable retry and exponential backoff. */ -private boolean isFuseMountDisconnected(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Transport endpoint is not connected"); +public class FuseErrorHandler { + private final int maxAttempts; + private final long initialDelayMs; + private final long maxDelayMs; + + public FuseErrorHandler(int maxAttempts, long initialDelayMs, long maxDelayMs) { + this.maxAttempts = maxAttempts; + this.initialDelayMs = initialDelayMs; + this.maxDelayMs = maxDelayMs; + } + + /** + * Check if error is FUSE mount disconnection + */ + public boolean isFuseMountDisconnected(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Transport endpoint is not connected"); + } + + /** + * Check if error is stale file handle + */ + public boolean isStaleFileHandle(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Stale file handle"); + } + + /** + * Check if error is device or resource busy + */ + public boolean isDeviceBusy(IOException e) { + String message = e.getMessage(); + return message != null && + message.contains("Device or resource busy"); + } + + /** + * Calculate exponential backoff delay. + * Formula: min(initialDelay * 2^attempt, maxDelay) + */ + private long calculateDelay(int attempt) { + long delay = initialDelayMs * (1L << attempt); // 2^attempt + return Math.min(delay, maxDelayMs); + } + + /** + * Execute file operation with FUSE-specific error handling. + * Uses exponential backoff for retryable errors. + */ + public T executeWithFuseErrorHandling( + SupplierWithIOException operation, + Path path, + String operationName) throws IOException { + + IOException lastException = null; + + for (int attempt = 0; attempt < maxAttempts; attempt++) { + try { + return operation.get(); + } catch (IOException e) { + lastException = e; + + // FUSE mount disconnected - fail immediately, no retry + if (isFuseMountDisconnected(e)) { + LOG.error("FUSE mount disconnected for path: {}. " + + "Please check: 1) FUSE process is running, " + + "2) Mount point exists, 3) Remount if needed.", path); + throw new IOException("FUSE mount disconnected: " + path, e); + } + + // Retryable errors: stale file handle, device busy + if (isStaleFileHandle(e) || isDeviceBusy(e)) { + if (attempt < maxAttempts - 1) { + long delay = calculateDelay(attempt); + LOG.warn("FUSE error ({}) for {}, retrying in {}ms (attempt {}/{})", + e.getMessage(), path, delay, attempt + 1, maxAttempts); + try { + Thread.sleep(delay); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + throw new IOException("Retry interrupted", ie); + } + continue; + } + } + + // Non-retryable errors or max attempts reached + throw e; + } + } + + // Should not reach here, but just in case + throw new IOException("FUSE operation failed after " + maxAttempts + " attempts: " + path, + lastException); + } + + @FunctionalInterface + interface SupplierWithIOException { + T get() throws IOException; + } } +``` + +### FuseAwareFileIO Wrapper +```java /** - * Check if error is stale file handle + * FileIO wrapper that handles FUSE-specific errors with configurable retry. + * Delegates to LocalFileIO for actual file operations. */ -private boolean isStaleFileHandle(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Stale file handle"); +public class FuseAwareFileIO implements FileIO { + private final FileIO delegate; + private final FuseErrorHandler errorHandler; + + public FuseAwareFileIO(Path fusePath, CatalogContext context) { + this.delegate = FileIO.get(fusePath, context); // LocalFileIO + + Options options = context.options(); + this.errorHandler = new FuseErrorHandler( + options.getInteger(FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), + options.getLong(FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), + options.getLong(FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) + ); + } + + @Override + public SeekableInputStream newInputStream(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.newInputStream(path), path, "newInputStream"); + } + + @Override + public FileStatus getFileStatus(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.getFileStatus(path), path, "getFileStatus"); + } + + @Override + public boolean exists(Path path) throws IOException { + return errorHandler.executeWithFuseErrorHandling( + () -> delegate.exists(path), path, "exists"); + } + + // ... other methods similarly wrapped } +``` -/** - * Execute file operation with FUSE-specific error handling - */ -private T executeWithFuseErrorHandling( - SupplierWithIOException operation, - Path path, - String operationName) throws IOException { - - try { - return operation.get(); - } catch (IOException e) { - // FUSE mount disconnected - fail immediately - if (isFuseMountDisconnected(e)) { - LOG.error("FUSE mount disconnected for path: {}. " + - "Please check: 1) FUSE process is running, " + - "2) Mount point exists, 3) Remount if needed.", path); - throw new IOException("FUSE mount disconnected: " + path, e); - } - - // Stale file handle - retry once - if (isStaleFileHandle(e)) { - LOG.warn("Stale file handle for {}, retrying once...", path); - try { - return operation.get(); - } catch (IOException retryError) { - throw new IOException("Stale file handle (retry failed): " + path, retryError); +### RESTCatalog Integration + +```java +private FileIO fileIOForData(Path path, Identifier identifier) { + // 1. Try to resolve FUSE local path + if (fuseLocalPathEnabled) { + Path localPath = resolveFUSELocalPath(path, identifier); + if (localPath != null) { + // 2. Validate if needed (see validation section) + if (validationMode != ValidationMode.NONE) { + ValidationResult result = validateFUSEPath(localPath, path, identifier); + if (!result.isValid()) { + handleValidationError(result, validationMode); + return createDefaultFileIO(path, identifier); + } } + + // 3. Return FuseAwareFileIO with error handling + return new FuseAwareFileIO(localPath, context); } - - // Other errors - propagate as-is (handled by existing mechanisms) - throw e; } -} -@FunctionalInterface -interface SupplierWithIOException { - T get() throws IOException; + // 4. Fallback to original logic + return dataTokenEnabled + ? new RESTTokenFileIO(context, api, identifier, path) + : fileIOFromOptions(path); } ``` +### Retry Behavior Examples + +| Error Type | Retry? | Delay Pattern (default settings) | +|------------|--------|----------------------------------| +| `Transport endpoint is not connected` | ❌ No | Fail immediately | +| `Stale file handle` | ✅ Yes | 100ms → 200ms → 400ms → fail | +| `Device or resource busy` | ✅ Yes | 100ms → 200ms → 400ms → fail | +| Other IOException | ❌ No | Propagate immediately | + ### Logging Guidelines | Log Level | Scenario | Example | From 211abaf2e90f72c775ff0625ad45da3e097cfea8 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Thu, 19 Mar 2026 11:00:00 +0800 Subject: [PATCH 20/23] update design --- designs/fuse-local-path-design-cn.md | 1591 ++++++++--------- ...gn.md => fuse-local-path-design-cn.md.bak} | 576 +++--- 2 files changed, 1073 insertions(+), 1094 deletions(-) rename designs/{fuse-local-path-design.md => fuse-local-path-design-cn.md.bak} (60%) diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md index 889c5888ff77..be320e836b76 100644 --- a/designs/fuse-local-path-design-cn.md +++ b/designs/fuse-local-path-design-cn.md @@ -31,38 +31,38 @@ limitations under the License. 3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写 4. 保持与现有 RESTCatalog 行为的向后兼容性 -## 配置参数 +--- -所有参数定义在 `RESTCatalogOptions.java` 中： +## 【高】需求一：增加 FUSE 相关配置 + +### 配置参数 + +所有参数定义在 `pypaimon/common/options/config.py` 中的 `FuseOptions` 类： | 参数 | 类型 | 默认值 | 描述 | |-----|------|--------|------| | `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | | `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | -| `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | -| `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | -| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | -| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | -| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | - -## 使用示例 - -### SQL 配置（Flink/Spark） - -```sql -CREATE CATALOG paimon_rest_catalog WITH ( - 'type' = 'paimon', - 'metastore' = 'rest', - 'uri' = 'http://rest-server:8080', - 'token' = 'xxx', - - -- FUSE 本地路径配置 - 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/fuse/warehouse', - 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', - 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' -); +| `fuse.local-path.database` | String | (无) | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | +| `fuse.local-path.table` | String | (无) | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | + +### 使用示例 + +```python +from pypaimon import Catalog + +# 创建 REST Catalog 并启用 FUSE 本地路径 +catalog = Catalog.create({ + 'metastore': 'rest', + 'uri': 'http://rest-server:8080', + 'token': 'xxx', + + # FUSE 本地路径配置 + 'fuse.local-path.enabled': 'true', + 'fuse.local-path.root': '/mnt/fuse/warehouse', + 'fuse.local-path.database': 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', + 'fuse.local-path.table': 'db1.table1:/mnt/special/t1' +}) ``` ### 路径解析优先级 @@ -78,665 +78,537 @@ CREATE CATALOG paimon_rest_catalog WITH ( - 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` - 否则，使用 `fuse.local-path.root`（如 `/mnt/fuse/warehouse`） -## 实现方案 - -### RESTCatalog 修改 - -修改 `RESTCatalog.java` 中的 `fileIOForData` 方法： - -```java -private FileIO fileIOForData(Path path, Identifier identifier) { - // 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 使用本地文件 IO，无需 token - return FileIO.get(localPath, CatalogContext.create(new Options(), context.hadoopConf())); - } - } - - // 原有逻辑：data token 或 ResolvingFileIO - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} - -/** - * 解析 FUSE 本地路径。优先级：table > database > root。 - * @return 本地路径，如果不适用则返回 null - */ -private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { - String pathStr = originalPath.toString(); - - // 1. 检查 Table 级别映射 - Map tableMappings = context.options().get(FUSE_LOCAL_PATH_TABLE); - String tableKey = identifier.getDatabaseName() + "." + identifier.getTableName(); - if (tableMappings.containsKey(tableKey)) { - String localRoot = tableMappings.get(tableKey); - return convertToLocalPath(pathStr, localRoot); - } - - // 2. 检查 Database 级别映射 - Map dbMappings = context.options().get(FUSE_LOCAL_PATH_DATABASE); - if (dbMappings.containsKey(identifier.getDatabaseName())) { - String localRoot = dbMappings.get(identifier.getDatabaseName()); - return convertToLocalPath(pathStr, localRoot); - } - - // 3. 使用根路径映射 - String fuseRoot = context.options().get(FUSE_LOCAL_PATH_ROOT); - if (fuseRoot != null) { - return convertToLocalPath(pathStr, fuseRoot); - } - - return null; -} - -private Path convertToLocalPath(String originalPath, String localRoot) { - // 将远端存储路径转换为本地 FUSE 路径 - // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 - // 具体实现取决于路径结构 -} -``` - ### 行为矩阵 | 配置 | 路径匹配 | 行为 | |-----|---------|------| | `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写 | | `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | -| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | +| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 FileIO.get） | -## 优势 +### FuseOptions 配置类定义 -1. **性能提升**：本地文件系统访问通常比基于网络的远端存储访问更快 -2. **灵活性**：支持为不同的数据库/表配置不同的本地路径 -3. **向后兼容**：默认禁用，现有行为不变 +在 `pypaimon/common/options/config.py` 中添加： -## 安全校验机制 +```python +class FuseOptions: + """FUSE 本地路径配置选项。""" + + FUSE_LOCAL_PATH_ENABLED = ( + ConfigOptions.key("fuse.local-path.enabled") + .boolean_type() + .default_value(False) + .with_description("是否启用 FUSE 本地路径映射") + ) + + FUSE_LOCAL_PATH_ROOT = ( + ConfigOptions.key("fuse.local-path.root") + .string_type() + .no_default_value() + .with_description("FUSE 挂载的本地根路径，如 /mnt/fuse") + ) + + FUSE_LOCAL_PATH_DATABASE = ( + ConfigOptions.key("fuse.local-path.database") + .string_type() + .no_default_value() + .with_description( + "Database 级别的本地路径映射。格式：db1:/local/path1,db2:/local/path2" + ) + ) + + FUSE_LOCAL_PATH_TABLE = ( + ConfigOptions.key("fuse.local-path.table") + .string_type() + .no_default_value() + .with_description( + "Table 级别的本地路径映射。格式：db1.table1:/local/path1,db2.table2:/local/path2" + ) + ) +``` -### 问题场景 +### RESTCatalog 修改 -错误的 FUSE 本地路径配置可能导致严重的数据一致性问题： +修改 `pypaimon/catalog/rest/rest_catalog.py` 中的 `file_io_for_data` 方法： + +```python +from pypaimon.filesystem.local_file_io import LocalFileIO +from pypaimon.common.options.config import FuseOptions + +class RESTCatalog(Catalog): + def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): + # ... 原有初始化代码 ... + self.fuse_local_path_enabled = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) + + def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: + """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" + # 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO + if self.fuse_local_path_enabled: + local_path = self._resolve_fuse_local_path(table_path, identifier) + if local_path is not None: + # 使用本地文件 IO，无需 token + return LocalFileIO(local_path, self.context.options) + + # 原有逻辑：data token 或 FileIO.get + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) + + def _resolve_fuse_local_path(self, original_path: str, identifier: Identifier) -> Optional[str]: + """ + 解析 FUSE 本地路径。优先级：table > database > root。 + + Returns: + 本地路径，如果不适用则返回 None + """ + # 1. 检查 Table 级别映射 + table_mappings = self._parse_map_option( + self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_TABLE)) + table_key = f"{identifier.get_database_name()}.{identifier.get_table_name()}" + if table_key in table_mappings: + return self._convert_to_local_path(original_path, table_mappings[table_key]) + + # 2. 检查 Database 级别映射 + db_mappings = self._parse_map_option( + self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_DATABASE)) + if identifier.get_database_name() in db_mappings: + return self._convert_to_local_path( + original_path, db_mappings[identifier.get_database_name()]) + + # 3. 使用根路径映射 + fuse_root = self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_ROOT) + if fuse_root: + return self._convert_to_local_path(original_path, fuse_root) + + return None + + def _convert_to_local_path(self, original_path: str, local_root: str) -> str: + """将远端存储路径转换为本地 FUSE 路径。 + + 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 + """ + from urllib.parse import urlparse + + uri = urlparse(original_path) + if not uri.scheme: + # 已经是本地路径 + return original_path + + # 提取路径部分，移除开头的 scheme 和 netloc + path_part = uri.path + + # 确保路径不以 / 开头（local_root 已经是完整路径） + if path_part.startswith('/'): + path_part = path_part[1:] + + return f"{local_root.rstrip('/')}/{path_part}" + + def _parse_map_option(self, value: Optional[str]) -> Dict[str, str]: + """解析 Map 类型的配置项。 + + 格式：key1:value1,key2:value2 + """ + if not value: + return {} + + result = {} + for item in value.split(','): + item = item.strip() + if ':' in item: + key, val = item.split(':', 1) + result[key.strip()] = val.strip() + return result +``` -| 场景 | 描述 | 后果 | -|-----|------|------| -| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到远端存储，导致数据丢失 | -| **远端路径错误** | 本地路径指向了其他库表的远端存储路径 | 数据写入错误的表，导致数据污染 | +--- -### 校验模式配置 +## 【高】需求二：FUSE 安全校验机制 -新增配置参数控制校验行为： +为防止本地路径被误配置或篡改，设计两层校验机制。 -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`（严格）、`warn`（警告）、`none`（不校验） | +### 问题场景 -**校验模式说明**： +以下场景可能导致 FUSE 本地路径配置错误，进而引发数据安全问题： -| 模式 | 行为 | -|-----|------| -| `strict` | 启用校验，失败时抛出异常，阻止操作 | -| `warn` | 启用校验，失败时输出警告日志，但允许操作继续 | -| `none` | 不进行校验（不推荐，可能导致数据丢失或污染） | +| 场景 | 问题描述 | 潜在风险 | +|------|---------|---------| +| **路径配置错误** | 用户将 `fuse.local-path.root` 配置为错误的挂载点，如将 `/mnt/fuse-a` 误配置为 `/mnt/fuse-b` | 读写到错误的数据目录，导致数据混乱或丢失 | +| **多租户环境混淆** | 同一机器上挂载了多个租户的 FUSE 路径，用户配置了错误的租户路径 | 跨租户数据泄露或越权访问 | +| **FUSE 挂载点漂移** | FUSE 进程重启后，挂载点路径发生变化但配置未更新 | 访问到其他表的数据，破坏数据一致性 | +| **恶意路径注入** | 攻击者通过篡改配置文件，将本地路径指向敏感数据目录 | 敏感数据泄露或被恶意修改 | +| **表路径重用** | 删除表后重建同名表，但 FUSE 挂载点仍指向旧数据 | 读写到过期数据，业务逻辑错误 | +| **并发挂载冲突** | 多个 FUSE 实例挂载到同一目录的不同状态 | 数据版本不一致，读写冲突 | -### 校验流程 +### 校验流程图 ``` ┌─────────────────────────────────────────────────────────────┐ -│ 访问表（getTable） │ +│ FUSE 本地路径安全校验流程 │ └─────────────────────────────────────────────────────────────┘ │ ▼ -┌─────────────────────────────────────────────────────────────┐ -│ fuse.local-path.enabled == true ? │ -└─────────────────────────────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌───────────────────┐ ┌───────────────────┐ - │ 解析本地路径 │ │ 使用原有逻辑 │ - │ resolveFUSELocalPath│ │ (RESTTokenFileIO) │ - └───────────────────┘ └───────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ validation-mode != none ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌───────────────────┐ ┌───────────────────┐ - │ 校验本地路径存在 │ │ 跳过校验 │ - │ 与远端数据比对 │ │ 直接使用本地路径 │ - └───────────────────┘ └───────────────────┘ - │ - ▼ ┌───────────────────────────────────────┐ - │ 校验通过 ? │ + │ validation-mode == NONE ? │ └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 使用本地路径 │ │ validation-mode: │ - │ LocalFileIO │ │ - strict: 抛异常 │ - └─────────────┘ │ - warn: 警告+回退 │ - └─────────────────────┘ + │ + ┌─────────┴─────────┐ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ 跳过校验 │ │ 开始校验 │ + │ 直接使用 │ │ │ + └─────────────┘ └─────────────────────┘ + │ + ▼ + ┌─────────────────────────────────┐ + │ 第一次校验：.identifier 文件 │ + │ 比对远端和本地表标识 UUID │ + └─────────────────────────────────┘ + │ + ┌───────────┴───────────┐ + 成功 失败 + │ │ + ▼ ▼ + ┌─────────────────────┐ ┌───────────────────────┐ + │ 第二次校验：远端数据 │ │ 根据校验模式处理： │ + │ 比对文件大小和哈希 │ │ - strict: 抛异常 │ + │ │ │ - warn: 警告并回退 │ + └─────────────────────┘ └───────────────────────┘ + │ + ┌─────────┴─────────┐ + 成功 失败 + │ │ + ▼ ▼ + ┌─────────────┐ ┌───────────────────────┐ + │ 使用本地 IO │ │ 根据校验模式处理 │ + └─────────────┘ └───────────────────────┘ ``` -### .identifier 文件 +### 配置参数 -每个表目录下都包含一个 `.identifier` 文件用于快速校验： +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | -**文件位置**：`<表路径>/.identifier` +### 校验模式说明 -**文件格式**： -```json -{"uuid":"xxx-xxx-xxx-xxx"} -``` +| 模式 | 校验失败时行为 | 适用场景 | +|------|---------------|----------| +| `strict` | 抛出异常，阻止操作 | 生产环境，安全优先 | +| `warn` | 记录警告，回退到默认 FileIO | 测试环境，兼容性优先 | +| `none` | 不校验，直接使用 | 信任环境，性能优先 | -**用途**： -- 比对本地和远端路径的表 UUID -- 在昂贵的文件内容比对前进行快速校验 -- 创建表时自动生成 -- 仅需 UUID（database/table 名称可能因重命名而变化） - -### 安全校验实现 - -使用远端数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取远端存储文件，与本地文件比对。 - -**完整实现**： - -```java -/** - * RESTCatalog 中 fileIOForData 的完整实现 - * 结合 FUSE 本地路径校验与远端数据校验 - */ -private FileIO fileIOForData(Path path, Identifier identifier) { - // 如果 FUSE 本地路径启用，尝试使用本地路径 - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 根据校验模式执行校验 - ValidationMode mode = getValidationMode(); - - if (mode != ValidationMode.NONE) { - ValidationResult result = validateFUSEPath(localPath, path, identifier); - if (!result.isValid()) { - handleValidationError(result, mode); - // 校验失败，回退到原有逻辑 - return createDefaultFileIO(path, identifier); - } - } - - // 校验通过或跳过校验，使用本地 FileIO - return createLocalFileIO(localPath); - } - } - - // 原有逻辑：data token 或 ResolvingFileIO - return createDefaultFileIO(path, identifier); -} - -/** - * 校验 FUSE 本地路径 - */ -private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { - // 1. 创建 LocalFileIO 用于本地路径操作 - LocalFileIO localFileIO = LocalFileIO.create(); - - // 2. 检查本地路径是否存在 - if (!localFileIO.exists(localPath)) { - return ValidationResult.fail("本地路径不存在: " + localPath); - } - - // 3. 第一次校验：表标识文件 - ValidationResult identifierResult = validateByIdentifierFile(localFileIO, localPath, remotePath, identifier); - if (!identifierResult.isSuccess()) { - return identifierResult; - } - - // 4. 第二次校验：远端数据校验 - return validateByRemoteData(localFileIO, localPath, remotePath, identifier); -} - -/** - * 第一次校验：检查 .identifier 文件 - * 比对本地和远端的表 UUID 确保路径正确性 - */ -private ValidationResult validateByIdentifierFile( - LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { - try { - // 1. 获取远端存储 FileIO - FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - - // 2. 读取远端标识文件 - Path remoteIdentifierFile = new Path(remotePath, ".identifier"); - if (!remoteFileIO.exists(remoteIdentifierFile)) { - // 无标识文件，跳过此次校验 - LOG.debug("未找到表 {} 的 .identifier 文件，跳过标识校验", identifier); - return ValidationResult.success(); - } - - String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); - - // 3. 读取本地标识文件 - Path localIdentifierFile = new Path(localPath, ".identifier"); - if (!localFileIO.exists(localIdentifierFile)) { - return ValidationResult.fail( - "本地 .identifier 文件未找到: " + localIdentifierFile + - "。FUSE 路径可能未正确挂载。"); - } - - String localIdentifier = readIdentifierFile(localFileIO, localIdentifierFile); - - // 4. 比对标识符 - if (!remoteIdentifier.equals(localIdentifier)) { - return ValidationResult.fail(String.format( - "表标识不匹配！本地: %s，远端: %s。" + - "本地路径可能指向了其他表。", - localIdentifier, remoteIdentifier)); - } - - return ValidationResult.success(); - - } catch (Exception e) { - LOG.warn("标识文件校验失败: {}", identifier, e); - return ValidationResult.fail("标识文件校验失败: " + e.getMessage()); - } -} - -/** - * 读取 .identifier 文件内容 - * 格式：{"uuid":"xxx-xxx-xxx-xxx"} - */ -private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { - try (InputStream in = fileIO.newInputStream(identifierFile)) { - String json = IOUtils.readUTF8Fully(in); - JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); - return node.get("uuid").asText(); - } -} - -/** - * 第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性 - * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取远端存储文件 - */ -private ValidationResult validateByRemoteData( - LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { - try { - // 1. 获取远端存储 FileIO（使用现有逻辑，可访问远端存储） - FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - - // 2. 使用 SnapshotManager 获取最新 snapshot - SnapshotManager snapshotManager = new SnapshotManager(remoteFileIO, remotePath); - Snapshot latestSnapshot = snapshotManager.latestSnapshot(); - - Path checksumFile; - if (latestSnapshot != null) { - // 有 snapshot，使用 snapshot 文件校验 - checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); - } else { - // 无 snapshot（新表），使用 schema 文件校验 - SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); - Optional latestSchema = schemaManager.latest(); - if (!latestSchema.isPresent()) { - // 无 schema（如 format 表、object 表），跳过验证 - LOG.info("未找到表 {} 的 snapshot 或 schema，跳过验证", identifier); - return ValidationResult.success(); - } - checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); - } - - // 3. 读取远端文件内容并计算 hash - FileStatus remoteStatus = remoteFileIO.getFileStatus(checksumFile); - String remoteHash = computeFileHash(remoteFileIO, checksumFile); - - // 4. 构建本地文件路径并计算 hash - Path localChecksumFile = new Path(localPath, remotePath.toUri().getPath()); - - if (!localFileIO.exists(localChecksumFile)) { - return ValidationResult.fail( - "本地文件未找到: " + localChecksumFile + - "。FUSE 路径可能未正确挂载。"); - } - - long localSize = localFileIO.getFileSize(localChecksumFile); - String localHash = computeFileHash(localFileIO, localChecksumFile); - - // 5. 比对文件特征 - if (localSize != remoteStatus.getLen()) { - return ValidationResult.fail(String.format( - "文件大小不匹配！本地: %d 字节, 远端: %d 字节。", - localSize, remoteStatus.getLen())); - } - - if (!localHash.equalsIgnoreCase(remoteHash)) { - return ValidationResult.fail(String.format( - "文件内容哈希不匹配！本地: %s, 远端: %s。", - localHash, remoteHash)); - } - - return ValidationResult.success(); - - } catch (Exception e) { - LOG.warn("通过远端数据验证 FUSE 路径失败: {}", identifier, e); - return ValidationResult.fail("远端数据验证失败: " + e.getMessage()); - } -} - -/** - * 使用 FileIO 计算文件内容哈希 - */ -private String computeFileHash(FileIO fileIO, Path file) throws IOException { - MessageDigest md; - try { - md = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 算法不可用", e); - } - - try (InputStream is = fileIO.newInputStream(file)) { - byte[] buffer = new byte[4096]; - int bytesRead; - while ((bytesRead = is.read(buffer)) != -1) { - md.update(buffer, 0, bytesRead); - } - } - return Hex.encodeHexString(md.digest()); -} - -/** - * 处理校验错误 - */ -private void handleValidationError(ValidationResult result, ValidationMode mode) { - String errorMsg = "FUSE local path validation failed: " + result.getErrorMessage(); - - switch (mode) { - case STRICT: - throw new IllegalArgumentException(errorMsg); - case WARN: - LOG.warn(errorMsg + ". Falling back to default FileIO."); - break; - case NONE: - // 不会执行到这里 - break; - } -} - -/** - * 使用现有 context 创建本地 FileIO - */ -private FileIO createLocalFileIO(Path localPath) { - return FileIO.get(localPath, context); -} - -/** - * 创建默认 FileIO（原有逻辑） - */ -private FileIO createDefaultFileIO(Path path, Identifier identifier) { - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} - -// ========== 辅助类 ========== - -enum ValidationMode { - STRICT, // 严格模式：校验失败抛异常 - WARN, // 警告模式：校验失败只警告，回退到默认逻辑 - NONE // 不校验 -} - -class ValidationResult { - private final boolean valid; - private final String errorMessage; - - private ValidationResult(boolean valid, String errorMessage) { - this.valid = valid; - this.errorMessage = errorMessage; - } - - static ValidationResult success() { - return new ValidationResult(true, null); - } - - static ValidationResult fail(String errorMessage) { - return new ValidationResult(false, errorMessage); - } - - boolean isValid() { return valid; } - String getErrorMessage() { return errorMessage; } -} -``` +### FuseOptions 配置扩展 -**方案优势**： +在 `pypaimon/common/options/config.py` 中添加： -| 优势 | 说明 | -|------|------| -| **无需扩展 API** | 使用现有 FileIO 和 SnapshotManager/SchemaManager | -| **使用 LATEST snapshot** | 通过 `SnapshotManager.latestSnapshot()` 直接获取，无需遍历 | -| **新表支持** | 无 snapshot 时自动回退到 schema 文件校验 | -| **准确性最高** | 直接验证数据一致性，确保路径正确 | -| **优雅降级** | 校验失败可回退到默认 FileIO | +```python +class FuseOptions: + # ... 原有配置项 ... + + FUSE_LOCAL_PATH_VALIDATION_MODE = ( + ConfigOptions.key("fuse.local-path.validation-mode") + .string_type() + .default_value("strict") + .with_description("校验模式：strict、warn 或 none") + ) +``` -**校验文件选择逻辑**： +### Python 安全校验实现 -| 场景 | 校验文件 | -|------|----------| -| 有 snapshot | 使用 `SnapshotManager.latestSnapshot()` 获取的最新 snapshot 文件 | -| 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | -| 无 schema（如 format 表、object 表）| 跳过校验 | +在 `pypaimon/catalog/rest/rest_catalog.py` 中添加安全校验逻辑： -**两步校验**： +```python +import hashlib +import logging +from enum import Enum +from typing import Optional, Tuple +from pathlib import Path -| 步骤 | 校验方式 | 描述 | -|------|----------|------| -| 1 | `.identifier` 文件 | 比对本地和远端的表 UUID | -| 2 | 远端数据校验 | 比对 snapshot/schema 文件内容 | +class ValidationMode(Enum): + """校验模式枚举。""" + STRICT = "strict" # 严格模式：校验失败抛异常 + WARN = "warn" # 警告模式：校验失败只警告，回退到默认逻辑 + NONE = "none" # 不校验 -**完整校验流程**： +class ValidationResult: + """校验结果类。""" + + def __init__(self, valid: bool, error_message: Optional[str] = None): + self.valid = valid + self.error_message = error_message + + @staticmethod + def success() -> 'ValidationResult': + return ValidationResult(True) + + @staticmethod + def fail(error_message: str) -> 'ValidationResult': + return ValidationResult(False, error_message) + + +class RESTCatalog(Catalog): + # ... 原有代码 ... + + def _get_validation_mode(self) -> ValidationMode: + """获取校验模式。""" + mode_str = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE, "strict") + return ValidationMode(mode_str.lower()) + + def _validate_fuse_path(self, local_path: str, remote_path: str, + identifier: Identifier) -> ValidationResult: + """校验 FUSE 本地路径。""" + local_file_io = LocalFileIO(local_path, self.context.options) + logger = logging.getLogger(__name__) + + # 1. 检查本地路径是否存在 + if not local_file_io.exists(local_path): + return ValidationResult.fail(f"本地路径不存在: {local_path}") + + # 2. 第一次校验：表标识文件 + identifier_result = self._validate_by_identifier_file( + local_file_io, local_path, remote_path, identifier) + if not identifier_result.valid: + return identifier_result + + # 3. 第二次校验：远端数据校验 + return self._validate_by_remote_data( + local_file_io, local_path, remote_path, identifier) + + def _validate_by_identifier_file(self, local_file_io: LocalFileIO, + local_path: str, remote_path: str, + identifier: Identifier) -> ValidationResult: + """第一次校验：检查 .identifier 文件。""" + logger = logging.getLogger(__name__) + + try: + # 获取远端存储 FileIO + remote_file_io = self._create_default_file_io(remote_path, identifier) + + # 读取远端标识文件 + remote_identifier_file = f"{remote_path.rstrip('/')}/.identifier" + if not remote_file_io.exists(remote_identifier_file): + logger.debug(f"未找到表 {identifier} 的 .identifier 文件，跳过标识校验") + return ValidationResult.success() + + remote_identifier = self._read_identifier_file(remote_file_io, remote_identifier_file) + + # 读取本地标识文件 + local_identifier_file = f"{local_path.rstrip('/')}/.identifier" + if not local_file_io.exists(local_identifier_file): + return ValidationResult.fail( + f"本地 .identifier 文件未找到: {local_identifier_file}。" + "FUSE 路径可能未正确挂载。") + + local_identifier = self._read_identifier_file(local_file_io, local_identifier_file) + + # 比对标识符 + if remote_identifier != local_identifier: + return ValidationResult.fail( + f"表标识不匹配！本地: {local_identifier}，远端: {remote_identifier}。" + "本地路径可能指向了其他表。") + + return ValidationResult.success() + + except Exception as e: + logger.warning(f"标识文件校验失败: {identifier}", exc_info=True) + return ValidationResult.fail(f"标识文件校验失败: {str(e)}") + + def _read_identifier_file(self, file_io: FileIO, identifier_file: str) -> str: + """读取 .identifier 文件内容。""" + import json + content = file_io.read_file_utf8(identifier_file) + data = json.loads(content) + return data.get("uuid", "") + + def _validate_by_remote_data(self, local_file_io: LocalFileIO, + local_path: str, remote_path: str, + identifier: Identifier) -> ValidationResult: + """第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性。""" + logger = logging.getLogger(__name__) + + try: + # 获取远端存储 FileIO + remote_file_io = self._create_default_file_io(remote_path, identifier) + + # 使用 SnapshotManager 获取最新 snapshot + from pypaimon.snapshot.snapshot_manager import SnapshotManager + snapshot_manager = SnapshotManager(remote_file_io, remote_path) + latest_snapshot = snapshot_manager.latest_snapshot() + + if latest_snapshot is not None: + checksum_file = snapshot_manager.snapshot_path(latest_snapshot.id()) + else: + # 无 snapshot（新表），使用 schema 文件校验 + from pypaimon.schema.schema_manager import SchemaManager + schema_manager = SchemaManager(remote_file_io, remote_path) + latest_schema = schema_manager.latest() + + if latest_schema is None: + logger.info(f"未找到表 {identifier} 的 snapshot 或 schema，跳过验证") + return ValidationResult.success() + + checksum_file = schema_manager.to_schema_path(latest_schema.id()) + + # 读取远端文件信息 + remote_status = remote_file_io.get_file_status(checksum_file) + remote_hash = self._compute_file_hash(remote_file_io, checksum_file) + + # 构建本地文件路径 + from urllib.parse import urlparse + uri = urlparse(checksum_file) + path_part = uri.path.lstrip('/') + local_checksum_file = f"{local_path.rstrip('/')}/{path_part}" + + if not local_file_io.exists(local_checksum_file): + return ValidationResult.fail( + f"本地文件未找到: {local_checksum_file}。" + "FUSE 路径可能未正确挂载。") + + local_size = local_file_io.get_file_size(local_checksum_file) + local_hash = self._compute_file_hash(local_file_io, local_checksum_file) + + # 比对文件特征 + if local_size != remote_status.size: + return ValidationResult.fail( + f"文件大小不匹配！本地: {local_size} 字节, 远端: {remote_status.size} 字节。") + + if local_hash.lower() != remote_hash.lower(): + return ValidationResult.fail( + f"文件内容哈希不匹配！本地: {local_hash}, 远端: {remote_hash}。") + + return ValidationResult.success() + + except Exception as e: + logger.warning(f"通过远端数据验证 FUSE 路径失败: {identifier}", exc_info=True) + return ValidationResult.fail(f"远端数据验证失败: {str(e)}") + + def _compute_file_hash(self, file_io: FileIO, file_path: str) -> str: + """计算文件内容的 MD5 哈希。""" + md5 = hashlib.md5() + with file_io.new_input_stream(file_path) as input_stream: + while True: + data = input_stream.read(4096) + if not data: + break + md5.update(data) + return md5.hexdigest() + + def _handle_validation_error(self, result: ValidationResult, mode: ValidationMode): + """处理校验错误。""" + error_msg = f"FUSE local path validation failed: {result.error_message}" + logger = logging.getLogger(__name__) + + if mode == ValidationMode.STRICT: + raise ValueError(error_msg) + elif mode == ValidationMode.WARN: + logger.warning(f"{error_msg}. Falling back to default FileIO.") + + def _create_default_file_io(self, path: str, identifier: Identifier) -> FileIO: + """创建默认 FileIO（原有逻辑）。""" + return RESTTokenFileIO(identifier, path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(path) ``` -┌─────────────────────────────────────────────────────────────┐ -│ 校验流程 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 第一步：.identifier 校验 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 远端存在 .identifier ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 比对 UUID │ │ 跳过第一步，进入第二步 │ -│ 本地 vs 远端 │ │ （远端数据校验） │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ UUID 匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 失败：表标识不匹配 - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 第二步：远端数据校验 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 1. 获取远端存储 FileIO（RESTTokenFileIO 或 ResolvingFileIO）│ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 2. 通过 SnapshotManager 获取最新 snapshot │ -│ snapshotManager.latestSnapshot() │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Snapshot 存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 使用 snapshot 文件 │ │ 通过 SchemaManager 获取最新 schema │ -│ 进行校验 │ │ schemaManager.latest() │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Schema 存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 跳过校验（format/object 表） - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 3. 获取远端文件元数据（大小） │ -│ 计算远端文件 hash │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 4. 读取本地对应文件 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 本地文件存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败（路径错误或未挂载） - │ - ▼ - ┌───────────────────────────────────────┐ - │ 文件大小匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败 - │ - ▼ - ┌───────────────────────────────────────┐ - │ 文件内容 hash 匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败（路径指向错误表） - │ - ▼ - ┌─────────────┐ - │ 校验通过 │ - │ 可安全使用 │ - └─────────────┘ -``` - -### 使用示例（启用安全校验） - -```sql -CREATE CATALOG paimon_rest_catalog WITH ( - 'type' = 'paimon', - 'metastore' = 'rest', - 'uri' = 'http://rest-server:8080', - 'token' = 'xxx', - - -- FUSE 本地路径配置 - 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/fuse/warehouse', - -- 安全校验配置（可选，默认 strict） - 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none -); +### RESTCatalog 集成校验 + +在 `file_io_for_data` 方法中集成校验逻辑： + +```python +class RESTCatalog(Catalog): + def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): + # ... 原有初始化代码 ... + self.fuse_local_path_enabled = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) + self._validation_mode_cache: Optional[ValidationMode] = None + + def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: + """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" + # 1. 尝试解析 FUSE 本地路径 + if self.fuse_local_path_enabled: + local_path = self._resolve_fuse_local_path(table_path, identifier) + if local_path is not None: + # 2. 如果需要，执行校验 + validation_mode = self._get_validation_mode() + if validation_mode != ValidationMode.NONE: + result = self._validate_fuse_path(local_path, table_path, identifier) + if not result.valid: + self._handle_validation_error(result, validation_mode) + return self._create_default_file_io(table_path, identifier) + + # 3. 返回本地 FileIO + return LocalFileIO(local_path, self.context.options) + + # 4. 回退到原有逻辑 + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) ``` -## 限制 +### 校验优势 -1. FUSE 挂载必须正确配置且可访问 -2. 本地路径必须与远端存储路径具有相同的目录结构 -3. 写操作需要本地 FUSE 挂载点具有适当的权限 -4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） - -## FUSE 错误处理 - -本节仅涵盖 FUSE 特有的错误。其他错误（网络、REST API、权限）已由 Paimon 现有机制处理。 +| 优势 | 说明 | +|------|------| +| 双重校验 | 先校验表标识，再校验数据一致性 | +| 防止路径混淆 | 通过 UUID 确保本地路径指向正确的表 | +| 灵活模式 | strict/warn/none 三种模式适应不同场景 | +| 性能优化 | 仅在启用时执行校验，不影响正常路径 | -### FUSE 特有错误 +--- -| 错误类型 | 场景 | 原因 | 处理策略 | -|----------|------|------|----------| -| `Transport endpoint is not connected` | 本地文件读写 | FUSE 挂载断开或崩溃 | 立即失败，记录错误并提示检查挂载状态 | -| `Stale file handle` | 文件操作 | 文件被其他进程删除/修改 | 重试一次（重新打开文件） | -| `Device or resource busy` | 删除/重命名操作 | 文件仍被其他进程占用 | 退避重试 | -| `Input/output error` | 任意文件操作 | FUSE 后端故障（远端存储问题） | 失败并给出明确错误信息 | -| `No such file or directory`（意外） | 文件操作 | FUSE 挂载点未就绪 | 检查挂载状态，失败 | +## 【低】需求三：FUSE 错误处理 -### 错误处理策略 +FUSE 挂载可能出现特有错误，需要特殊处理以保证系统稳定性。 -#### FUSE 挂载断开（最关键） +### 常见 FUSE 错误 -最关键的 FUSE 特有错误是挂载断开（`Transport endpoint is not connected`）。该错误表示： -- FUSE 进程崩溃 -- 网络问题导致 FUSE 与远端存储断开连接 -- FUSE 挂载被手动卸载 +| 错误类型 | 错误信息 | 原因 | 处理策略 | +|---------|---------|------|---------| +| 挂载断开 | `Transport endpoint is not connected` | FUSE 进程崩溃或挂载失效 | 立即失败，不重试 | +| 过期文件句柄 | `Stale file handle` | 文件被其他进程修改或删除 | 指数退避重试 | +| 设备忙 | `Device or resource busy` | 资源竞争 | 指数退避重试 | -**处理方式**：立即失败并给出明确错误信息。不要重试，因为必须先恢复挂载。 +### 错误处理流程图 ``` ┌─────────────────────────────────────────────────────────────┐ -│ FUSE 挂载断开处理 │ +│ FUSE 错误处理流程 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ IOException: "Transport endpoint is │ - │ not connected" ? │ + │ 执行文件操作 │ └───────────────────────────────────────┘ │ - Yes - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ LOG.error("FUSE 挂载断开，路径: {}。请检查: │ -│ 1. FUSE 进程是否运行 │ -│ 2. 挂载点是否存在: ls -la /mnt/fuse/... │ -│ 3. 如需重新挂载: fusermount -u /mnt/fuse && ...") │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 抛出 IOException 并附带明确信息 │ - └───────────────────────────────────────┘ -``` - -#### Stale File Handle（过期文件句柄） - -当文件在我们持有打开句柄时被其他进程删除或修改，会触发此错误。 - -**处理方式**：重试一次，重新打开文件。 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Stale File Handle 处理 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ + ┌─────────┴─────────┐ + 成功 失败 + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────┐ + │ 返回结果 │ │ 检查错误类型 │ + └─────────────┘ └─────────────────────┘ + │ + ┌───────┴───────┐ + │ │ + ▼ ▼ + ┌─────────────────────────────┐ ┌─────────────────────────────┐ + │ FUSE 挂载断开？ │ │ 其他错误 │ + │ "Transport endpoint is not │ │ │ + │ connected" │ │ │ + └─────────────────────────────┘ └─────────────────────────────┘ + │ │ + Yes │ + │ │ + ▼ ▼ + ┌─────────────────────────────┐ ┌─────────────────────────────┐ + │ LOG.error("FUSE 挂载断开") │ │ 直接抛出异常 │ + │ 抛出异常，不重试 │ └─────────────────────────────┘ + └─────────────────────────────┘ + ┌───────────────────────────────────────┐ - │ IOException: "Stale file handle" ? │ + │ Stale file handle 或 Device busy ? │ └───────────────────────────────────────┘ │ Yes @@ -756,192 +628,277 @@ CREATE CATALOG paimon_rest_catalog WITH ( │ │ ▼ ▼ ┌─────────────┐ ┌─────────────────────┐ - │ 继续操作 │ │ 抛出 IOException │ + │ 继续操作 │ │ 抛出 OSError │ │ │ │ 并附带详细信息 │ └─────────────┘ └─────────────────────┘ ``` -### 实现示例 - -```java -/** - * FUSE 错误处理器，支持可配置的重试和指数退避。 - */ -public class FuseErrorHandler { - private final int maxAttempts; - private final long initialDelayMs; - private final long maxDelayMs; - - public FuseErrorHandler(int maxAttempts, long initialDelayMs, long maxDelayMs) { - this.maxAttempts = maxAttempts; - this.initialDelayMs = initialDelayMs; - this.maxDelayMs = maxDelayMs; - } - - /** - * 检查是否为 FUSE 挂载断开错误 - */ - public boolean isFuseMountDisconnected(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Transport endpoint is not connected"); - } - - /** - * 检查是否为 Stale file handle 错误 - */ - public boolean isStaleFileHandle(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Stale file handle"); - } - - /** - * 检查是否为 Device or resource busy 错误 - */ - public boolean isDeviceBusy(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Device or resource busy"); - } - - /** - * 计算指数退避延迟。 - * 公式: min(initialDelay * 2^attempt, maxDelay) - */ - private long calculateDelay(int attempt) { - long delay = initialDelayMs * (1L << attempt); // 2^attempt - return Math.min(delay, maxDelayMs); - } - - /** - * 执行文件操作，处理 FUSE 特有错误。 - * 对可重试错误使用指数退避策略。 - */ - public T executeWithFuseErrorHandling( - SupplierWithIOException operation, - Path path, - String operationName) throws IOException { - - IOException lastException = null; - - for (int attempt = 0; attempt < maxAttempts; attempt++) { - try { - return operation.get(); - } catch (IOException e) { - lastException = e; - - // FUSE 挂载断开 - 立即失败，不重试 - if (isFuseMountDisconnected(e)) { - LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + - "2) 挂载点是否存在, 3) 如需重新挂载", path); - throw new IOException("FUSE 挂载断开: " + path, e); - } - - // 可重试错误: stale file handle, device busy - if (isStaleFileHandle(e) || isDeviceBusy(e)) { - if (attempt < maxAttempts - 1) { - long delay = calculateDelay(attempt); - LOG.warn("FUSE 错误 ({}) 路径: {}, {}ms 后重试 (第 {}/{} 次)", - e.getMessage(), path, delay, attempt + 1, maxAttempts); - try { - Thread.sleep(delay); - } catch (InterruptedException ie) { - Thread.currentThread().interrupt(); - throw new IOException("重试被中断", ie); - } - continue; - } - } - - // 不可重试错误或达到最大重试次数 - throw e; - } - } - - // 不应到达这里，但以防万一 - throw new IOException("FUSE 操作失败，已重试 " + maxAttempts + " 次: " + path, - lastException); - } - - @FunctionalInterface - interface SupplierWithIOException { - T get() throws IOException; - } -} +### 配置参数 + +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | +| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | +| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | + +### FuseOptions 配置扩展 + +在 `pypaimon/common/options/config.py` 中添加： + +```python +class FuseOptions: + # ... 原有配置项 ... + + FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS = ( + ConfigOptions.key("fuse.local-path.retry.max-attempts") + .int_type() + .default_value(3) + .with_description("FUSE 特有错误的最大重试次数（如 stale file handle）") + ) + + FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS = ( + ConfigOptions.key("fuse.local-path.retry.initial-delay-ms") + .int_type() + .default_value(100) + .with_description("初始重试延迟（毫秒）") + ) + + FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS = ( + ConfigOptions.key("fuse.local-path.retry.max-delay-ms") + .int_type() + .default_value(5000) + .with_description("最大重试延迟（毫秒）") + ) ``` -### FuseAwareFileIO 包装器 - -```java -/** - * FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 - * 委托给 LocalFileIO 执行实际文件操作。 - */ -public class FuseAwareFileIO implements FileIO { - private final FileIO delegate; - private final FuseErrorHandler errorHandler; - - public FuseAwareFileIO(Path fusePath, CatalogContext context) { - this.delegate = FileIO.get(fusePath, context); // LocalFileIO - - Options options = context.options(); - this.errorHandler = new FuseErrorHandler( - options.getInteger(FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), - options.getLong(FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), - options.getLong(FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) - ); - } - - @Override - public SeekableInputStream newInputStream(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.newInputStream(path), path, "newInputStream"); - } - - @Override - public FileStatus getFileStatus(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.getFileStatus(path), path, "getFileStatus"); - } - - @Override - public boolean exists(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.exists(path), path, "exists"); - } - - // ... 其他方法类似包装 -} +### Python FUSE 错误处理实现 + +在 `pypaimon/filesystem/fuse_aware_file_io.py` 中创建 FUSE 错误处理器： + +```python +""" +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" +import logging +import time +from typing import Callable, TypeVar, Optional + +from pypaimon.common.file_io import FileIO +from pypaimon.filesystem.local_file_io import LocalFileIO +from pypaimon.common.options import Options +from pypaimon.common.options.config import FuseOptions + + +T = TypeVar('T') +logger = logging.getLogger(__name__) + + +class FuseErrorHandler: + """FUSE 错误处理器，支持可配置的重试和指数退避。""" + + def __init__(self, max_attempts: int = 3, initial_delay_ms: int = 100, + max_delay_ms: int = 5000): + self.max_attempts = max_attempts + self.initial_delay_ms = initial_delay_ms + self.max_delay_ms = max_delay_ms + + def is_fuse_mount_disconnected(self, error: OSError) -> bool: + """检查是否为 FUSE 挂载断开错误。""" + msg = str(error) + return "Transport endpoint is not connected" in msg + + def is_stale_file_handle(self, error: OSError) -> bool: + """检查是否为 Stale file handle 错误。""" + msg = str(error) + return "Stale file handle" in msg + + def is_device_busy(self, error: OSError) -> bool: + """检查是否为 Device or resource busy 错误。""" + msg = str(error) + return "Device or resource busy" in msg + + def calculate_delay(self, attempt: int) -> float: + """计算指数退避延迟（秒）。""" + delay = self.initial_delay_ms * (2 ** attempt) + return min(delay, self.max_delay_ms) / 1000.0 # 转换为秒 + + def execute_with_fuse_error_handling( + self, + operation: Callable[[], T], + path: str, + operation_name: str + ) -> T: + """执行文件操作，处理 FUSE 特有错误。""" + last_exception: Optional[Exception] = None + + for attempt in range(self.max_attempts): + try: + return operation() + except OSError as e: + last_exception = e + + # FUSE 挂载断开 - 立即失败，不重试 + if self.is_fuse_mount_disconnected(e): + logger.error( + f"FUSE 挂载断开，路径: {path}。请检查: " + "1) FUSE 进程是否运行, 2) 挂载点是否存在, 3) 如需重新挂载" + ) + raise OSError(f"FUSE 挂载断开: {path}") from e + + # 可重试错误: stale file handle, device busy + if self.is_stale_file_handle(e) or self.is_device_busy(e): + if attempt < self.max_attempts - 1: + delay = self.calculate_delay(attempt) + logger.warning( + f"FUSE 错误 ({e}) 路径: {path}, " + f"{delay * 1000:.0f}ms 后重试 (第 {attempt + 1}/{self.max_attempts} 次)" + ) + time.sleep(delay) + continue + + # 不可重试错误或达到最大重试次数 + raise + + # 不应到达这里，但以防万一 + raise OSError( + f"FUSE 操作失败，已重试 {self.max_attempts} 次: {path}" + ) from last_exception + + +class FuseAwareFileIO(FileIO): + """FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 + + 委托给 LocalFileIO 执行实际文件操作。 + """ + + def __init__(self, fuse_path: str, catalog_options: Optional[Options] = None): + self.delegate = LocalFileIO(fuse_path, catalog_options) + + options = catalog_options or Options({}) + self.error_handler = FuseErrorHandler( + max_attempts=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), + initial_delay_ms=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), + max_delay_ms=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) + ) + + def new_input_stream(self, path: str): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.new_input_stream(path), path, "new_input_stream" + ) + + def new_output_stream(self, path: str): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.new_output_stream(path), path, "new_output_stream" + ) + + def get_file_status(self, path: str): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.get_file_status(path), path, "get_file_status" + ) + + def list_status(self, path: str): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.list_status(path), path, "list_status" + ) + + def exists(self, path: str) -> bool: + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.exists(path), path, "exists" + ) + + def delete(self, path: str, recursive: bool = False) -> bool: + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.delete(path, recursive), path, "delete" + ) + + def mkdirs(self, path: str) -> bool: + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.mkdirs(path), path, "mkdirs" + ) + + def rename(self, src: str, dst: str) -> bool: + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.rename(src, dst), src, "rename" + ) + + def to_filesystem_path(self, path: str) -> str: + return self.delegate.to_filesystem_path(path) + + def write_parquet(self, path: str, data, compression: str = 'zstd', + zstd_level: int = 1, **kwargs): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.write_parquet(path, data, compression, zstd_level, **kwargs), + path, "write_parquet" + ) + + def write_orc(self, path: str, data, compression: str = 'zstd', + zstd_level: int = 1, **kwargs): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.write_orc(path, data, compression, zstd_level, **kwargs), + path, "write_orc" + ) + + def write_avro(self, path: str, data, avro_schema=None, + compression: str = 'zstd', zstd_level: int = 1, **kwargs): + return self.error_handler.execute_with_fuse_error_handling( + lambda: self.delegate.write_avro(path, data, avro_schema, compression, zstd_level, **kwargs), + path, "write_avro" + ) + + @property + def filesystem(self): + return self.delegate.filesystem + + @property + def uri_reader_factory(self): + return self.delegate.uri_reader_factory + + def close(self): + self.delegate.close() ``` -### RESTCatalog 集成 - -```java -private FileIO fileIOForData(Path path, Identifier identifier) { - // 1. 尝试解析 FUSE 本地路径 - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 2. 如果需要，执行校验（参见校验章节） - if (validationMode != ValidationMode.NONE) { - ValidationResult result = validateFUSEPath(localPath, path, identifier); - if (!result.isValid()) { - handleValidationError(result, validationMode); - return createDefaultFileIO(path, identifier); - } - } - - // 3. 返回带错误处理的 FuseAwareFileIO - return new FuseAwareFileIO(localPath, context); - } - } - - // 4. 回退到原有逻辑 - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} +### RESTCatalog 集成 FuseAwareFileIO + +在 `pypaimon/catalog/rest/rest_catalog.py` 中集成： + +```python +from pypaimon.filesystem.fuse_aware_file_io import FuseAwareFileIO + +class RESTCatalog(Catalog): + def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: + """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" + # 1. 尝试解析 FUSE 本地路径 + if self.fuse_local_path_enabled: + local_path = self._resolve_fuse_local_path(table_path, identifier) + if local_path is not None: + # 2. 如果需要，执行校验 + validation_mode = self._get_validation_mode() + if validation_mode != ValidationMode.NONE: + result = self._validate_fuse_path(local_path, table_path, identifier) + if not result.valid: + self._handle_validation_error(result, validation_mode) + return self._create_default_file_io(table_path, identifier) + + # 3. 返回带错误处理的 FuseAwareFileIO + return FuseAwareFileIO(local_path, self.context.options) + + # 4. 回退到原有逻辑 + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) ``` ### 重试行为示例 @@ -951,7 +908,7 @@ private FileIO fileIOForData(Path path, Identifier identifier) { | `Transport endpoint is not connected` | ❌ 否 | 立即失败 | | `Stale file handle` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | | `Device or resource busy` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | -| 其他 IOException | ❌ 否 | 直接抛出 | +| 其他 OSError | ❌ 否 | 直接抛出 | ### 日志规范 @@ -968,3 +925,31 @@ private FileIO fileIOForData(Path path, Identifier identifier) { 3. **自动重启**：考虑使用 systemd 或 supervisor 在崩溃时自动重启 FUSE 4. **日志分析**：查看 `dmesg` 或 FUSE 日志进行根因分析 +--- + +## 文件结构 + +新增/修改的文件： + +``` +paimon-python/ +├── pypaimon/ +│ ├── catalog/ +│ │ └── rest/ +│ │ └── rest_catalog.py # 修改：添加 FUSE 支持 +│ ├── common/ +│ │ └── options/ +│ │ └── config.py # 修改：添加 FuseOptions +│ └── filesystem/ +│ └── fuse_aware_file_io.py # 新增：FUSE 错误处理 FileIO +``` + +## 总结 + +本设计为 PyPaimon 的 RESTCatalog 提供 FUSE 本地路径支持，主要特性： + +1. **分层路径映射**：支持 Catalog、Database、Table 三级路径配置 +2. **安全校验**：双重校验机制（表标识 + 数据一致性）防止路径配置错误 +3. **错误处理**：针对 FUSE 特有错误（stale file handle 等）的重试机制 +4. **向后兼容**：默认禁用，不影响现有功能 +5. **灵活配置**：三种校验模式（strict/warn/none）适应不同场景 diff --git a/designs/fuse-local-path-design.md b/designs/fuse-local-path-design-cn.md.bak similarity index 60% rename from designs/fuse-local-path-design.md rename to designs/fuse-local-path-design-cn.md.bak index f38991d9dd21..889c5888ff77 100644 --- a/designs/fuse-local-path-design.md +++ b/designs/fuse-local-path-design-cn.md.bak @@ -16,39 +16,39 @@ See the License for the specific language governing permissions and limitations under the License. --> -# FUSE Local Path Configuration for RESTCatalog +# RESTCatalog FUSE 本地路径配置设计 -## Background +## 背景 -When using Paimon RESTCatalog with remote object storage (e.g., OSS, S3, HDFS), data access typically goes through remote storage SDKs. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data through local file system paths directly, achieving better performance. +在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 -This design introduces configuration parameters to support FUSE-mounted remote storage paths, allowing users to specify local path mappings at catalog, database, and table levels. +本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 -## Goals +## 目标 -1. Enable local file system access for FUSE-mounted remote storage paths -2. Support hierarchical path mapping: catalog root > database > table -3. Use local FileIO for data read/write when FUSE local path is applicable -4. Maintain backward compatibility with existing RESTCatalog behavior +1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 +2. 支持分层路径映射：Catalog 根路径 > Database > Table +3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写 +4. 保持与现有 RESTCatalog 行为的向后兼容性 -## Configuration Parameters +## 配置参数 -All parameters are defined in `RESTCatalogOptions.java`: +所有参数定义在 `RESTCatalogOptions.java` 中： -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `fuse.local-path.enabled` | Boolean | `false` | Whether to enable FUSE local path mapping for remote storage paths | -| `fuse.local-path.root` | String | (none) | The root local path for FUSE-mounted storage, e.g., `/mnt/fuse` | -| `fuse.local-path.database` | Map | `{}` | Database-level local path mapping. Format: `db1:/local/path1,db2:/local/path2` | -| `fuse.local-path.table` | Map | `{}` | Table-level local path mapping. Format: `db1.table1:/local/path1,db2.table2:/local/path2` | -| `fuse.local-path.validation-mode` | String | `strict` | Validation mode: `strict`, `warn`, or `none` | -| `fuse.local-path.retry.max-attempts` | Integer | `3` | Maximum retry attempts for FUSE-specific errors (e.g., stale file handle) | -| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | Initial retry delay in milliseconds | -| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | Maximum retry delay in milliseconds | +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | +| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | +| `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | +| `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | +| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | +| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | +| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | -## Usage Example +## 使用示例 -### SQL Configuration (Flink/Spark) +### SQL 配置（Flink/Spark） ```sql CREATE CATALOG paimon_rest_catalog WITH ( @@ -57,7 +57,7 @@ CREATE CATALOG paimon_rest_catalog WITH ( 'uri' = 'http://rest-server:8080', 'token' = 'xxx', - -- FUSE local path configuration + -- FUSE 本地路径配置 'fuse.local-path.enabled' = 'true', 'fuse.local-path.root' = '/mnt/fuse/warehouse', 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', @@ -65,50 +65,50 @@ CREATE CATALOG paimon_rest_catalog WITH ( ); ``` -### Path Resolution Priority +### 路径解析优先级 -When resolving a path, the system checks in the following order (higher priority wins): +解析路径时，系统按以下顺序检查（优先级从高到低）： -1. **Table-level mapping** (`fuse.local-path.table`) -2. **Database-level mapping** (`fuse.local-path.database`) -3. **Root mapping** (`fuse.local-path.root`) +1. **Table 级别映射**（`fuse.local-path.table`） +2. **Database 级别映射**（`fuse.local-path.database`） +3. **根路径映射**（`fuse.local-path.root`） -Example: For table `db1.table1`: -- If `fuse.local-path.table` contains `db1.table1:/mnt/special/t1`, use `/mnt/special/t1` -- Else if `fuse.local-path.database` contains `db1:/mnt/custom/db1`, use `/mnt/custom/db1` -- Else use `fuse.local-path.root` (e.g., `/mnt/fuse/warehouse`) +示例：对于表 `db1.table1`： +- 如果 `fuse.local-path.table` 包含 `db1.table1:/mnt/special/t1`，使用 `/mnt/special/t1` +- 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` +- 否则，使用 `fuse.local-path.root`（如 `/mnt/fuse/warehouse`） -## Implementation +## 实现方案 -### RESTCatalog Modification +### RESTCatalog 修改 -The `fileIOForData` method in `RESTCatalog.java` will be modified: +修改 `RESTCatalog.java` 中的 `fileIOForData` 方法： ```java private FileIO fileIOForData(Path path, Identifier identifier) { - // If FUSE local path is enabled and path matches, use local FileIO + // 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO if (fuseLocalPathEnabled) { Path localPath = resolveFUSELocalPath(path, identifier); if (localPath != null) { - // Use local file IO, no token needed + // 使用本地文件 IO，无需 token return FileIO.get(localPath, CatalogContext.create(new Options(), context.hadoopConf())); } } - // Original logic: data token or ResolvingFileIO + // 原有逻辑：data token 或 ResolvingFileIO return dataTokenEnabled ? new RESTTokenFileIO(context, api, identifier, path) : fileIOFromOptions(path); } /** - * Resolve FUSE local path. Priority: table > database > root. - * @return Local path, or null if not applicable + * 解析 FUSE 本地路径。优先级：table > database > root。 + * @return 本地路径，如果不适用则返回 null */ private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { String pathStr = originalPath.toString(); - // 1. Check table-level mapping + // 1. 检查 Table 级别映射 Map tableMappings = context.options().get(FUSE_LOCAL_PATH_TABLE); String tableKey = identifier.getDatabaseName() + "." + identifier.getTableName(); if (tableMappings.containsKey(tableKey)) { @@ -116,14 +116,14 @@ private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { return convertToLocalPath(pathStr, localRoot); } - // 2. Check database-level mapping + // 2. 检查 Database 级别映射 Map dbMappings = context.options().get(FUSE_LOCAL_PATH_DATABASE); if (dbMappings.containsKey(identifier.getDatabaseName())) { String localRoot = dbMappings.get(identifier.getDatabaseName()); return convertToLocalPath(pathStr, localRoot); } - // 3. Use root mapping + // 3. 使用根路径映射 String fuseRoot = context.options().get(FUSE_LOCAL_PATH_ROOT); if (fuseRoot != null) { return convertToLocalPath(pathStr, fuseRoot); @@ -133,58 +133,58 @@ private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { } private Path convertToLocalPath(String originalPath, String localRoot) { - // Convert remote storage path to local FUSE path - // Example: oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 - // Implementation depends on path structure + // 将远端存储路径转换为本地 FUSE 路径 + // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 + // 具体实现取决于路径结构 } ``` -### Behavior Matrix +### 行为矩阵 -| Configuration | Path Match | Behavior | -|---------------|------------|----------| -| `fuse.local-path.enabled=true` | Yes | Local FileIO for data read/write | -| `fuse.local-path.enabled=true` | No | Fallback to original logic | -| `fuse.local-path.enabled=false` | N/A | Original logic (data token or ResolvingFileIO) | +| 配置 | 路径匹配 | 行为 | +|-----|---------|------| +| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写 | +| `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | +| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | -## Benefits +## 优势 -1. **Performance**: Local file system access is typically faster than network-based remote storage access -2. **Flexibility**: Supports different local paths for different databases/tables -3. **Backward Compatibility**: Disabled by default, existing behavior unchanged +1. **性能提升**：本地文件系统访问通常比基于网络的远端存储访问更快 +2. **灵活性**：支持为不同的数据库/表配置不同的本地路径 +3. **向后兼容**：默认禁用，现有行为不变 -## Security Validation Mechanism +## 安全校验机制 -### Problem Scenarios +### 问题场景 -Incorrect FUSE local path configuration can lead to serious data consistency issues: +错误的 FUSE 本地路径配置可能导致严重的数据一致性问题： -| Scenario | Description | Consequence | -|----------|-------------|-------------| -| **Local path not mounted** | User's configured `/local/table` is not actually FUSE-mounted | Data is written only to local disk, not synced to remote storage, causing data loss | -| **Remote path mismatch** | Local path points to a different table's remote storage path | Data is written to the wrong table, causing data pollution | +| 场景 | 描述 | 后果 | +|-----|------|------| +| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到远端存储，导致数据丢失 | +| **远端路径错误** | 本地路径指向了其他库表的远端存储路径 | 数据写入错误的表，导致数据污染 | -### Validation Mode Configuration +### 校验模式配置 -New configuration parameter to control validation behavior: +新增配置参数控制校验行为： -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `fuse.local-path.validation-mode` | String | `strict` | Validation mode: `strict`, `warn`, or `none` | +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`（严格）、`warn`（警告）、`none`（不校验） | -**Validation Mode Description**: +**校验模式说明**： -| Mode | Behavior | -|------|----------| -| `strict` | Enable validation, throw exception on failure, block the operation | -| `warn` | Enable validation, log warning on failure, but allow operation to proceed | -| `none` | No validation (not recommended, may cause data loss or pollution) | +| 模式 | 行为 | +|-----|------| +| `strict` | 启用校验，失败时抛出异常，阻止操作 | +| `warn` | 启用校验，失败时输出警告日志，但允许操作继续 | +| `none` | 不进行校验（不推荐，可能导致数据丢失或污染） | -### Validation Flow +### 校验流程 ``` ┌─────────────────────────────────────────────────────────────┐ -│ Access Table (getTable) │ +│ 访问表（getTable） │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -196,7 +196,7 @@ New configuration parameter to control validation behavior: │ │ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ - │ Resolve local path│ │ Use original logic│ + │ 解析本地路径 │ │ 使用原有逻辑 │ │ resolveFUSELocalPath│ │ (RESTTokenFileIO) │ └───────────────────┘ └───────────────────┘ │ @@ -209,152 +209,150 @@ New configuration parameter to control validation behavior: │ │ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ - │ Validate local │ │ Skip validation │ - │ path exists │ │ Use local path │ - │ Compare with │ │ directly │ - │ remote data │ │ │ + │ 校验本地路径存在 │ │ 跳过校验 │ + │ 与远端数据比对 │ │ 直接使用本地路径 │ └───────────────────┘ └───────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Validation passed ? │ + │ 校验通过 ? │ └───────────────────────────────────────┘ │ │ Yes No │ │ ▼ ▼ ┌─────────────┐ ┌─────────────────────┐ - │ Use local │ │ validation-mode: │ - │ path │ │ - strict: throw │ - │ LocalFileIO │ │ - warn: warn+fallback│ - └─────────────┘ └─────────────────────┘ + │ 使用本地路径 │ │ validation-mode: │ + │ LocalFileIO │ │ - strict: 抛异常 │ + └─────────────┘ │ - warn: 警告+回退 │ + └─────────────────────┘ ``` -### .identifier File +### .identifier 文件 -Each table directory contains a `.identifier` file for quick validation: +每个表目录下都包含一个 `.identifier` 文件用于快速校验： -**File Location**: `/.identifier` +**文件位置**：`<表路径>/.identifier` -**File Format**: +**文件格式**： ```json {"uuid":"xxx-xxx-xxx-xxx"} ``` -**Usage**: -- Compare table UUID between local and remote paths -- Quick validation before expensive data comparison -- Automatically generated when table is created -- Only UUID is needed (database/table names can change via rename) +**用途**： +- 比对本地和远端路径的表 UUID +- 在昂贵的文件内容比对前进行快速校验 +- 创建表时自动生成 +- 仅需 UUID（database/table 名称可能因重命名而变化） -### Security Validation Implementation +### 安全校验实现 -Use remote data validation to verify FUSE path correctness: read remote storage files via existing FileIO and compare with local files. +使用远端数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取远端存储文件，与本地文件比对。 -**Complete Implementation**: +**完整实现**： ```java /** - * Complete implementation of fileIOForData in RESTCatalog - * Combining FUSE local path validation with remote data validation + * RESTCatalog 中 fileIOForData 的完整实现 + * 结合 FUSE 本地路径校验与远端数据校验 */ private FileIO fileIOForData(Path path, Identifier identifier) { - // If FUSE local path is enabled, try using local path + // 如果 FUSE 本地路径启用，尝试使用本地路径 if (fuseLocalPathEnabled) { Path localPath = resolveFUSELocalPath(path, identifier); if (localPath != null) { - // Execute validation based on validation mode + // 根据校验模式执行校验 ValidationMode mode = getValidationMode(); if (mode != ValidationMode.NONE) { ValidationResult result = validateFUSEPath(localPath, path, identifier); if (!result.isValid()) { handleValidationError(result, mode); - // Validation failed, fallback to default logic + // 校验失败，回退到原有逻辑 return createDefaultFileIO(path, identifier); } } - // Validation passed or skipped, use local FileIO + // 校验通过或跳过校验，使用本地 FileIO return createLocalFileIO(localPath); } } - // Original logic: data token or ResolvingFileIO + // 原有逻辑：data token 或 ResolvingFileIO return createDefaultFileIO(path, identifier); } /** - * Validate FUSE local path + * 校验 FUSE 本地路径 */ private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { - // 1. Create LocalFileIO for local path + // 1. 创建 LocalFileIO 用于本地路径操作 LocalFileIO localFileIO = LocalFileIO.create(); - // 2. Check if local path exists + // 2. 检查本地路径是否存在 if (!localFileIO.exists(localPath)) { - return ValidationResult.fail("Local path does not exist: " + localPath); + return ValidationResult.fail("本地路径不存在: " + localPath); } - // 3. First validation: Table identifier file + // 3. 第一次校验：表标识文件 ValidationResult identifierResult = validateByIdentifierFile(localFileIO, localPath, remotePath, identifier); if (!identifierResult.isSuccess()) { return identifierResult; } - // 4. Second validation: Remote data validation + // 4. 第二次校验：远端数据校验 return validateByRemoteData(localFileIO, localPath, remotePath, identifier); } /** - * First validation: Check .identifier file - * Compare table UUID between local and remote to ensure path correctness + * 第一次校验：检查 .identifier 文件 + * 比对本地和远端的表 UUID 确保路径正确性 */ private ValidationResult validateByIdentifierFile( LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { try { - // 1. Get remote FileIO + // 1. 获取远端存储 FileIO FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - // 2. Read remote identifier file + // 2. 读取远端标识文件 Path remoteIdentifierFile = new Path(remotePath, ".identifier"); if (!remoteFileIO.exists(remoteIdentifierFile)) { - // No identifier file, skip this validation - LOG.debug("No .identifier file found for table: {}, skip identifier validation", identifier); + // 无标识文件，跳过此次校验 + LOG.debug("未找到表 {} 的 .identifier 文件，跳过标识校验", identifier); return ValidationResult.success(); } String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); - // 3. Read local identifier file + // 3. 读取本地标识文件 Path localIdentifierFile = new Path(localPath, ".identifier"); if (!localFileIO.exists(localIdentifierFile)) { return ValidationResult.fail( - "Local .identifier file not found: " + localIdentifierFile + - ". The FUSE path may not be mounted correctly."); + "本地 .identifier 文件未找到: " + localIdentifierFile + + "。FUSE 路径可能未正确挂载。"); } String localIdentifier = readIdentifierFile(localFileIO, localIdentifierFile); - // 4. Compare identifiers + // 4. 比对标识符 if (!remoteIdentifier.equals(localIdentifier)) { return ValidationResult.fail(String.format( - "Table identifier mismatch! Local: %s, Remote: %s. " + - "The local path may point to a different table.", + "表标识不匹配！本地: %s，远端: %s。" + + "本地路径可能指向了其他表。", localIdentifier, remoteIdentifier)); } return ValidationResult.success(); } catch (Exception e) { - LOG.warn("Failed to validate by identifier file for: {}", identifier, e); - return ValidationResult.fail("Identifier validation failed: " + e.getMessage()); + LOG.warn("标识文件校验失败: {}", identifier, e); + return ValidationResult.fail("标识文件校验失败: " + e.getMessage()); } } /** - * Read .identifier file content - * Format: {"uuid":"xxx-xxx-xxx-xxx"} + * 读取 .identifier 文件内容 + * 格式：{"uuid":"xxx-xxx-xxx-xxx"} */ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { try (InputStream in = fileIO.newInputStream(identifierFile)) { @@ -365,81 +363,81 @@ private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOE } /** - * Second validation: Validate FUSE path correctness by comparing remote and local file data - * Uses existing FileIO (RESTTokenFileIO or ResolvingFileIO) to read remote files + * 第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性 + * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取远端存储文件 */ private ValidationResult validateByRemoteData( LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { try { - // 1. Get remote FileIO (using existing logic, can access remote storage) + // 1. 获取远端存储 FileIO（使用现有逻辑，可访问远端存储） FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - // 2. Get latest snapshot via SnapshotManager + // 2. 使用 SnapshotManager 获取最新 snapshot SnapshotManager snapshotManager = new SnapshotManager(remoteFileIO, remotePath); Snapshot latestSnapshot = snapshotManager.latestSnapshot(); Path checksumFile; if (latestSnapshot != null) { - // Has snapshot, use snapshot file for validation + // 有 snapshot，使用 snapshot 文件校验 checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); } else { - // No snapshot (new table), use schema file for validation + // 无 snapshot（新表），使用 schema 文件校验 SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); Optional latestSchema = schemaManager.latest(); if (!latestSchema.isPresent()) { - // No schema (e.g., format table, object table), skip validation - LOG.info("No snapshot or schema found for table: {}, skip validation", identifier); + // 无 schema（如 format 表、object 表），跳过验证 + LOG.info("未找到表 {} 的 snapshot 或 schema，跳过验证", identifier); return ValidationResult.success(); } checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); } - // 3. Read remote file content and compute hash + // 3. 读取远端文件内容并计算 hash FileStatus remoteStatus = remoteFileIO.getFileStatus(checksumFile); String remoteHash = computeFileHash(remoteFileIO, checksumFile); - // 4. Build local file path and compute hash + // 4. 构建本地文件路径并计算 hash Path localChecksumFile = new Path(localPath, remotePath.toUri().getPath()); if (!localFileIO.exists(localChecksumFile)) { return ValidationResult.fail( - "Local file not found: " + localChecksumFile + - ". The FUSE path may not be mounted correctly."); + "本地文件未找到: " + localChecksumFile + + "。FUSE 路径可能未正确挂载。"); } long localSize = localFileIO.getFileSize(localChecksumFile); String localHash = computeFileHash(localFileIO, localChecksumFile); - // 5. Compare file features + // 5. 比对文件特征 if (localSize != remoteStatus.getLen()) { return ValidationResult.fail(String.format( - "File size mismatch! Local: %d bytes, Remote: %d bytes.", + "文件大小不匹配！本地: %d 字节, 远端: %d 字节。", localSize, remoteStatus.getLen())); } if (!localHash.equalsIgnoreCase(remoteHash)) { return ValidationResult.fail(String.format( - "File content hash mismatch! Local: %s, Remote: %s.", + "文件内容哈希不匹配！本地: %s, 远端: %s。", localHash, remoteHash)); } return ValidationResult.success(); } catch (Exception e) { - LOG.warn("Failed to validate FUSE path by remote data for: {}", identifier, e); - return ValidationResult.fail("Remote data validation failed: " + e.getMessage()); + LOG.warn("通过远端数据验证 FUSE 路径失败: {}", identifier, e); + return ValidationResult.fail("远端数据验证失败: " + e.getMessage()); } } /** - * Compute file content hash using FileIO + * 使用 FileIO 计算文件内容哈希 */ private String computeFileHash(FileIO fileIO, Path file) throws IOException { MessageDigest md; try { md = MessageDigest.getInstance("MD5"); } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 algorithm not available", e); + throw new IOException("MD5 算法不可用", e); } try (InputStream is = fileIO.newInputStream(file)) { @@ -453,7 +451,7 @@ private String computeFileHash(FileIO fileIO, Path file) throws IOException { } /** - * Handle validation error + * 处理校验错误 */ private void handleValidationError(ValidationResult result, ValidationMode mode) { String errorMsg = "FUSE local path validation failed: " + result.getErrorMessage(); @@ -465,20 +463,20 @@ private void handleValidationError(ValidationResult result, ValidationMode mode) LOG.warn(errorMsg + ". Falling back to default FileIO."); break; case NONE: - // Won't reach here + // 不会执行到这里 break; } } /** - * Create local FileIO using existing context + * 使用现有 context 创建本地 FileIO */ private FileIO createLocalFileIO(Path localPath) { return FileIO.get(localPath, context); } /** - * Create default FileIO (original logic) + * 创建默认 FileIO（原有逻辑） */ private FileIO createDefaultFileIO(Path path, Identifier identifier) { return dataTokenEnabled @@ -486,12 +484,12 @@ private FileIO createDefaultFileIO(Path path, Identifier identifier) { : fileIOFromOptions(path); } -// ========== Helper Classes ========== +// ========== 辅助类 ========== enum ValidationMode { - STRICT, // Strict mode: throw exception on validation failure - WARN, // Warn mode: log warning on failure, fallback to default logic - NONE // No validation + STRICT, // 严格模式：校验失败抛异常 + WARN, // 警告模式：校验失败只警告，回退到默认逻辑 + NONE // 不校验 } class ValidationResult { @@ -516,140 +514,139 @@ class ValidationResult { } ``` -**Advantages**: +**方案优势**： -| Advantage | Description | -|-----------|-------------| -| **No API Extension Needed** | Uses existing FileIO and SnapshotManager/SchemaManager | -| **Uses LATEST snapshot** | Gets via `SnapshotManager.latestSnapshot()`, no traversal needed | -| **New Table Support** | Falls back to schema file validation when no snapshot | -| **Most Accurate** | Directly validates data consistency, ensures path correctness | -| **Graceful Degradation** | Validation failure falls back to default FileIO | +| 优势 | 说明 | +|------|------| +| **无需扩展 API** | 使用现有 FileIO 和 SnapshotManager/SchemaManager | +| **使用 LATEST snapshot** | 通过 `SnapshotManager.latestSnapshot()` 直接获取，无需遍历 | +| **新表支持** | 无 snapshot 时自动回退到 schema 文件校验 | +| **准确性最高** | 直接验证数据一致性，确保路径正确 | +| **优雅降级** | 校验失败可回退到默认 FileIO | -**Validation File Selection Logic**: +**校验文件选择逻辑**： -| Scenario | Validation File | -|----------|-----------------| -| Has snapshot | Latest snapshot file via `SnapshotManager.latestSnapshot()` | -| No snapshot (new table) | Latest schema file via `SchemaManager.latest()` | -| No schema (e.g., format table, object table) | Skip validation | +| 场景 | 校验文件 | +|------|----------| +| 有 snapshot | 使用 `SnapshotManager.latestSnapshot()` 获取的最新 snapshot 文件 | +| 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | +| 无 schema（如 format 表、object 表）| 跳过校验 | -**Two-Step Validation**: +**两步校验**： -| Step | Validation | Description | -|------|------------|-------------| -| 1 | `.identifier` file | Compare table UUID between local and remote | -| 2 | Remote data validation | Compare snapshot/schema file content | +| 步骤 | 校验方式 | 描述 | +|------|----------|------| +| 1 | `.identifier` 文件 | 比对本地和远端的表 UUID | +| 2 | 远端数据校验 | 比对 snapshot/schema 文件内容 | -**Complete Validation Flow**: +**完整校验流程**： ``` ┌─────────────────────────────────────────────────────────────┐ -│ Validation Flow │ +│ 校验流程 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ Step 1: .identifier Validation │ +│ 第一步：.identifier 校验 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ .identifier exists remotely ? │ + │ 远端存在 .identifier ? │ └───────────────────────────────────────┘ │ │ Yes No │ │ ▼ ▼ ┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ Compare UUID │ │ Skip to Step 2 (remote data │ -│ local vs remote │ │ validation) │ +│ 比对 UUID │ │ 跳过第一步，进入第二步 │ +│ 本地 vs 远端 │ │ （远端数据校验） │ └─────────────────────┘ └─────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ UUID matches ? │ + │ UUID 匹配 ? │ └───────────────────────────────────────┘ │ │ - Yes No → FAIL: Table identifier mismatch + Yes No → 失败：表标识不匹配 │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ Step 2: Remote Data Validation │ +│ 第二步：远端数据校验 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 1. Get remote FileIO (RESTTokenFileIO or ResolvingFileIO) │ +│ 1. 获取远端存储 FileIO（RESTTokenFileIO 或 ResolvingFileIO）│ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 2. Get latest snapshot via SnapshotManager │ +│ 2. 通过 SnapshotManager 获取最新 snapshot │ │ snapshotManager.latestSnapshot() │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Snapshot exists ? │ + │ Snapshot 存在 ? │ └───────────────────────────────────────┘ │ │ Yes No │ │ ▼ ▼ ┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ Use snapshot file │ │ Get latest schema via SchemaManager │ -│ for validation │ │ schemaManager.latest() │ +│ 使用 snapshot 文件 │ │ 通过 SchemaManager 获取最新 schema │ +│ 进行校验 │ │ schemaManager.latest() │ └─────────────────────┘ └─────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Schema exists ? │ + │ Schema 存在 ? │ └───────────────────────────────────────┘ │ │ - Yes No → Skip validation (format/object table) + Yes No → 跳过校验（format/object 表） │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 3. Get remote file metadata (size) │ -│ Compute remote file hash │ +│ 3. 获取远端文件元数据（大小） │ +│ 计算远端文件 hash │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ 4. Read local corresponding file │ +│ 4. 读取本地对应文件 │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Local file exists ? │ + │ 本地文件存在 ? │ └───────────────────────────────────────┘ │ │ - Yes No → Validation failed (wrong path or not mounted) + Yes No → 校验失败（路径错误或未挂载） │ ▼ ┌───────────────────────────────────────┐ - │ File size matches ? │ + │ 文件大小匹配 ? │ └───────────────────────────────────────┘ │ │ - Yes No → Validation failed + Yes No → 校验失败 │ ▼ ┌───────────────────────────────────────┐ - │ File content hash matches ? │ + │ 文件内容 hash 匹配 ? │ └───────────────────────────────────────┘ │ │ - Yes No → Validation failed (path points to wrong table) + Yes No → 校验失败（路径指向错误表） │ ▼ ┌─────────────┐ - │ Validation │ - │ passed │ - │ Safe to use │ + │ 校验通过 │ + │ 可安全使用 │ └─────────────┘ ``` -### Usage Example (with Security Validation) +### 使用示例（启用安全校验） ```sql CREATE CATALOG paimon_rest_catalog WITH ( @@ -658,50 +655,50 @@ CREATE CATALOG paimon_rest_catalog WITH ( 'uri' = 'http://rest-server:8080', 'token' = 'xxx', - -- FUSE local path configuration + -- FUSE 本地路径配置 'fuse.local-path.enabled' = 'true', 'fuse.local-path.root' = '/mnt/fuse/warehouse', - -- Security validation configuration (optional, default: strict) + -- 安全校验配置（可选，默认 strict） 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none ); ``` -## Limitations +## 限制 -1. FUSE mount must be properly configured and accessible -2. Local path must have the same directory structure as the remote storage path -3. Write operations require proper permissions on the local FUSE mount -4. Windows platform has limited FUSE support (requires third-party tools like WinFsp) +1. FUSE 挂载必须正确配置且可访问 +2. 本地路径必须与远端存储路径具有相同的目录结构 +3. 写操作需要本地 FUSE 挂载点具有适当的权限 +4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） -## FUSE Error Handling +## FUSE 错误处理 -This section covers FUSE-specific errors only. Other errors (network, REST API, permission) are already handled by existing mechanisms in Paimon. +本节仅涵盖 FUSE 特有的错误。其他错误（网络、REST API、权限）已由 Paimon 现有机制处理。 -### FUSE-Specific Errors +### FUSE 特有错误 -| Error Type | Scenario | Cause | Handling Strategy | -|------------|----------|-------|-------------------| -| `Transport endpoint is not connected` | Local file read/write | FUSE mount disconnected or crashed | Fail immediately, log error with mount check suggestion | -| `Stale file handle` | File operation | File deleted/modified by another process | Retry once (reopen file) | -| `Device or resource busy` | Delete/rename operation | File still open by another process | Retry with backoff | -| `Input/output error` | Any file operation | FUSE backend failure (remote storage issue) | Fail with clear error message | -| `No such file or directory` (unexpected) | File operation | FUSE mount point not ready | Check mount status, fail | +| 错误类型 | 场景 | 原因 | 处理策略 | +|----------|------|------|----------| +| `Transport endpoint is not connected` | 本地文件读写 | FUSE 挂载断开或崩溃 | 立即失败，记录错误并提示检查挂载状态 | +| `Stale file handle` | 文件操作 | 文件被其他进程删除/修改 | 重试一次（重新打开文件） | +| `Device or resource busy` | 删除/重命名操作 | 文件仍被其他进程占用 | 退避重试 | +| `Input/output error` | 任意文件操作 | FUSE 后端故障（远端存储问题） | 失败并给出明确错误信息 | +| `No such file or directory`（意外） | 文件操作 | FUSE 挂载点未就绪 | 检查挂载状态，失败 | -### Error Handling Strategy +### 错误处理策略 -#### FUSE Mount Disconnection (Most Critical) +#### FUSE 挂载断开（最关键） -The most critical FUSE-specific error is mount disconnection (`Transport endpoint is not connected`). This error indicates: -- FUSE process crashed -- Network issue caused FUSE to disconnect from remote storage -- FUSE mount was manually unmounted +最关键的 FUSE 特有错误是挂载断开（`Transport endpoint is not connected`）。该错误表示： +- FUSE 进程崩溃 +- 网络问题导致 FUSE 与远端存储断开连接 +- FUSE 挂载被手动卸载 -**Handling**: Fail immediately with clear error message. Do NOT retry as the mount must be restored first. +**处理方式**：立即失败并给出明确错误信息。不要重试，因为必须先恢复挂载。 ``` ┌─────────────────────────────────────────────────────────────┐ -│ FUSE Mount Disconnection Handling │ +│ FUSE 挂载断开处理 │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -714,27 +711,27 @@ The most critical FUSE-specific error is mount disconnection (`Transport endpoin │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ LOG.error("FUSE mount disconnected. Please check: │ -│ 1. FUSE process is running │ -│ 2. Mount point exists: ls -la /mnt/fuse/... │ -│ 3. Remount if needed: fusermount -u /mnt/fuse && ...") │ +│ LOG.error("FUSE 挂载断开，路径: {}。请检查: │ +│ 1. FUSE 进程是否运行 │ +│ 2. 挂载点是否存在: ls -la /mnt/fuse/... │ +│ 3. 如需重新挂载: fusermount -u /mnt/fuse && ...") │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Throw IOException with clear message │ + │ 抛出 IOException 并附带明确信息 │ └───────────────────────────────────────┘ ``` -#### Stale File Handle +#### Stale File Handle（过期文件句柄） -This error occurs when a file is deleted or modified by another process while we have an open handle. +当文件在我们持有打开句柄时被其他进程删除或修改，会触发此错误。 -**Handling**: Retry once by reopening the file. +**处理方式**：重试一次，重新打开文件。 ``` ┌─────────────────────────────────────────────────────────────┐ -│ Stale File Handle Handling │ +│ Stale File Handle 处理 │ └─────────────────────────────────────────────────────────────┘ │ ▼ @@ -746,31 +743,29 @@ This error occurs when a file is deleted or modified by another process while we │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ LOG.warn("Stale file handle for {}, retrying...", path) │ +│ LOG.warn("Stale file handle: {}, 重试中...", path) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────┐ - │ Retry once: reopen file │ + │ 重试一次: 重新打开文件 │ └───────────────────────────────────────┘ │ ┌─────────┴─────────┐ - Success Fail + 成功 失败 │ │ ▼ ▼ ┌─────────────┐ ┌─────────────────────┐ - │ Continue │ │ Throw IOException │ - │ operation │ │ with details │ + │ 继续操作 │ │ 抛出 IOException │ + │ │ │ 并附带详细信息 │ └─────────────┘ └─────────────────────┘ ``` -### Implementation Example +### 实现示例 ```java -import org.apache.paimon.utils.RetryUtils; - /** - * FUSE error handler with configurable retry and exponential backoff. + * FUSE 错误处理器，支持可配置的重试和指数退避。 */ public class FuseErrorHandler { private final int maxAttempts; @@ -784,7 +779,7 @@ public class FuseErrorHandler { } /** - * Check if error is FUSE mount disconnection + * 检查是否为 FUSE 挂载断开错误 */ public boolean isFuseMountDisconnected(IOException e) { String message = e.getMessage(); @@ -793,7 +788,7 @@ public class FuseErrorHandler { } /** - * Check if error is stale file handle + * 检查是否为 Stale file handle 错误 */ public boolean isStaleFileHandle(IOException e) { String message = e.getMessage(); @@ -802,7 +797,7 @@ public class FuseErrorHandler { } /** - * Check if error is device or resource busy + * 检查是否为 Device or resource busy 错误 */ public boolean isDeviceBusy(IOException e) { String message = e.getMessage(); @@ -811,8 +806,8 @@ public class FuseErrorHandler { } /** - * Calculate exponential backoff delay. - * Formula: min(initialDelay * 2^attempt, maxDelay) + * 计算指数退避延迟。 + * 公式: min(initialDelay * 2^attempt, maxDelay) */ private long calculateDelay(int attempt) { long delay = initialDelayMs * (1L << attempt); // 2^attempt @@ -820,8 +815,8 @@ public class FuseErrorHandler { } /** - * Execute file operation with FUSE-specific error handling. - * Uses exponential backoff for retryable errors. + * 执行文件操作，处理 FUSE 特有错误。 + * 对可重试错误使用指数退避策略。 */ public T executeWithFuseErrorHandling( SupplierWithIOException operation, @@ -836,37 +831,36 @@ public class FuseErrorHandler { } catch (IOException e) { lastException = e; - // FUSE mount disconnected - fail immediately, no retry + // FUSE 挂载断开 - 立即失败，不重试 if (isFuseMountDisconnected(e)) { - LOG.error("FUSE mount disconnected for path: {}. " + - "Please check: 1) FUSE process is running, " + - "2) Mount point exists, 3) Remount if needed.", path); - throw new IOException("FUSE mount disconnected: " + path, e); + LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + + "2) 挂载点是否存在, 3) 如需重新挂载", path); + throw new IOException("FUSE 挂载断开: " + path, e); } - // Retryable errors: stale file handle, device busy + // 可重试错误: stale file handle, device busy if (isStaleFileHandle(e) || isDeviceBusy(e)) { if (attempt < maxAttempts - 1) { long delay = calculateDelay(attempt); - LOG.warn("FUSE error ({}) for {}, retrying in {}ms (attempt {}/{})", + LOG.warn("FUSE 错误 ({}) 路径: {}, {}ms 后重试 (第 {}/{} 次)", e.getMessage(), path, delay, attempt + 1, maxAttempts); try { Thread.sleep(delay); } catch (InterruptedException ie) { Thread.currentThread().interrupt(); - throw new IOException("Retry interrupted", ie); + throw new IOException("重试被中断", ie); } continue; } } - // Non-retryable errors or max attempts reached + // 不可重试错误或达到最大重试次数 throw e; } } - // Should not reach here, but just in case - throw new IOException("FUSE operation failed after " + maxAttempts + " attempts: " + path, + // 不应到达这里，但以防万一 + throw new IOException("FUSE 操作失败，已重试 " + maxAttempts + " 次: " + path, lastException); } @@ -877,12 +871,12 @@ public class FuseErrorHandler { } ``` -### FuseAwareFileIO Wrapper +### FuseAwareFileIO 包装器 ```java /** - * FileIO wrapper that handles FUSE-specific errors with configurable retry. - * Delegates to LocalFileIO for actual file operations. + * FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 + * 委托给 LocalFileIO 执行实际文件操作。 */ public class FuseAwareFileIO implements FileIO { private final FileIO delegate; @@ -917,19 +911,19 @@ public class FuseAwareFileIO implements FileIO { () -> delegate.exists(path), path, "exists"); } - // ... other methods similarly wrapped + // ... 其他方法类似包装 } ``` -### RESTCatalog Integration +### RESTCatalog 集成 ```java private FileIO fileIOForData(Path path, Identifier identifier) { - // 1. Try to resolve FUSE local path + // 1. 尝试解析 FUSE 本地路径 if (fuseLocalPathEnabled) { Path localPath = resolveFUSELocalPath(path, identifier); if (localPath != null) { - // 2. Validate if needed (see validation section) + // 2. 如果需要，执行校验（参见校验章节） if (validationMode != ValidationMode.NONE) { ValidationResult result = validateFUSEPath(localPath, path, identifier); if (!result.isValid()) { @@ -938,39 +932,39 @@ private FileIO fileIOForData(Path path, Identifier identifier) { } } - // 3. Return FuseAwareFileIO with error handling + // 3. 返回带错误处理的 FuseAwareFileIO return new FuseAwareFileIO(localPath, context); } } - // 4. Fallback to original logic + // 4. 回退到原有逻辑 return dataTokenEnabled ? new RESTTokenFileIO(context, api, identifier, path) : fileIOFromOptions(path); } ``` -### Retry Behavior Examples +### 重试行为示例 -| Error Type | Retry? | Delay Pattern (default settings) | -|------------|--------|----------------------------------| -| `Transport endpoint is not connected` | ❌ No | Fail immediately | -| `Stale file handle` | ✅ Yes | 100ms → 200ms → 400ms → fail | -| `Device or resource busy` | ✅ Yes | 100ms → 200ms → 400ms → fail | -| Other IOException | ❌ No | Propagate immediately | +| 错误类型 | 是否重试 | 延迟模式（默认设置） | +|----------|----------|---------------------| +| `Transport endpoint is not connected` | ❌ 否 | 立即失败 | +| `Stale file handle` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | +| `Device or resource busy` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | +| 其他 IOException | ❌ 否 | 直接抛出 | -### Logging Guidelines +### 日志规范 -| Log Level | Scenario | Example | -|-----------|----------|---------| -| ERROR | FUSE mount disconnected | `FUSE mount disconnected for path: /mnt/fuse/db/table/snapshot-1` | -| WARN | Stale file handle | `Stale file handle for /mnt/fuse/..., retrying once...` | -| INFO | Normal FUSE operations | (Optional, for debugging) | +| 日志级别 | 场景 | 示例 | +|----------|------|------| +| ERROR | FUSE 挂载断开 | `FUSE 挂载断开，路径: /mnt/fuse/db/table/snapshot-1` | +| WARN | Stale file handle | `Stale file handle: /mnt/fuse/..., 重试一次...` | +| INFO | 正常 FUSE 操作 | （可选，用于调试） | -### Best Practices for FUSE Users +### FUSE 用户最佳实践 -1. **Monitor FUSE process**: Use `ps aux | grep fusermount` or your FUSE tool's monitoring -2. **Health check**: Periodically check mount point with `ls` or `stat` -3. **Auto-restart**: Consider using systemd or supervisor to auto-restart FUSE on crash -4. **Log FUSE errors**: Check `dmesg` or FUSE logs for root cause analysis +1. **监控 FUSE 进程**：使用 `ps aux | grep fusermount` 或 FUSE 工具的监控功能 +2. **健康检查**：定期使用 `ls` 或 `stat` 检查挂载点 +3. **自动重启**：考虑使用 systemd 或 supervisor 在崩溃时自动重启 FUSE +4. **日志分析**：查看 `dmesg` 或 FUSE 日志进行根因分析 From bbf6f5c30973763867050878b2c704814d45f473 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Thu, 19 Mar 2026 16:13:46 +0800 Subject: [PATCH 21/23] Implement FUSE local path support for REST Catalog Add simplified FUSE local path mapping to allow accessing remote data through locally mounted FUSE filesystem. This enables direct local file access when data is synchronized via FUSE mount. Features: - Three config options: enabled, root, validation-mode - Path conversion: skip catalog/bucket level from remote path - Validation via default database location check - Three validation modes: strict (error), warn (fallback), none (skip) - Tri-state variable to avoid repeated validation Tests: - 13 test cases covering path conversion and validation logic Co-authored-by: Qwen-Coder --- .../fuse-local-path-simplified-design-cn.md | 488 ++++++++++++++++++ .../pypaimon/catalog/rest/rest_catalog.py | 114 +++- .../pypaimon/common/options/config.py | 25 + .../tests/rest/test_fuse_local_path.py | 230 +++++++++ 4 files changed, 856 insertions(+), 1 deletion(-) create mode 100644 designs/fuse-local-path-simplified-design-cn.md create mode 100644 paimon-python/pypaimon/tests/rest/test_fuse_local_path.py diff --git a/designs/fuse-local-path-simplified-design-cn.md b/designs/fuse-local-path-simplified-design-cn.md new file mode 100644 index 000000000000..9e33e35ade86 --- /dev/null +++ b/designs/fuse-local-path-simplified-design-cn.md @@ -0,0 +1,488 @@ + + +# 简化版 FUSE 本地路径映射设计 + +## 背景 + +在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 + +本文档描述一个**简化版本**的实现，仅支持 Catalog 级别挂载，通过检查 `default` 数据库实现校验。 + +## 目标 + +1. 仅支持 **Catalog 级别**挂载（单一 `fuse.local-path.root` 配置） +2. 校验通过检查 `default` 数据库的 `location` 与本地路径是否匹配 +3. 保持与现有 RESTCatalog 行为的向后兼容性 + +--- + +## 配置参数 + +所有参数定义在 `pypaimon/common/options/config.py` 中的 `FuseOptions` 类： + +| 参数 | 类型 | 默认值 | 描述 | +|-----|------|--------|------| +| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | +| `fuse.local-path.root` | String | 无 | FUSE 挂载的本地根路径，如 `/mnt/fuse/warehouse` | +| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | + +### 使用示例 + +```python +from pypaimon import Catalog + +# 创建 REST Catalog 并启用 FUSE 本地路径 +catalog = Catalog.create({ + 'metastore': 'rest', + 'uri': 'http://rest-server:8080', + 'token': 'xxx', + + # FUSE 本地路径配置 + 'fuse.local-path.enabled': 'true', + 'fuse.local-path.root': '/mnt/fuse/warehouse', + 'fuse.local-path.validation-mode': 'strict' +}) +``` + +--- + +## 校验模式说明 + +**校验模式仅用于校验 FUSE 挂载是否正确**（检查本地路径是否存在），不处理配置错误。 + +| 模式 | 校验失败时行为 | 适用场景 | +|------|---------------|----------| +| `strict` | 抛出异常，阻止操作 | 生产环境，安全优先 | +| `warn` | 记录警告，回退到默认 FileIO | 测试环境，兼容性优先 | +| `none` | 不校验，直接使用 | 信任环境，性能优先 | + +**配置错误**（如 `fuse.local-path.enabled=true` 但 `fuse.local-path.root` 未配置）直接抛异常，不走校验模式。 + +--- + +## 校验流程图 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ FUSE 本地路径校验流程 │ +│ （首次访问数据时触发） │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ validation-mode == NONE ? │ + └───────────────────────────────────────┘ + │ + ┌─────────┴─────────┐ + Yes No + │ │ + ▼ ▼ + ┌─────────────┐ ┌─────────────────────────────┐ + │ 跳过校验 │ │ 调用 get_database("default") │ + │ state=True │ └─────────────────────────────┘ + └─────────────┘ │ + ▼ + ┌─────────────────────────────────┐ + │ 获取 default 数据库的 location │ + │ 转换为本地 FUSE 路径 │ + │ 检查本地路径是否存在 │ + └─────────────────────────────────┘ + │ + ┌───────────┴───────────┐ + 存在 不存在 + │ │ + ▼ ▼ + ┌─────────────────┐ ┌─────────────────────────┐ + │ 校验通过 │ │ 根据 validation-mode: │ + │ state=True │ │ - strict: 抛出异常 │ + └─────────────────┘ │ - warn: state=False │ + └─────────────────────────┘ +``` + +--- + +## 路径转换逻辑 + +将远程存储路径转换为本地 FUSE 路径，需要跳过 catalog 层级： + +```python +def _resolve_fuse_local_path(self, original_path: str) -> str: + """ + 将远程存储路径转换为本地 FUSE 路径。 + + 示例: + - 输入: oss://catalog/db1/table1 + - fuse_root: /mnt/fuse/warehouse + - 输出: /mnt/fuse/warehouse/db1/table1 + + 说明: FUSE 挂载点已映射到 catalog 层级，因此需要跳过路径中的 catalog 名称。 + + Args: + original_path: 原始远程存储路径 + + Returns: + 本地 FUSE 路径 + + Raises: + ValueError: 如果 fuse.local-path.root 未配置 + """ + if not self.fuse_local_path_root: + raise ValueError( + "FUSE local path is enabled but fuse.local-path.root is not configured" + ) + + uri = urlparse(original_path) + + # 提取路径部分 + # 有 scheme 时（如 oss://catalog/db/table）： + # - netloc 是 bucket 名（对应 catalog 名） + # - path 是剩余部分（如 /db/table） + # - 跳过 netloc，只保留 path + # 无 scheme 时（如 catalog/db/table）： + # - 跳过第一段（catalog 名） + if uri.scheme: + # 跳过 netloc（bucket/catalog），只保留 path 部分 + path_part = uri.path.lstrip('/') + else: + # 无 scheme：路径格式为 "catalog/db/table"，跳过第一段 + path_part = original_path.lstrip('/') + segments = path_part.split('/') + if len(segments) > 1: + path_part = '/'.join(segments[1:]) + + return f"{self.fuse_local_path_root.rstrip('/')}/{path_part}" +``` + +--- + +## 行为矩阵 + +| 配置 | 校验结果 | 行为 | +|-----|---------|------| +| `enabled=true, mode=strict` | 通过 | 使用 LocalFileIO | +| `enabled=true, mode=strict` | 失败 | 抛出 ValueError | +| `enabled=true, mode=warn` | 通过 | 使用 LocalFileIO | +| `enabled=true, mode=warn` | 失败 | 警告，回退到默认 FileIO | +| `enabled=true, mode=none` | - | 直接使用 LocalFileIO | +| `enabled=false` | - | 使用原有逻辑（data token 或 FileIO.get） | + +--- + +## 代码改动清单 + +### 1. 新增配置类 FuseOptions + +**文件**: `pypaimon/common/options/config.py` + +```python +class FuseOptions: + """FUSE 本地路径配置选项。""" + + FUSE_LOCAL_PATH_ENABLED = ( + ConfigOptions.key("fuse.local-path.enabled") + .boolean_type() + .default_value(False) + .with_description("是否启用 FUSE 本地路径映射") + ) + + FUSE_LOCAL_PATH_ROOT = ( + ConfigOptions.key("fuse.local-path.root") + .string_type() + .no_default_value() + .with_description("FUSE 挂载的本地根路径，如 /mnt/fuse") + ) + + FUSE_LOCAL_PATH_VALIDATION_MODE = ( + ConfigOptions.key("fuse.local-path.validation-mode") + .string_type() + .default_value("strict") + .with_description("校验模式：strict、warn 或 none") + ) +``` + +### 2. 修改 RESTCatalog + +**文件**: `pypaimon/catalog/rest/rest_catalog.py` + +#### 2.1 新增导入 + +```python +import logging +from urllib.parse import urlparse +from pypaimon.common.options.config import FuseOptions +from pypaimon.filesystem.local_file_io import LocalFileIO + +logger = logging.getLogger(__name__) +``` + +#### 2.2 修改 `__init__` 方法 + +```python +def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): + # ... 原有初始化代码 ... + self.data_token_enabled = self.rest_api.options.get(CatalogOptions.DATA_TOKEN_ENABLED) + + # FUSE 本地路径配置 + self.fuse_local_path_enabled = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) + self.fuse_local_path_root = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ROOT) + self.fuse_validation_mode = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE, "strict") + self._fuse_validation_state = None # None=未校验, True=通过, False=失败 +``` + +#### 2.3 修改 `file_io_for_data` 方法 + +```python +def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: + """ + 获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。 + """ + # 尝试使用 FUSE 本地路径 + if self.fuse_local_path_enabled: + # 配置错误直接抛异常 + local_path = self._resolve_fuse_local_path(table_path) + + # 执行校验（仅首次） + if self._fuse_validation_state is None: + self._validate_fuse_path() + + # 校验通过，返回本地 FileIO + if self._fuse_validation_state: + return LocalFileIO(local_path, self.context.options) + + # warn 模式校验失败后回退 + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) + + # 回退到原有逻辑 + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) +``` + +#### 2.4 新增 `_resolve_fuse_local_path` 方法 + +```python +def _resolve_fuse_local_path(self, original_path: str) -> str: + """ + 解析 FUSE 本地路径。 + + FUSE 挂载点已映射到 catalog 层级，需要跳过路径中的 catalog 名称。 + + Returns: + 本地路径 + + Raises: + ValueError: 如果 fuse.local-path.root 未配置 + """ + if not self.fuse_local_path_root: + raise ValueError( + "FUSE local path is enabled but fuse.local-path.root is not configured" + ) + + uri = urlparse(original_path) + + # 提取路径部分 + # 有 scheme 时（如 oss://catalog/db/table）： + # - netloc 是 bucket 名（对应 catalog 名） + # - path 是剩余部分（如 /db/table） + # - 跳过 netloc，只保留 path + # 无 scheme 时（如 catalog/db/table）： + # - 跳过第一段（catalog 名） + if uri.scheme: + # 跳过 netloc（bucket/catalog），只保留 path 部分 + path_part = uri.path.lstrip('/') + else: + # 无 scheme：路径格式为 "catalog/db/table"，跳过第一段 + path_part = original_path.lstrip('/') + segments = path_part.split('/') + if len(segments) > 1: + path_part = '/'.join(segments[1:]) + + return f"{self.fuse_local_path_root.rstrip('/')}/{path_part}" +``` + +#### 2.5 新增 `_validate_fuse_path` 方法 + +```python +def _validate_fuse_path(self) -> None: + """ + 校验 FUSE 本地路径是否正确挂载。 + + 获取 default 数据库的 location，转换为本地路径后检查是否存在。 + """ + if self.fuse_validation_mode == "none": + self._fuse_validation_state = True + return + + # 获取 default 数据库详情，API 调用失败直接抛异常 + db = self.rest_api.get_database("default") + remote_location = db.location + + if not remote_location: + logger.info("Default database has no location, skipping FUSE validation") + self._fuse_validation_state = True + return + + expected_local = self._resolve_fuse_local_path(remote_location) + local_file_io = LocalFileIO(expected_local, self.context.options) + + # 只校验本地路径是否存在，根据 validation mode 处理 + if not local_file_io.exists(expected_local): + error_msg = ( + f"FUSE local path validation failed: " + f"local path '{expected_local}' does not exist " + f"for default database location '{remote_location}'" + ) + self._handle_validation_error(error_msg) + else: + self._fuse_validation_state = True + logger.info("FUSE local path validation passed") +``` + +**说明**： +- 只负责校验 FUSE 是否正确挂载，不处理配置错误 +- API 调用失败等系统异常直接抛出，不走校验模式 +- 直接调用 `get_database("default")` 获取 default 数据库 +- 将远端 location 转换为本地 FUSE 路径后检查是否存在 +- 简单高效，只需一次 API 调用 + +#### 2.6 新增 `_handle_validation_error` 方法 + +```python +def _handle_validation_error(self, error_msg: str) -> None: + """根据校验模式处理错误。""" + if self.fuse_validation_mode == "strict": + raise ValueError(error_msg) + elif self.fuse_validation_mode == "warn": + logger.warning(f"{error_msg}. Falling back to default FileIO.") + self._fuse_validation_state = False # 标记校验失败，回退到默认 FileIO +``` + +--- + +## 文件结构 + +``` +paimon-python/ +├── pypaimon/ +│ ├── catalog/ +│ │ └── rest/ +│ │ └── rest_catalog.py # 修改：添加 FUSE 支持 +│ └── common/ +│ └── options/ +│ └── config.py # 修改：添加 FuseOptions +``` + +--- + +## 与完整版设计的区别 + +| 特性 | 简化版 | 完整版 | +|------|--------|--------| +| 挂载级别 | 仅 Catalog 级别 | Catalog / Database / Table 三级 | +| 配置项 | 3 个 | 7+ 个 | +| 校验方式 | 检查 default 数据库路径存在性 | 双重校验（标识文件 + 数据哈希） | +| 错误处理 | 基本模式 | FUSE 特有错误重试机制 | +| 复杂度 | 低 | 高 | + +--- + +## 测试用例 + +### 1. 基本功能测试 + +```python +def test_fuse_local_path_basic(): + """测试基本 FUSE 本地路径功能""" + options = { + 'metastore': 'rest', + 'uri': 'http://localhost:8080', + 'token': 'xxx', + 'fuse.local-path.enabled': 'true', + 'fuse.local-path.root': '/mnt/fuse/warehouse', + } + catalog = Catalog.create(options) + # 验证 FUSE 配置已加载 + assert catalog.fuse_local_path_enabled == True +``` + +### 2. 校验模式测试 + +```python +def test_validation_mode_strict(): + """测试 strict 模式校验失败抛出异常""" + # 配置不存在的 FUSE 路径 + # 预期抛出 ValueError + +def test_validation_mode_warn(): + """测试 warn 模式校验失败回退""" + # 配置不存在的 FUSE 路径 + # 预期使用默认 FileIO + +def test_validation_mode_none(): + """测试 none 模式跳过校验""" + # 配置不存在的 FUSE 路径 + # 预期直接使用 LocalFileIO +``` + +### 3. 边界条件测试 + +```python +def test_default_db_no_location(): + """测试 default 数据库没有 location 的情况""" + +def test_default_db_not_exist(): + """测试 default 数据库不存在的情况""" + +def test_resolve_fuse_path_missing_root(): + """测试启用 FUSE 但未配置 root 时报错""" + +def test_disabled_fuse(): + """测试 FUSE 未启用时使用默认逻辑""" +``` + +### 4. `_resolve_fuse_local_path` 路径转换测试 + +```python +def test_resolve_fuse_local_path_basic(): + """测试基本路径转换""" + # 输入: oss://catalog/db1/table1 + # fuse_root: /mnt/fuse/warehouse + # 预期输出: /mnt/fuse/warehouse/db1/table1 + +def test_resolve_fuse_local_path_with_trailing_slash(): + """测试 fuse_root 带尾部斜杠""" + # 输入: oss://catalog/db1/table1 + # fuse_root: /mnt/fuse/warehouse/ + # 预期输出: /mnt/fuse/warehouse/db1/table1 + +def test_resolve_fuse_local_path_deep_path(): + """测试深层路径""" + # 输入: oss://catalog/db1/table1/partition1/file.parquet + # fuse_root: /mnt/fuse/warehouse + # 预期输出: /mnt/fuse/warehouse/db1/table1/partition1/file.parquet + +def test_resolve_fuse_local_path_without_scheme(): + """测试路径没有 scheme""" + # 输入: catalog/db1/table1 + # fuse_root: /mnt/fuse/warehouse + # 预期输出: /mnt/fuse/warehouse/db1/table1 +``` diff --git a/paimon-python/pypaimon/catalog/rest/rest_catalog.py b/paimon-python/pypaimon/catalog/rest/rest_catalog.py index 943d88840e2a..58f3e21f3d23 100644 --- a/paimon-python/pypaimon/catalog/rest/rest_catalog.py +++ b/paimon-python/pypaimon/catalog/rest/rest_catalog.py @@ -15,7 +15,9 @@ See the License for the specific language governing permissions and limitations under the License. """ +import logging from typing import Any, Callable, Dict, List, Optional, Union +from urllib.parse import urlparse from pypaimon.api.api_response import GetTableResponse, PagedList from pypaimon.api.rest_api import RESTApi @@ -32,10 +34,11 @@ from pypaimon.catalog.rest.property_change import PropertyChange from pypaimon.catalog.rest.rest_token_file_io import RESTTokenFileIO from pypaimon.catalog.rest.table_metadata import TableMetadata -from pypaimon.common.options.config import CatalogOptions +from pypaimon.common.options.config import CatalogOptions, FuseOptions from pypaimon.common.options.core_options import CoreOptions from pypaimon.common.file_io import FileIO from pypaimon.common.identifier import Identifier +from pypaimon.filesystem.local_file_io import LocalFileIO from pypaimon.schema.schema import Schema from pypaimon.schema.schema_change import SchemaChange from pypaimon.schema.table_schema import TableSchema @@ -45,6 +48,8 @@ from pypaimon.table.format.format_table import FormatTable, Format from pypaimon.table.iceberg.iceberg_table import IcebergTable +logger = logging.getLogger(__name__) + FORMAT_TABLE_TYPE = "format-table" ICEBERG_TABLE_TYPE = "iceberg-table" @@ -57,6 +62,15 @@ def __init__(self, context: CatalogContext, config_required: Optional[bool] = Tr context.prefer_io_loader, context.fallback_io_loader) self.data_token_enabled = self.rest_api.options.get(CatalogOptions.DATA_TOKEN_ENABLED) + # FUSE local path configuration + self.fuse_local_path_enabled = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) + self.fuse_local_path_root = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_ROOT) + self.fuse_validation_mode = self.context.options.get( + FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE, "strict") + self._fuse_validation_state = None # None=not validated, True=passed, False=failed + def catalog_loader(self): """ Create and return a RESTCatalogLoader for this catalog. @@ -338,9 +352,107 @@ def file_io_from_options(self, table_path: str) -> FileIO: return FileIO.get(table_path, self.context.options) def file_io_for_data(self, table_path: str, identifier: Identifier): + """ + Get FileIO for data access, supporting FUSE local path mapping. + """ + # Try to use FUSE local path + if self.fuse_local_path_enabled: + # Configuration error raises exception directly + local_path = self._resolve_fuse_local_path(table_path) + + # Perform validation (only once) + if self._fuse_validation_state is None: + self._validate_fuse_path() + + # Validation passed, return local FileIO + if self._fuse_validation_state: + return LocalFileIO(local_path, self.context.options) + + # warn mode validation failed, fallback to default FileIO + return RESTTokenFileIO(identifier, table_path, self.context.options) \ + if self.data_token_enabled else self.file_io_from_options(table_path) + + # Fallback to original logic return RESTTokenFileIO(identifier, table_path, self.context.options) \ if self.data_token_enabled else self.file_io_from_options(table_path) + def _resolve_fuse_local_path(self, original_path: str) -> str: + """ + Resolve FUSE local path. + + FUSE mount point is mapped to catalog level, so skip the catalog name in the path. + + Returns: + Local path + + Raises: + ValueError: If fuse.local-path.root is not configured + """ + if not self.fuse_local_path_root: + raise ValueError( + "FUSE local path is enabled but fuse.local-path.root is not configured" + ) + + uri = urlparse(original_path) + + # For URIs with scheme (e.g., oss://bucket/db/table): + # - netloc is the bucket name (which corresponds to catalog name) + # - path is the rest (e.g., /db/table) + # We skip the catalog/bucket level and keep only db/table path. + if uri.scheme: + # Skip netloc (bucket/catalog), only use path part + path_part = uri.path.lstrip('/') + else: + # No scheme: path format is "catalog/db/table", skip first segment + path_part = original_path.lstrip('/') + segments = path_part.split('/') + if len(segments) > 1: + path_part = '/'.join(segments[1:]) + + return f"{self.fuse_local_path_root.rstrip('/')}/{path_part}" + + def _validate_fuse_path(self) -> None: + """ + Validate FUSE local path is correctly mounted. + + Get default database's location, convert to local path and check if it exists. + """ + if self.fuse_validation_mode == "none": + self._fuse_validation_state = True + return + + # Get default database details, API call failure raises exception directly + db = self.rest_api.get_database("default") + remote_location = db.location + + if not remote_location: + logger.info("Default database has no location, skipping FUSE validation") + self._fuse_validation_state = True + return + + expected_local = self._resolve_fuse_local_path(remote_location) + local_file_io = LocalFileIO(expected_local, self.context.options) + + # Only validate if local path exists, handle based on validation mode + if not local_file_io.exists(expected_local): + error_msg = ( + f"FUSE local path validation failed: " + f"local path '{expected_local}' does not exist " + f"for default database location '{remote_location}'" + ) + self._handle_validation_error(error_msg) + else: + self._fuse_validation_state = True + logger.info("FUSE local path validation passed") + + def _handle_validation_error(self, error_msg: str) -> None: + """Handle validation error based on validation mode.""" + if self.fuse_validation_mode == "strict": + raise ValueError(error_msg) + elif self.fuse_validation_mode == "warn": + logger.warning(f"{error_msg}. Falling back to default FileIO.") + self._fuse_validation_state = False # Mark validation failed, fallback to default FileIO + def load_table(self, identifier: Identifier, internal_file_io: Callable[[str], Any], diff --git a/paimon-python/pypaimon/common/options/config.py b/paimon-python/pypaimon/common/options/config.py index fb6a46446ea0..6f5d290495c0 100644 --- a/paimon-python/pypaimon/common/options/config.py +++ b/paimon-python/pypaimon/common/options/config.py @@ -83,3 +83,28 @@ class CatalogOptions: HTTP_USER_AGENT_HEADER = ConfigOptions.key( "header.HTTP_USER_AGENT").string_type().no_default_value().with_description("HTTP User Agent header") BLOB_FILE_IO_DEFAULT_CACHE_SIZE = 2 ** 31 - 1 + + +class FuseOptions: + """FUSE local path configuration options.""" + + FUSE_LOCAL_PATH_ENABLED = ( + ConfigOptions.key("fuse.local-path.enabled") + .boolean_type() + .default_value(False) + .with_description("Whether to enable FUSE local path mapping") + ) + + FUSE_LOCAL_PATH_ROOT = ( + ConfigOptions.key("fuse.local-path.root") + .string_type() + .no_default_value() + .with_description("FUSE mounted local root path, e.g., /mnt/fuse/warehouse") + ) + + FUSE_LOCAL_PATH_VALIDATION_MODE = ( + ConfigOptions.key("fuse.local-path.validation-mode") + .string_type() + .default_value("strict") + .with_description("Validation mode: strict, warn, or none") + ) diff --git a/paimon-python/pypaimon/tests/rest/test_fuse_local_path.py b/paimon-python/pypaimon/tests/rest/test_fuse_local_path.py new file mode 100644 index 000000000000..a1e8e4cc63a5 --- /dev/null +++ b/paimon-python/pypaimon/tests/rest/test_fuse_local_path.py @@ -0,0 +1,230 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +import unittest +from unittest.mock import MagicMock, patch + +from pypaimon.catalog.rest.rest_catalog import RESTCatalog +from pypaimon.common.options import Options +from pypaimon.common.options.config import FuseOptions + + +class TestFuseLocalPath(unittest.TestCase): + """Test cases for FUSE local path functionality.""" + + def _create_catalog_with_fuse( + self, + enabled: bool = True, + root: str = "/mnt/fuse/warehouse", + validation_mode: str = "strict" + ) -> RESTCatalog: + """Helper to create a mock RESTCatalog with FUSE configuration.""" + options = Options({ + "uri": "http://localhost:8080", + "warehouse": "oss://catalog/warehouse", + FuseOptions.FUSE_LOCAL_PATH_ENABLED.key(): str(enabled).lower(), + FuseOptions.FUSE_LOCAL_PATH_ROOT.key(): root, + FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE.key(): validation_mode, + }) + + # Create a mock catalog directly without going through __init__ + catalog = MagicMock(spec=RESTCatalog) + catalog.fuse_local_path_enabled = enabled + catalog.fuse_local_path_root = root + catalog.fuse_validation_mode = validation_mode + catalog._fuse_validation_state = None + catalog.data_token_enabled = False + catalog.rest_api = MagicMock() + catalog.context = MagicMock() + catalog.context.options = options + + # Bind actual methods to the mock + catalog._resolve_fuse_local_path = RESTCatalog._resolve_fuse_local_path.__get__(catalog) + catalog._validate_fuse_path = RESTCatalog._validate_fuse_path.__get__(catalog) + catalog._handle_validation_error = RESTCatalog._handle_validation_error.__get__(catalog) + catalog.file_io_for_data = RESTCatalog.file_io_for_data.__get__(catalog) + catalog.file_io_from_options = MagicMock(return_value=MagicMock()) + + return catalog + + # ========== _resolve_fuse_local_path Tests ========== + + def test_resolve_fuse_local_path_basic(self): + """Test basic path conversion.""" + catalog = self._create_catalog_with_fuse() + + result = catalog._resolve_fuse_local_path("oss://catalog/db1/table1") + self.assertEqual(result, "/mnt/fuse/warehouse/db1/table1") + + def test_resolve_fuse_local_path_with_trailing_slash(self): + """Test fuse_root with trailing slash.""" + catalog = self._create_catalog_with_fuse(root="/mnt/fuse/warehouse/") + + result = catalog._resolve_fuse_local_path("oss://catalog/db1/table1") + self.assertEqual(result, "/mnt/fuse/warehouse/db1/table1") + + def test_resolve_fuse_local_path_deep_path(self): + """Test deep path with multiple levels.""" + catalog = self._create_catalog_with_fuse() + + result = catalog._resolve_fuse_local_path( + "oss://catalog/db1/table1/partition1/file.parquet" + ) + self.assertEqual( + result, + "/mnt/fuse/warehouse/db1/table1/partition1/file.parquet" + ) + + def test_resolve_fuse_local_path_without_scheme(self): + """Test path without scheme.""" + catalog = self._create_catalog_with_fuse() + + result = catalog._resolve_fuse_local_path("catalog/db1/table1") + self.assertEqual(result, "/mnt/fuse/warehouse/db1/table1") + + def test_resolve_fuse_local_path_missing_root(self): + """Test error when root is not configured.""" + catalog = self._create_catalog_with_fuse(root=None) + + with self.assertRaises(ValueError) as context: + catalog._resolve_fuse_local_path("oss://catalog/db1/table1") + + self.assertIn("fuse.local-path.root is not configured", str(context.exception)) + + # ========== Validation Tests ========== + + def test_validation_mode_none_skips_validation(self): + """Test none mode skips validation.""" + catalog = self._create_catalog_with_fuse(validation_mode="none") + + catalog._validate_fuse_path() + + self.assertTrue(catalog._fuse_validation_state) + + def test_validation_mode_strict_raises_on_failure(self): + """Test strict mode raises exception on validation failure.""" + catalog = self._create_catalog_with_fuse(validation_mode="strict") + + # Mock default database with location + mock_db = MagicMock() + mock_db.location = "oss://catalog/default" + catalog.rest_api.get_database.return_value = mock_db + + # Mock LocalFileIO to return False for exists + with patch('pypaimon.catalog.rest.rest_catalog.LocalFileIO') as mock_local_io: + mock_instance = MagicMock() + mock_instance.exists.return_value = False + mock_local_io.return_value = mock_instance + + with self.assertRaises(ValueError) as context: + catalog._validate_fuse_path() + + self.assertIn("FUSE local path validation failed", str(context.exception)) + + def test_validation_mode_warn_fallback_on_failure(self): + """Test warn mode falls back to default FileIO on validation failure.""" + catalog = self._create_catalog_with_fuse(validation_mode="warn") + + # Mock default database with location + mock_db = MagicMock() + mock_db.location = "oss://catalog/default" + catalog.rest_api.get_database.return_value = mock_db + + # Mock LocalFileIO to return False for exists + with patch('pypaimon.catalog.rest.rest_catalog.LocalFileIO') as mock_local_io: + mock_instance = MagicMock() + mock_instance.exists.return_value = False + mock_local_io.return_value = mock_instance + + # Should not raise, just set state to False + catalog._validate_fuse_path() + + self.assertFalse(catalog._fuse_validation_state) + + def test_validation_passes_when_local_exists(self): + """Test validation passes when local path exists.""" + catalog = self._create_catalog_with_fuse(validation_mode="strict") + + # Mock default database with location + mock_db = MagicMock() + mock_db.location = "oss://catalog/default" + catalog.rest_api.get_database.return_value = mock_db + + # Mock LocalFileIO to return True for exists + with patch('pypaimon.catalog.rest.rest_catalog.LocalFileIO') as mock_local_io: + mock_instance = MagicMock() + mock_instance.exists.return_value = True + mock_local_io.return_value = mock_instance + + catalog._validate_fuse_path() + + self.assertTrue(catalog._fuse_validation_state) + + def test_validation_skips_when_no_location(self): + """Test validation skips when default database has no location.""" + catalog = self._create_catalog_with_fuse(validation_mode="strict") + + # Mock default database without location + mock_db = MagicMock() + mock_db.location = None + catalog.rest_api.get_database.return_value = mock_db + + catalog._validate_fuse_path() + + self.assertTrue(catalog._fuse_validation_state) + + # ========== file_io_for_data Tests ========== + + def test_file_io_for_data_disabled_fuse(self): + """Test that disabled FUSE uses default FileIO.""" + catalog = self._create_catalog_with_fuse(enabled=False) + catalog.data_token_enabled = False + + from pypaimon.common.identifier import Identifier + identifier = Identifier.create("db1", "table1") + + _ = catalog.file_io_for_data("oss://catalog/db1/table1", identifier) + catalog.file_io_from_options.assert_called_once() + + def test_file_io_for_data_uses_local_when_validated(self): + """Test that validated FUSE uses LocalFileIO.""" + catalog = self._create_catalog_with_fuse(enabled=True, validation_mode="none") + catalog._fuse_validation_state = True # Already validated + + from pypaimon.common.identifier import Identifier + identifier = Identifier.create("db1", "table1") + + with patch('pypaimon.catalog.rest.rest_catalog.LocalFileIO') as mock_local_io: + mock_local_io.return_value = MagicMock() + _ = catalog.file_io_for_data("oss://catalog/db1/table1", identifier) + mock_local_io.assert_called_once() + + def test_file_io_for_data_fallback_when_validation_failed(self): + """Test that failed validation falls back to default FileIO.""" + catalog = self._create_catalog_with_fuse(enabled=True, validation_mode="warn") + catalog._fuse_validation_state = False # Validation failed + catalog.data_token_enabled = False + + from pypaimon.common.identifier import Identifier + identifier = Identifier.create("db1", "table1") + + _ = catalog.file_io_for_data("oss://catalog/db1/table1", identifier) + catalog.file_io_from_options.assert_called_once() + + +if __name__ == '__main__': + unittest.main() From 54fea93b817426de68155401077691c04b701213 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Thu, 19 Mar 2026 17:27:17 +0800 Subject: [PATCH 22/23] Add documentation for FUSE local path feature Add user-facing documentation for the FUSE local path mapping feature in PyPaimon REST Catalog. Covers configuration options, validation modes, path conversion logic, and usage examples. Co-authored-by: Qwen-Coder --- docs/content/pypaimon/fuse-support.md | 93 +++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/content/pypaimon/fuse-support.md diff --git a/docs/content/pypaimon/fuse-support.md b/docs/content/pypaimon/fuse-support.md new file mode 100644 index 000000000000..41ea330e68bf --- /dev/null +++ b/docs/content/pypaimon/fuse-support.md @@ -0,0 +1,93 @@ +--- +title: "FUSE Support" +weight: 7 +type: docs +aliases: + - /pypaimon/fuse-support.html +--- + +# FUSE Support + +When using PyPaimon REST Catalog to access remote object storage (such as OSS, S3, or HDFS), data access typically goes through remote storage SDKs. However, in scenarios where remote storage paths are mounted locally via FUSE (Filesystem in Userspace), users can access data directly through local filesystem paths for better performance. + +This feature enables PyPaimon to use local file access when FUSE mount is available, bypassing remote storage SDKs. + +## Configuration + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `fuse.local-path.enabled` | Boolean | `false` | Whether to enable FUSE local path mapping | +| `fuse.local-path.root` | String | (none) | FUSE mounted local root path, e.g., `/mnt/fuse/warehouse` | +| `fuse.local-path.validation-mode` | String | `strict` | Validation mode: `strict`, `warn`, or `none` | + +## Usage + +```python +from pypaimon import CatalogFactory + +catalog_options = { + 'metastore': 'rest', + 'uri': 'http://rest-server:8080', + 'warehouse': 'oss://my-catalog/', + 'token.provider': 'xxx', + + # FUSE local path configuration + 'fuse.local-path.enabled': 'true', + 'fuse.local-path.root': '/mnt/fuse/warehouse', + 'fuse.local-path.validation-mode': 'strict' +} + +catalog = CatalogFactory.create(catalog_options) +``` + +## Validation Modes + +Validation is performed on first data access to verify FUSE mount correctness. The `validation-mode` controls behavior when the local path does not exist: + +| Mode | Behavior | Use Case | +|------|----------|----------| +| `strict` | Throw exception, block operation | Production, safety first | +| `warn` | Log warning, fallback to default FileIO | Testing, compatibility first | +| `none` | Skip validation, use directly | Trusted environment, performance first | + +**Note**: Configuration errors (e.g., `fuse.local-path.enabled=true` but `fuse.local-path.root` not configured) will throw exceptions directly, regardless of validation mode. + +## How It Works + +1. When `fuse.local-path.enabled=true`, PyPaimon attempts to use local file access +2. On first data access, validation is triggered (unless mode is `none`) +3. Validation fetches the `default` database location and converts it to local path +4. If local path exists, subsequent data access uses `LocalFileIO` +5. If validation fails, behavior depends on `validation-mode` + +## Example Scenario + +Assume you have: +- Remote storage: `oss://my-catalog/` +- FUSE mount: `/mnt/fuse/warehouse` (mounted to `oss://my-catalog/`) + +```python +from pypaimon import CatalogFactory + +# Create catalog with FUSE enabled +catalog = CatalogFactory.create({ + 'metastore': 'rest', + 'uri': 'http://rest-server:8080', + 'warehouse': 'oss://my-catalog/', + 'fuse.local-path.enabled': 'true', + 'fuse.local-path.root': '/mnt/fuse/warehouse' +}) + +# When reading table data, PyPaimon will: +# 1. Convert "oss://my-catalog/db/table" to "/mnt/fuse/warehouse/db/table" +# 2. Use LocalFileIO to read from local path +# 3. Bypass remote OSS SDK for better performance +table = catalog.get_table('db.table') +reader = table.new_read_builder().new_read() +``` + +## Limitations + +- Only catalog-level FUSE mount is supported (single `fuse.local-path.root` configuration) +- Validation only checks if local path exists, not data consistency +- If FUSE mount becomes unavailable after validation, file operations may fail From 8621ea2e5c00a3ebbb85a6485c5ba25c8d2765f6 Mon Sep 17 00:00:00 2001 From: shyjsarah Date: Thu, 19 Mar 2026 17:42:08 +0800 Subject: [PATCH 23/23] Remove design documents Co-authored-by: Qwen-Coder --- designs/fuse-local-path-design-cn.md | 955 ----------------- designs/fuse-local-path-design-cn.md.bak | 970 ------------------ .../fuse-local-path-simplified-design-cn.md | 488 --------- 3 files changed, 2413 deletions(-) delete mode 100644 designs/fuse-local-path-design-cn.md delete mode 100644 designs/fuse-local-path-design-cn.md.bak delete mode 100644 designs/fuse-local-path-simplified-design-cn.md diff --git a/designs/fuse-local-path-design-cn.md b/designs/fuse-local-path-design-cn.md deleted file mode 100644 index be320e836b76..000000000000 --- a/designs/fuse-local-path-design-cn.md +++ /dev/null @@ -1,955 +0,0 @@ - - -# RESTCatalog FUSE 本地路径配置设计 - -## 背景 - -在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 - -本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 - -## 目标 - -1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 -2. 支持分层路径映射：Catalog 根路径 > Database > Table -3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写 -4. 保持与现有 RESTCatalog 行为的向后兼容性 - ---- - -## 【高】需求一：增加 FUSE 相关配置 - -### 配置参数 - -所有参数定义在 `pypaimon/common/options/config.py` 中的 `FuseOptions` 类： - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | -| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | -| `fuse.local-path.database` | String | (无) | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | -| `fuse.local-path.table` | String | (无) | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | - -### 使用示例 - -```python -from pypaimon import Catalog - -# 创建 REST Catalog 并启用 FUSE 本地路径 -catalog = Catalog.create({ - 'metastore': 'rest', - 'uri': 'http://rest-server:8080', - 'token': 'xxx', - - # FUSE 本地路径配置 - 'fuse.local-path.enabled': 'true', - 'fuse.local-path.root': '/mnt/fuse/warehouse', - 'fuse.local-path.database': 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', - 'fuse.local-path.table': 'db1.table1:/mnt/special/t1' -}) -``` - -### 路径解析优先级 - -解析路径时，系统按以下顺序检查（优先级从高到低）： - -1. **Table 级别映射**（`fuse.local-path.table`） -2. **Database 级别映射**（`fuse.local-path.database`） -3. **根路径映射**（`fuse.local-path.root`） - -示例：对于表 `db1.table1`： -- 如果 `fuse.local-path.table` 包含 `db1.table1:/mnt/special/t1`，使用 `/mnt/special/t1` -- 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` -- 否则，使用 `fuse.local-path.root`（如 `/mnt/fuse/warehouse`） - -### 行为矩阵 - -| 配置 | 路径匹配 | 行为 | -|-----|---------|------| -| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写 | -| `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | -| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 FileIO.get） | - -### FuseOptions 配置类定义 - -在 `pypaimon/common/options/config.py` 中添加： - -```python -class FuseOptions: - """FUSE 本地路径配置选项。""" - - FUSE_LOCAL_PATH_ENABLED = ( - ConfigOptions.key("fuse.local-path.enabled") - .boolean_type() - .default_value(False) - .with_description("是否启用 FUSE 本地路径映射") - ) - - FUSE_LOCAL_PATH_ROOT = ( - ConfigOptions.key("fuse.local-path.root") - .string_type() - .no_default_value() - .with_description("FUSE 挂载的本地根路径，如 /mnt/fuse") - ) - - FUSE_LOCAL_PATH_DATABASE = ( - ConfigOptions.key("fuse.local-path.database") - .string_type() - .no_default_value() - .with_description( - "Database 级别的本地路径映射。格式：db1:/local/path1,db2:/local/path2" - ) - ) - - FUSE_LOCAL_PATH_TABLE = ( - ConfigOptions.key("fuse.local-path.table") - .string_type() - .no_default_value() - .with_description( - "Table 级别的本地路径映射。格式：db1.table1:/local/path1,db2.table2:/local/path2" - ) - ) -``` - -### RESTCatalog 修改 - -修改 `pypaimon/catalog/rest/rest_catalog.py` 中的 `file_io_for_data` 方法： - -```python -from pypaimon.filesystem.local_file_io import LocalFileIO -from pypaimon.common.options.config import FuseOptions - -class RESTCatalog(Catalog): - def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): - # ... 原有初始化代码 ... - self.fuse_local_path_enabled = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) - - def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: - """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" - # 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO - if self.fuse_local_path_enabled: - local_path = self._resolve_fuse_local_path(table_path, identifier) - if local_path is not None: - # 使用本地文件 IO，无需 token - return LocalFileIO(local_path, self.context.options) - - # 原有逻辑：data token 或 FileIO.get - return RESTTokenFileIO(identifier, table_path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(table_path) - - def _resolve_fuse_local_path(self, original_path: str, identifier: Identifier) -> Optional[str]: - """ - 解析 FUSE 本地路径。优先级：table > database > root。 - - Returns: - 本地路径，如果不适用则返回 None - """ - # 1. 检查 Table 级别映射 - table_mappings = self._parse_map_option( - self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_TABLE)) - table_key = f"{identifier.get_database_name()}.{identifier.get_table_name()}" - if table_key in table_mappings: - return self._convert_to_local_path(original_path, table_mappings[table_key]) - - # 2. 检查 Database 级别映射 - db_mappings = self._parse_map_option( - self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_DATABASE)) - if identifier.get_database_name() in db_mappings: - return self._convert_to_local_path( - original_path, db_mappings[identifier.get_database_name()]) - - # 3. 使用根路径映射 - fuse_root = self.context.options.get(FuseOptions.FUSE_LOCAL_PATH_ROOT) - if fuse_root: - return self._convert_to_local_path(original_path, fuse_root) - - return None - - def _convert_to_local_path(self, original_path: str, local_root: str) -> str: - """将远端存储路径转换为本地 FUSE 路径。 - - 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 - """ - from urllib.parse import urlparse - - uri = urlparse(original_path) - if not uri.scheme: - # 已经是本地路径 - return original_path - - # 提取路径部分，移除开头的 scheme 和 netloc - path_part = uri.path - - # 确保路径不以 / 开头（local_root 已经是完整路径） - if path_part.startswith('/'): - path_part = path_part[1:] - - return f"{local_root.rstrip('/')}/{path_part}" - - def _parse_map_option(self, value: Optional[str]) -> Dict[str, str]: - """解析 Map 类型的配置项。 - - 格式：key1:value1,key2:value2 - """ - if not value: - return {} - - result = {} - for item in value.split(','): - item = item.strip() - if ':' in item: - key, val = item.split(':', 1) - result[key.strip()] = val.strip() - return result -``` - ---- - -## 【高】需求二：FUSE 安全校验机制 - -为防止本地路径被误配置或篡改，设计两层校验机制。 - -### 问题场景 - -以下场景可能导致 FUSE 本地路径配置错误，进而引发数据安全问题： - -| 场景 | 问题描述 | 潜在风险 | -|------|---------|---------| -| **路径配置错误** | 用户将 `fuse.local-path.root` 配置为错误的挂载点，如将 `/mnt/fuse-a` 误配置为 `/mnt/fuse-b` | 读写到错误的数据目录，导致数据混乱或丢失 | -| **多租户环境混淆** | 同一机器上挂载了多个租户的 FUSE 路径，用户配置了错误的租户路径 | 跨租户数据泄露或越权访问 | -| **FUSE 挂载点漂移** | FUSE 进程重启后，挂载点路径发生变化但配置未更新 | 访问到其他表的数据，破坏数据一致性 | -| **恶意路径注入** | 攻击者通过篡改配置文件，将本地路径指向敏感数据目录 | 敏感数据泄露或被恶意修改 | -| **表路径重用** | 删除表后重建同名表，但 FUSE 挂载点仍指向旧数据 | 读写到过期数据，业务逻辑错误 | -| **并发挂载冲突** | 多个 FUSE 实例挂载到同一目录的不同状态 | 数据版本不一致，读写冲突 | - -### 校验流程图 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ FUSE 本地路径安全校验流程 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ validation-mode == NONE ? │ - └───────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 跳过校验 │ │ 开始校验 │ - │ 直接使用 │ │ │ - └─────────────┘ └─────────────────────┘ - │ - ▼ - ┌─────────────────────────────────┐ - │ 第一次校验：.identifier 文件 │ - │ 比对远端和本地表标识 UUID │ - └─────────────────────────────────┘ - │ - ┌───────────┴───────────┐ - 成功 失败 - │ │ - ▼ ▼ - ┌─────────────────────┐ ┌───────────────────────┐ - │ 第二次校验：远端数据 │ │ 根据校验模式处理： │ - │ 比对文件大小和哈希 │ │ - strict: 抛异常 │ - │ │ │ - warn: 警告并回退 │ - └─────────────────────┘ └───────────────────────┘ - │ - ┌─────────┴─────────┐ - 成功 失败 - │ │ - ▼ ▼ - ┌─────────────┐ ┌───────────────────────┐ - │ 使用本地 IO │ │ 根据校验模式处理 │ - └─────────────┘ └───────────────────────┘ -``` - -### 配置参数 - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | - -### 校验模式说明 - -| 模式 | 校验失败时行为 | 适用场景 | -|------|---------------|----------| -| `strict` | 抛出异常，阻止操作 | 生产环境，安全优先 | -| `warn` | 记录警告，回退到默认 FileIO | 测试环境，兼容性优先 | -| `none` | 不校验，直接使用 | 信任环境，性能优先 | - -### FuseOptions 配置扩展 - -在 `pypaimon/common/options/config.py` 中添加： - -```python -class FuseOptions: - # ... 原有配置项 ... - - FUSE_LOCAL_PATH_VALIDATION_MODE = ( - ConfigOptions.key("fuse.local-path.validation-mode") - .string_type() - .default_value("strict") - .with_description("校验模式：strict、warn 或 none") - ) -``` - -### Python 安全校验实现 - -在 `pypaimon/catalog/rest/rest_catalog.py` 中添加安全校验逻辑： - -```python -import hashlib -import logging -from enum import Enum -from typing import Optional, Tuple -from pathlib import Path - -class ValidationMode(Enum): - """校验模式枚举。""" - STRICT = "strict" # 严格模式：校验失败抛异常 - WARN = "warn" # 警告模式：校验失败只警告，回退到默认逻辑 - NONE = "none" # 不校验 - - -class ValidationResult: - """校验结果类。""" - - def __init__(self, valid: bool, error_message: Optional[str] = None): - self.valid = valid - self.error_message = error_message - - @staticmethod - def success() -> 'ValidationResult': - return ValidationResult(True) - - @staticmethod - def fail(error_message: str) -> 'ValidationResult': - return ValidationResult(False, error_message) - - -class RESTCatalog(Catalog): - # ... 原有代码 ... - - def _get_validation_mode(self) -> ValidationMode: - """获取校验模式。""" - mode_str = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE, "strict") - return ValidationMode(mode_str.lower()) - - def _validate_fuse_path(self, local_path: str, remote_path: str, - identifier: Identifier) -> ValidationResult: - """校验 FUSE 本地路径。""" - local_file_io = LocalFileIO(local_path, self.context.options) - logger = logging.getLogger(__name__) - - # 1. 检查本地路径是否存在 - if not local_file_io.exists(local_path): - return ValidationResult.fail(f"本地路径不存在: {local_path}") - - # 2. 第一次校验：表标识文件 - identifier_result = self._validate_by_identifier_file( - local_file_io, local_path, remote_path, identifier) - if not identifier_result.valid: - return identifier_result - - # 3. 第二次校验：远端数据校验 - return self._validate_by_remote_data( - local_file_io, local_path, remote_path, identifier) - - def _validate_by_identifier_file(self, local_file_io: LocalFileIO, - local_path: str, remote_path: str, - identifier: Identifier) -> ValidationResult: - """第一次校验：检查 .identifier 文件。""" - logger = logging.getLogger(__name__) - - try: - # 获取远端存储 FileIO - remote_file_io = self._create_default_file_io(remote_path, identifier) - - # 读取远端标识文件 - remote_identifier_file = f"{remote_path.rstrip('/')}/.identifier" - if not remote_file_io.exists(remote_identifier_file): - logger.debug(f"未找到表 {identifier} 的 .identifier 文件，跳过标识校验") - return ValidationResult.success() - - remote_identifier = self._read_identifier_file(remote_file_io, remote_identifier_file) - - # 读取本地标识文件 - local_identifier_file = f"{local_path.rstrip('/')}/.identifier" - if not local_file_io.exists(local_identifier_file): - return ValidationResult.fail( - f"本地 .identifier 文件未找到: {local_identifier_file}。" - "FUSE 路径可能未正确挂载。") - - local_identifier = self._read_identifier_file(local_file_io, local_identifier_file) - - # 比对标识符 - if remote_identifier != local_identifier: - return ValidationResult.fail( - f"表标识不匹配！本地: {local_identifier}，远端: {remote_identifier}。" - "本地路径可能指向了其他表。") - - return ValidationResult.success() - - except Exception as e: - logger.warning(f"标识文件校验失败: {identifier}", exc_info=True) - return ValidationResult.fail(f"标识文件校验失败: {str(e)}") - - def _read_identifier_file(self, file_io: FileIO, identifier_file: str) -> str: - """读取 .identifier 文件内容。""" - import json - content = file_io.read_file_utf8(identifier_file) - data = json.loads(content) - return data.get("uuid", "") - - def _validate_by_remote_data(self, local_file_io: LocalFileIO, - local_path: str, remote_path: str, - identifier: Identifier) -> ValidationResult: - """第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性。""" - logger = logging.getLogger(__name__) - - try: - # 获取远端存储 FileIO - remote_file_io = self._create_default_file_io(remote_path, identifier) - - # 使用 SnapshotManager 获取最新 snapshot - from pypaimon.snapshot.snapshot_manager import SnapshotManager - snapshot_manager = SnapshotManager(remote_file_io, remote_path) - latest_snapshot = snapshot_manager.latest_snapshot() - - if latest_snapshot is not None: - checksum_file = snapshot_manager.snapshot_path(latest_snapshot.id()) - else: - # 无 snapshot（新表），使用 schema 文件校验 - from pypaimon.schema.schema_manager import SchemaManager - schema_manager = SchemaManager(remote_file_io, remote_path) - latest_schema = schema_manager.latest() - - if latest_schema is None: - logger.info(f"未找到表 {identifier} 的 snapshot 或 schema，跳过验证") - return ValidationResult.success() - - checksum_file = schema_manager.to_schema_path(latest_schema.id()) - - # 读取远端文件信息 - remote_status = remote_file_io.get_file_status(checksum_file) - remote_hash = self._compute_file_hash(remote_file_io, checksum_file) - - # 构建本地文件路径 - from urllib.parse import urlparse - uri = urlparse(checksum_file) - path_part = uri.path.lstrip('/') - local_checksum_file = f"{local_path.rstrip('/')}/{path_part}" - - if not local_file_io.exists(local_checksum_file): - return ValidationResult.fail( - f"本地文件未找到: {local_checksum_file}。" - "FUSE 路径可能未正确挂载。") - - local_size = local_file_io.get_file_size(local_checksum_file) - local_hash = self._compute_file_hash(local_file_io, local_checksum_file) - - # 比对文件特征 - if local_size != remote_status.size: - return ValidationResult.fail( - f"文件大小不匹配！本地: {local_size} 字节, 远端: {remote_status.size} 字节。") - - if local_hash.lower() != remote_hash.lower(): - return ValidationResult.fail( - f"文件内容哈希不匹配！本地: {local_hash}, 远端: {remote_hash}。") - - return ValidationResult.success() - - except Exception as e: - logger.warning(f"通过远端数据验证 FUSE 路径失败: {identifier}", exc_info=True) - return ValidationResult.fail(f"远端数据验证失败: {str(e)}") - - def _compute_file_hash(self, file_io: FileIO, file_path: str) -> str: - """计算文件内容的 MD5 哈希。""" - md5 = hashlib.md5() - with file_io.new_input_stream(file_path) as input_stream: - while True: - data = input_stream.read(4096) - if not data: - break - md5.update(data) - return md5.hexdigest() - - def _handle_validation_error(self, result: ValidationResult, mode: ValidationMode): - """处理校验错误。""" - error_msg = f"FUSE local path validation failed: {result.error_message}" - logger = logging.getLogger(__name__) - - if mode == ValidationMode.STRICT: - raise ValueError(error_msg) - elif mode == ValidationMode.WARN: - logger.warning(f"{error_msg}. Falling back to default FileIO.") - - def _create_default_file_io(self, path: str, identifier: Identifier) -> FileIO: - """创建默认 FileIO（原有逻辑）。""" - return RESTTokenFileIO(identifier, path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(path) -``` - -### RESTCatalog 集成校验 - -在 `file_io_for_data` 方法中集成校验逻辑： - -```python -class RESTCatalog(Catalog): - def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): - # ... 原有初始化代码 ... - self.fuse_local_path_enabled = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) - self._validation_mode_cache: Optional[ValidationMode] = None - - def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: - """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" - # 1. 尝试解析 FUSE 本地路径 - if self.fuse_local_path_enabled: - local_path = self._resolve_fuse_local_path(table_path, identifier) - if local_path is not None: - # 2. 如果需要，执行校验 - validation_mode = self._get_validation_mode() - if validation_mode != ValidationMode.NONE: - result = self._validate_fuse_path(local_path, table_path, identifier) - if not result.valid: - self._handle_validation_error(result, validation_mode) - return self._create_default_file_io(table_path, identifier) - - # 3. 返回本地 FileIO - return LocalFileIO(local_path, self.context.options) - - # 4. 回退到原有逻辑 - return RESTTokenFileIO(identifier, table_path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(table_path) -``` - -### 校验优势 - -| 优势 | 说明 | -|------|------| -| 双重校验 | 先校验表标识，再校验数据一致性 | -| 防止路径混淆 | 通过 UUID 确保本地路径指向正确的表 | -| 灵活模式 | strict/warn/none 三种模式适应不同场景 | -| 性能优化 | 仅在启用时执行校验，不影响正常路径 | - ---- - -## 【低】需求三：FUSE 错误处理 - -FUSE 挂载可能出现特有错误，需要特殊处理以保证系统稳定性。 - -### 常见 FUSE 错误 - -| 错误类型 | 错误信息 | 原因 | 处理策略 | -|---------|---------|------|---------| -| 挂载断开 | `Transport endpoint is not connected` | FUSE 进程崩溃或挂载失效 | 立即失败，不重试 | -| 过期文件句柄 | `Stale file handle` | 文件被其他进程修改或删除 | 指数退避重试 | -| 设备忙 | `Device or resource busy` | 资源竞争 | 指数退避重试 | - -### 错误处理流程图 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ FUSE 错误处理流程 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 执行文件操作 │ - └───────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - 成功 失败 - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 返回结果 │ │ 检查错误类型 │ - └─────────────┘ └─────────────────────┘ - │ - ┌───────┴───────┐ - │ │ - ▼ ▼ - ┌─────────────────────────────┐ ┌─────────────────────────────┐ - │ FUSE 挂载断开？ │ │ 其他错误 │ - │ "Transport endpoint is not │ │ │ - │ connected" │ │ │ - └─────────────────────────────┘ └─────────────────────────────┘ - │ │ - Yes │ - │ │ - ▼ ▼ - ┌─────────────────────────────┐ ┌─────────────────────────────┐ - │ LOG.error("FUSE 挂载断开") │ │ 直接抛出异常 │ - │ 抛出异常，不重试 │ └─────────────────────────────┘ - └─────────────────────────────┘ - - ┌───────────────────────────────────────┐ - │ Stale file handle 或 Device busy ? │ - └───────────────────────────────────────┘ - │ - Yes - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ LOG.warn("Stale file handle: {}, 重试中...", path) │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 重试一次: 重新打开文件 │ - └───────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - 成功 失败 - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 继续操作 │ │ 抛出 OSError │ - │ │ │ 并附带详细信息 │ - └─────────────┘ └─────────────────────┘ -``` - -### 配置参数 - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | -| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | -| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | - -### FuseOptions 配置扩展 - -在 `pypaimon/common/options/config.py` 中添加： - -```python -class FuseOptions: - # ... 原有配置项 ... - - FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS = ( - ConfigOptions.key("fuse.local-path.retry.max-attempts") - .int_type() - .default_value(3) - .with_description("FUSE 特有错误的最大重试次数（如 stale file handle）") - ) - - FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS = ( - ConfigOptions.key("fuse.local-path.retry.initial-delay-ms") - .int_type() - .default_value(100) - .with_description("初始重试延迟（毫秒）") - ) - - FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS = ( - ConfigOptions.key("fuse.local-path.retry.max-delay-ms") - .int_type() - .default_value(5000) - .with_description("最大重试延迟（毫秒）") - ) -``` - -### Python FUSE 错误处理实现 - -在 `pypaimon/filesystem/fuse_aware_file_io.py` 中创建 FUSE 错误处理器： - -```python -""" -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -""" -import logging -import time -from typing import Callable, TypeVar, Optional - -from pypaimon.common.file_io import FileIO -from pypaimon.filesystem.local_file_io import LocalFileIO -from pypaimon.common.options import Options -from pypaimon.common.options.config import FuseOptions - - -T = TypeVar('T') -logger = logging.getLogger(__name__) - - -class FuseErrorHandler: - """FUSE 错误处理器，支持可配置的重试和指数退避。""" - - def __init__(self, max_attempts: int = 3, initial_delay_ms: int = 100, - max_delay_ms: int = 5000): - self.max_attempts = max_attempts - self.initial_delay_ms = initial_delay_ms - self.max_delay_ms = max_delay_ms - - def is_fuse_mount_disconnected(self, error: OSError) -> bool: - """检查是否为 FUSE 挂载断开错误。""" - msg = str(error) - return "Transport endpoint is not connected" in msg - - def is_stale_file_handle(self, error: OSError) -> bool: - """检查是否为 Stale file handle 错误。""" - msg = str(error) - return "Stale file handle" in msg - - def is_device_busy(self, error: OSError) -> bool: - """检查是否为 Device or resource busy 错误。""" - msg = str(error) - return "Device or resource busy" in msg - - def calculate_delay(self, attempt: int) -> float: - """计算指数退避延迟（秒）。""" - delay = self.initial_delay_ms * (2 ** attempt) - return min(delay, self.max_delay_ms) / 1000.0 # 转换为秒 - - def execute_with_fuse_error_handling( - self, - operation: Callable[[], T], - path: str, - operation_name: str - ) -> T: - """执行文件操作，处理 FUSE 特有错误。""" - last_exception: Optional[Exception] = None - - for attempt in range(self.max_attempts): - try: - return operation() - except OSError as e: - last_exception = e - - # FUSE 挂载断开 - 立即失败，不重试 - if self.is_fuse_mount_disconnected(e): - logger.error( - f"FUSE 挂载断开，路径: {path}。请检查: " - "1) FUSE 进程是否运行, 2) 挂载点是否存在, 3) 如需重新挂载" - ) - raise OSError(f"FUSE 挂载断开: {path}") from e - - # 可重试错误: stale file handle, device busy - if self.is_stale_file_handle(e) or self.is_device_busy(e): - if attempt < self.max_attempts - 1: - delay = self.calculate_delay(attempt) - logger.warning( - f"FUSE 错误 ({e}) 路径: {path}, " - f"{delay * 1000:.0f}ms 后重试 (第 {attempt + 1}/{self.max_attempts} 次)" - ) - time.sleep(delay) - continue - - # 不可重试错误或达到最大重试次数 - raise - - # 不应到达这里，但以防万一 - raise OSError( - f"FUSE 操作失败，已重试 {self.max_attempts} 次: {path}" - ) from last_exception - - -class FuseAwareFileIO(FileIO): - """FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 - - 委托给 LocalFileIO 执行实际文件操作。 - """ - - def __init__(self, fuse_path: str, catalog_options: Optional[Options] = None): - self.delegate = LocalFileIO(fuse_path, catalog_options) - - options = catalog_options or Options({}) - self.error_handler = FuseErrorHandler( - max_attempts=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), - initial_delay_ms=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), - max_delay_ms=options.get(FuseOptions.FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) - ) - - def new_input_stream(self, path: str): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.new_input_stream(path), path, "new_input_stream" - ) - - def new_output_stream(self, path: str): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.new_output_stream(path), path, "new_output_stream" - ) - - def get_file_status(self, path: str): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.get_file_status(path), path, "get_file_status" - ) - - def list_status(self, path: str): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.list_status(path), path, "list_status" - ) - - def exists(self, path: str) -> bool: - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.exists(path), path, "exists" - ) - - def delete(self, path: str, recursive: bool = False) -> bool: - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.delete(path, recursive), path, "delete" - ) - - def mkdirs(self, path: str) -> bool: - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.mkdirs(path), path, "mkdirs" - ) - - def rename(self, src: str, dst: str) -> bool: - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.rename(src, dst), src, "rename" - ) - - def to_filesystem_path(self, path: str) -> str: - return self.delegate.to_filesystem_path(path) - - def write_parquet(self, path: str, data, compression: str = 'zstd', - zstd_level: int = 1, **kwargs): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.write_parquet(path, data, compression, zstd_level, **kwargs), - path, "write_parquet" - ) - - def write_orc(self, path: str, data, compression: str = 'zstd', - zstd_level: int = 1, **kwargs): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.write_orc(path, data, compression, zstd_level, **kwargs), - path, "write_orc" - ) - - def write_avro(self, path: str, data, avro_schema=None, - compression: str = 'zstd', zstd_level: int = 1, **kwargs): - return self.error_handler.execute_with_fuse_error_handling( - lambda: self.delegate.write_avro(path, data, avro_schema, compression, zstd_level, **kwargs), - path, "write_avro" - ) - - @property - def filesystem(self): - return self.delegate.filesystem - - @property - def uri_reader_factory(self): - return self.delegate.uri_reader_factory - - def close(self): - self.delegate.close() -``` - -### RESTCatalog 集成 FuseAwareFileIO - -在 `pypaimon/catalog/rest/rest_catalog.py` 中集成： - -```python -from pypaimon.filesystem.fuse_aware_file_io import FuseAwareFileIO - -class RESTCatalog(Catalog): - def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: - """获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。""" - # 1. 尝试解析 FUSE 本地路径 - if self.fuse_local_path_enabled: - local_path = self._resolve_fuse_local_path(table_path, identifier) - if local_path is not None: - # 2. 如果需要，执行校验 - validation_mode = self._get_validation_mode() - if validation_mode != ValidationMode.NONE: - result = self._validate_fuse_path(local_path, table_path, identifier) - if not result.valid: - self._handle_validation_error(result, validation_mode) - return self._create_default_file_io(table_path, identifier) - - # 3. 返回带错误处理的 FuseAwareFileIO - return FuseAwareFileIO(local_path, self.context.options) - - # 4. 回退到原有逻辑 - return RESTTokenFileIO(identifier, table_path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(table_path) -``` - -### 重试行为示例 - -| 错误类型 | 是否重试 | 延迟模式（默认设置） | -|----------|----------|---------------------| -| `Transport endpoint is not connected` | ❌ 否 | 立即失败 | -| `Stale file handle` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | -| `Device or resource busy` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | -| 其他 OSError | ❌ 否 | 直接抛出 | - -### 日志规范 - -| 日志级别 | 场景 | 示例 | -|----------|------|------| -| ERROR | FUSE 挂载断开 | `FUSE 挂载断开，路径: /mnt/fuse/db/table/snapshot-1` | -| WARN | Stale file handle | `Stale file handle: /mnt/fuse/..., 重试一次...` | -| INFO | 正常 FUSE 操作 | （可选，用于调试） | - -### FUSE 用户最佳实践 - -1. **监控 FUSE 进程**：使用 `ps aux | grep fusermount` 或 FUSE 工具的监控功能 -2. **健康检查**：定期使用 `ls` 或 `stat` 检查挂载点 -3. **自动重启**：考虑使用 systemd 或 supervisor 在崩溃时自动重启 FUSE -4. **日志分析**：查看 `dmesg` 或 FUSE 日志进行根因分析 - ---- - -## 文件结构 - -新增/修改的文件： - -``` -paimon-python/ -├── pypaimon/ -│ ├── catalog/ -│ │ └── rest/ -│ │ └── rest_catalog.py # 修改：添加 FUSE 支持 -│ ├── common/ -│ │ └── options/ -│ │ └── config.py # 修改：添加 FuseOptions -│ └── filesystem/ -│ └── fuse_aware_file_io.py # 新增：FUSE 错误处理 FileIO -``` - -## 总结 - -本设计为 PyPaimon 的 RESTCatalog 提供 FUSE 本地路径支持，主要特性： - -1. **分层路径映射**：支持 Catalog、Database、Table 三级路径配置 -2. **安全校验**：双重校验机制（表标识 + 数据一致性）防止路径配置错误 -3. **错误处理**：针对 FUSE 特有错误（stale file handle 等）的重试机制 -4. **向后兼容**：默认禁用，不影响现有功能 -5. **灵活配置**：三种校验模式（strict/warn/none）适应不同场景 diff --git a/designs/fuse-local-path-design-cn.md.bak b/designs/fuse-local-path-design-cn.md.bak deleted file mode 100644 index 889c5888ff77..000000000000 --- a/designs/fuse-local-path-design-cn.md.bak +++ /dev/null @@ -1,970 +0,0 @@ - - -# RESTCatalog FUSE 本地路径配置设计 - -## 背景 - -在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 - -本设计引入配置参数以支持 FUSE 挂载的远端存储路径，允许用户在 Catalog、Database 和 Table 三个层级指定本地路径映射。 - -## 目标 - -1. 支持通过本地文件系统访问 FUSE 挂载的远端存储路径 -2. 支持分层路径映射：Catalog 根路径 > Database > Table -3. 当 FUSE 本地路径适用时，使用本地 FileIO 进行数据读写 -4. 保持与现有 RESTCatalog 行为的向后兼容性 - -## 配置参数 - -所有参数定义在 `RESTCatalogOptions.java` 中： - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | -| `fuse.local-path.root` | String | (无) | FUSE 挂载的本地根路径，如 `/mnt/fuse` | -| `fuse.local-path.database` | Map | `{}` | Database 级别的本地路径映射。格式：`db1:/local/path1,db2:/local/path2` | -| `fuse.local-path.table` | Map | `{}` | Table 级别的本地路径映射。格式：`db1.table1:/local/path1,db2.table2:/local/path2` | -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | -| `fuse.local-path.retry.max-attempts` | Integer | `3` | FUSE 特有错误的最大重试次数（如 stale file handle） | -| `fuse.local-path.retry.initial-delay-ms` | Integer | `100` | 初始重试延迟（毫秒） | -| `fuse.local-path.retry.max-delay-ms` | Integer | `5000` | 最大重试延迟（毫秒） | - -## 使用示例 - -### SQL 配置（Flink/Spark） - -```sql -CREATE CATALOG paimon_rest_catalog WITH ( - 'type' = 'paimon', - 'metastore' = 'rest', - 'uri' = 'http://rest-server:8080', - 'token' = 'xxx', - - -- FUSE 本地路径配置 - 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/fuse/warehouse', - 'fuse.local-path.database' = 'db1:/mnt/custom/db1,db2:/mnt/custom/db2', - 'fuse.local-path.table' = 'db1.table1:/mnt/special/t1' -); -``` - -### 路径解析优先级 - -解析路径时，系统按以下顺序检查（优先级从高到低）： - -1. **Table 级别映射**（`fuse.local-path.table`） -2. **Database 级别映射**（`fuse.local-path.database`） -3. **根路径映射**（`fuse.local-path.root`） - -示例：对于表 `db1.table1`： -- 如果 `fuse.local-path.table` 包含 `db1.table1:/mnt/special/t1`，使用 `/mnt/special/t1` -- 否则，如果 `fuse.local-path.database` 包含 `db1:/mnt/custom/db1`，使用 `/mnt/custom/db1` -- 否则，使用 `fuse.local-path.root`（如 `/mnt/fuse/warehouse`） - -## 实现方案 - -### RESTCatalog 修改 - -修改 `RESTCatalog.java` 中的 `fileIOForData` 方法： - -```java -private FileIO fileIOForData(Path path, Identifier identifier) { - // 如果 FUSE 本地路径启用且路径匹配，使用本地 FileIO - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 使用本地文件 IO，无需 token - return FileIO.get(localPath, CatalogContext.create(new Options(), context.hadoopConf())); - } - } - - // 原有逻辑：data token 或 ResolvingFileIO - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} - -/** - * 解析 FUSE 本地路径。优先级：table > database > root。 - * @return 本地路径，如果不适用则返回 null - */ -private Path resolveFUSELocalPath(Path originalPath, Identifier identifier) { - String pathStr = originalPath.toString(); - - // 1. 检查 Table 级别映射 - Map tableMappings = context.options().get(FUSE_LOCAL_PATH_TABLE); - String tableKey = identifier.getDatabaseName() + "." + identifier.getTableName(); - if (tableMappings.containsKey(tableKey)) { - String localRoot = tableMappings.get(tableKey); - return convertToLocalPath(pathStr, localRoot); - } - - // 2. 检查 Database 级别映射 - Map dbMappings = context.options().get(FUSE_LOCAL_PATH_DATABASE); - if (dbMappings.containsKey(identifier.getDatabaseName())) { - String localRoot = dbMappings.get(identifier.getDatabaseName()); - return convertToLocalPath(pathStr, localRoot); - } - - // 3. 使用根路径映射 - String fuseRoot = context.options().get(FUSE_LOCAL_PATH_ROOT); - if (fuseRoot != null) { - return convertToLocalPath(pathStr, fuseRoot); - } - - return null; -} - -private Path convertToLocalPath(String originalPath, String localRoot) { - // 将远端存储路径转换为本地 FUSE 路径 - // 示例：oss://bucket/warehouse/db1/table1 -> /mnt/fuse/warehouse/db1/table1 - // 具体实现取决于路径结构 -} -``` - -### 行为矩阵 - -| 配置 | 路径匹配 | 行为 | -|-----|---------|------| -| `fuse.local-path.enabled=true` | 是 | 本地 FileIO 进行数据读写 | -| `fuse.local-path.enabled=true` | 否 | 回退到原有逻辑 | -| `fuse.local-path.enabled=false` | 不适用 | 原有逻辑（data token 或 ResolvingFileIO） | - -## 优势 - -1. **性能提升**：本地文件系统访问通常比基于网络的远端存储访问更快 -2. **灵活性**：支持为不同的数据库/表配置不同的本地路径 -3. **向后兼容**：默认禁用，现有行为不变 - -## 安全校验机制 - -### 问题场景 - -错误的 FUSE 本地路径配置可能导致严重的数据一致性问题： - -| 场景 | 描述 | 后果 | -|-----|------|------| -| **本地路径未挂载** | 用户配置的 `/local/table` 实际没有 FUSE 挂载 | 数据仅写入本地磁盘，未同步到远端存储，导致数据丢失 | -| **远端路径错误** | 本地路径指向了其他库表的远端存储路径 | 数据写入错误的表，导致数据污染 | - -### 校验模式配置 - -新增配置参数控制校验行为： - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`（严格）、`warn`（警告）、`none`（不校验） | - -**校验模式说明**： - -| 模式 | 行为 | -|-----|------| -| `strict` | 启用校验，失败时抛出异常，阻止操作 | -| `warn` | 启用校验，失败时输出警告日志，但允许操作继续 | -| `none` | 不进行校验（不推荐，可能导致数据丢失或污染） | - -### 校验流程 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ 访问表（getTable） │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ fuse.local-path.enabled == true ? │ -└─────────────────────────────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌───────────────────┐ ┌───────────────────┐ - │ 解析本地路径 │ │ 使用原有逻辑 │ - │ resolveFUSELocalPath│ │ (RESTTokenFileIO) │ - └───────────────────┘ └───────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ validation-mode != none ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌───────────────────┐ ┌───────────────────┐ - │ 校验本地路径存在 │ │ 跳过校验 │ - │ 与远端数据比对 │ │ 直接使用本地路径 │ - └───────────────────┘ └───────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 校验通过 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 使用本地路径 │ │ validation-mode: │ - │ LocalFileIO │ │ - strict: 抛异常 │ - └─────────────┘ │ - warn: 警告+回退 │ - └─────────────────────┘ -``` - -### .identifier 文件 - -每个表目录下都包含一个 `.identifier` 文件用于快速校验： - -**文件位置**：`<表路径>/.identifier` - -**文件格式**： -```json -{"uuid":"xxx-xxx-xxx-xxx"} -``` - -**用途**： -- 比对本地和远端路径的表 UUID -- 在昂贵的文件内容比对前进行快速校验 -- 创建表时自动生成 -- 仅需 UUID（database/table 名称可能因重命名而变化） - -### 安全校验实现 - -使用远端数据校验验证 FUSE 路径正确性：通过现有 FileIO 读取远端存储文件，与本地文件比对。 - -**完整实现**： - -```java -/** - * RESTCatalog 中 fileIOForData 的完整实现 - * 结合 FUSE 本地路径校验与远端数据校验 - */ -private FileIO fileIOForData(Path path, Identifier identifier) { - // 如果 FUSE 本地路径启用，尝试使用本地路径 - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 根据校验模式执行校验 - ValidationMode mode = getValidationMode(); - - if (mode != ValidationMode.NONE) { - ValidationResult result = validateFUSEPath(localPath, path, identifier); - if (!result.isValid()) { - handleValidationError(result, mode); - // 校验失败，回退到原有逻辑 - return createDefaultFileIO(path, identifier); - } - } - - // 校验通过或跳过校验，使用本地 FileIO - return createLocalFileIO(localPath); - } - } - - // 原有逻辑：data token 或 ResolvingFileIO - return createDefaultFileIO(path, identifier); -} - -/** - * 校验 FUSE 本地路径 - */ -private ValidationResult validateFUSEPath(Path localPath, Path remotePath, Identifier identifier) { - // 1. 创建 LocalFileIO 用于本地路径操作 - LocalFileIO localFileIO = LocalFileIO.create(); - - // 2. 检查本地路径是否存在 - if (!localFileIO.exists(localPath)) { - return ValidationResult.fail("本地路径不存在: " + localPath); - } - - // 3. 第一次校验：表标识文件 - ValidationResult identifierResult = validateByIdentifierFile(localFileIO, localPath, remotePath, identifier); - if (!identifierResult.isSuccess()) { - return identifierResult; - } - - // 4. 第二次校验：远端数据校验 - return validateByRemoteData(localFileIO, localPath, remotePath, identifier); -} - -/** - * 第一次校验：检查 .identifier 文件 - * 比对本地和远端的表 UUID 确保路径正确性 - */ -private ValidationResult validateByIdentifierFile( - LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { - try { - // 1. 获取远端存储 FileIO - FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - - // 2. 读取远端标识文件 - Path remoteIdentifierFile = new Path(remotePath, ".identifier"); - if (!remoteFileIO.exists(remoteIdentifierFile)) { - // 无标识文件，跳过此次校验 - LOG.debug("未找到表 {} 的 .identifier 文件，跳过标识校验", identifier); - return ValidationResult.success(); - } - - String remoteIdentifier = readIdentifierFile(remoteFileIO, remoteIdentifierFile); - - // 3. 读取本地标识文件 - Path localIdentifierFile = new Path(localPath, ".identifier"); - if (!localFileIO.exists(localIdentifierFile)) { - return ValidationResult.fail( - "本地 .identifier 文件未找到: " + localIdentifierFile + - "。FUSE 路径可能未正确挂载。"); - } - - String localIdentifier = readIdentifierFile(localFileIO, localIdentifierFile); - - // 4. 比对标识符 - if (!remoteIdentifier.equals(localIdentifier)) { - return ValidationResult.fail(String.format( - "表标识不匹配！本地: %s，远端: %s。" + - "本地路径可能指向了其他表。", - localIdentifier, remoteIdentifier)); - } - - return ValidationResult.success(); - - } catch (Exception e) { - LOG.warn("标识文件校验失败: {}", identifier, e); - return ValidationResult.fail("标识文件校验失败: " + e.getMessage()); - } -} - -/** - * 读取 .identifier 文件内容 - * 格式：{"uuid":"xxx-xxx-xxx-xxx"} - */ -private String readIdentifierFile(FileIO fileIO, Path identifierFile) throws IOException { - try (InputStream in = fileIO.newInputStream(identifierFile)) { - String json = IOUtils.readUTF8Fully(in); - JsonNode node = JsonSerdeUtil.fromJson(json, JsonNode.class); - return node.get("uuid").asText(); - } -} - -/** - * 第二次校验：通过比对远端存储和本地文件验证 FUSE 路径正确性 - * 使用现有 FileIO（RESTTokenFileIO 或 ResolvingFileIO）读取远端存储文件 - */ -private ValidationResult validateByRemoteData( - LocalFileIO localFileIO, Path localPath, Path remotePath, Identifier identifier) { - try { - // 1. 获取远端存储 FileIO（使用现有逻辑，可访问远端存储） - FileIO remoteFileIO = createDefaultFileIO(remotePath, identifier); - - // 2. 使用 SnapshotManager 获取最新 snapshot - SnapshotManager snapshotManager = new SnapshotManager(remoteFileIO, remotePath); - Snapshot latestSnapshot = snapshotManager.latestSnapshot(); - - Path checksumFile; - if (latestSnapshot != null) { - // 有 snapshot，使用 snapshot 文件校验 - checksumFile = snapshotManager.snapshotPath(latestSnapshot.id()); - } else { - // 无 snapshot（新表），使用 schema 文件校验 - SchemaManager schemaManager = new SchemaManager(remoteFileIO, remotePath); - Optional latestSchema = schemaManager.latest(); - if (!latestSchema.isPresent()) { - // 无 schema（如 format 表、object 表），跳过验证 - LOG.info("未找到表 {} 的 snapshot 或 schema，跳过验证", identifier); - return ValidationResult.success(); - } - checksumFile = schemaManager.toSchemaPath(latestSchema.get().id()); - } - - // 3. 读取远端文件内容并计算 hash - FileStatus remoteStatus = remoteFileIO.getFileStatus(checksumFile); - String remoteHash = computeFileHash(remoteFileIO, checksumFile); - - // 4. 构建本地文件路径并计算 hash - Path localChecksumFile = new Path(localPath, remotePath.toUri().getPath()); - - if (!localFileIO.exists(localChecksumFile)) { - return ValidationResult.fail( - "本地文件未找到: " + localChecksumFile + - "。FUSE 路径可能未正确挂载。"); - } - - long localSize = localFileIO.getFileSize(localChecksumFile); - String localHash = computeFileHash(localFileIO, localChecksumFile); - - // 5. 比对文件特征 - if (localSize != remoteStatus.getLen()) { - return ValidationResult.fail(String.format( - "文件大小不匹配！本地: %d 字节, 远端: %d 字节。", - localSize, remoteStatus.getLen())); - } - - if (!localHash.equalsIgnoreCase(remoteHash)) { - return ValidationResult.fail(String.format( - "文件内容哈希不匹配！本地: %s, 远端: %s。", - localHash, remoteHash)); - } - - return ValidationResult.success(); - - } catch (Exception e) { - LOG.warn("通过远端数据验证 FUSE 路径失败: {}", identifier, e); - return ValidationResult.fail("远端数据验证失败: " + e.getMessage()); - } -} - -/** - * 使用 FileIO 计算文件内容哈希 - */ -private String computeFileHash(FileIO fileIO, Path file) throws IOException { - MessageDigest md; - try { - md = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new IOException("MD5 算法不可用", e); - } - - try (InputStream is = fileIO.newInputStream(file)) { - byte[] buffer = new byte[4096]; - int bytesRead; - while ((bytesRead = is.read(buffer)) != -1) { - md.update(buffer, 0, bytesRead); - } - } - return Hex.encodeHexString(md.digest()); -} - -/** - * 处理校验错误 - */ -private void handleValidationError(ValidationResult result, ValidationMode mode) { - String errorMsg = "FUSE local path validation failed: " + result.getErrorMessage(); - - switch (mode) { - case STRICT: - throw new IllegalArgumentException(errorMsg); - case WARN: - LOG.warn(errorMsg + ". Falling back to default FileIO."); - break; - case NONE: - // 不会执行到这里 - break; - } -} - -/** - * 使用现有 context 创建本地 FileIO - */ -private FileIO createLocalFileIO(Path localPath) { - return FileIO.get(localPath, context); -} - -/** - * 创建默认 FileIO（原有逻辑） - */ -private FileIO createDefaultFileIO(Path path, Identifier identifier) { - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} - -// ========== 辅助类 ========== - -enum ValidationMode { - STRICT, // 严格模式：校验失败抛异常 - WARN, // 警告模式：校验失败只警告，回退到默认逻辑 - NONE // 不校验 -} - -class ValidationResult { - private final boolean valid; - private final String errorMessage; - - private ValidationResult(boolean valid, String errorMessage) { - this.valid = valid; - this.errorMessage = errorMessage; - } - - static ValidationResult success() { - return new ValidationResult(true, null); - } - - static ValidationResult fail(String errorMessage) { - return new ValidationResult(false, errorMessage); - } - - boolean isValid() { return valid; } - String getErrorMessage() { return errorMessage; } -} -``` - -**方案优势**： - -| 优势 | 说明 | -|------|------| -| **无需扩展 API** | 使用现有 FileIO 和 SnapshotManager/SchemaManager | -| **使用 LATEST snapshot** | 通过 `SnapshotManager.latestSnapshot()` 直接获取，无需遍历 | -| **新表支持** | 无 snapshot 时自动回退到 schema 文件校验 | -| **准确性最高** | 直接验证数据一致性，确保路径正确 | -| **优雅降级** | 校验失败可回退到默认 FileIO | - -**校验文件选择逻辑**： - -| 场景 | 校验文件 | -|------|----------| -| 有 snapshot | 使用 `SnapshotManager.latestSnapshot()` 获取的最新 snapshot 文件 | -| 无 snapshot（新表）| 使用 `SchemaManager.latest()` 获取的最新 schema 文件 | -| 无 schema（如 format 表、object 表）| 跳过校验 | - -**两步校验**： - -| 步骤 | 校验方式 | 描述 | -|------|----------|------| -| 1 | `.identifier` 文件 | 比对本地和远端的表 UUID | -| 2 | 远端数据校验 | 比对 snapshot/schema 文件内容 | - -**完整校验流程**： - -``` -┌─────────────────────────────────────────────────────────────┐ -│ 校验流程 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 第一步：.identifier 校验 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 远端存在 .identifier ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 比对 UUID │ │ 跳过第一步，进入第二步 │ -│ 本地 vs 远端 │ │ （远端数据校验） │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ UUID 匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 失败：表标识不匹配 - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 第二步：远端数据校验 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 1. 获取远端存储 FileIO（RESTTokenFileIO 或 ResolvingFileIO）│ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 2. 通过 SnapshotManager 获取最新 snapshot │ -│ snapshotManager.latestSnapshot() │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Snapshot 存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No - │ │ - ▼ ▼ -┌─────────────────────┐ ┌─────────────────────────────────────┐ -│ 使用 snapshot 文件 │ │ 通过 SchemaManager 获取最新 schema │ -│ 进行校验 │ │ schemaManager.latest() │ -└─────────────────────┘ └─────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ Schema 存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 跳过校验（format/object 表） - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 3. 获取远端文件元数据（大小） │ -│ 计算远端文件 hash │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ 4. 读取本地对应文件 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 本地文件存在 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败（路径错误或未挂载） - │ - ▼ - ┌───────────────────────────────────────┐ - │ 文件大小匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败 - │ - ▼ - ┌───────────────────────────────────────┐ - │ 文件内容 hash 匹配 ? │ - └───────────────────────────────────────┘ - │ │ - Yes No → 校验失败（路径指向错误表） - │ - ▼ - ┌─────────────┐ - │ 校验通过 │ - │ 可安全使用 │ - └─────────────┘ -``` - -### 使用示例（启用安全校验） - -```sql -CREATE CATALOG paimon_rest_catalog WITH ( - 'type' = 'paimon', - 'metastore' = 'rest', - 'uri' = 'http://rest-server:8080', - 'token' = 'xxx', - - -- FUSE 本地路径配置 - 'fuse.local-path.enabled' = 'true', - 'fuse.local-path.root' = '/mnt/fuse/warehouse', - - -- 安全校验配置（可选，默认 strict） - 'fuse.local-path.validation-mode' = 'strict' -- strict/warn/none -); -``` - -## 限制 - -1. FUSE 挂载必须正确配置且可访问 -2. 本地路径必须与远端存储路径具有相同的目录结构 -3. 写操作需要本地 FUSE 挂载点具有适当的权限 -4. Windows 平台 FUSE 支持有限（需第三方工具如 WinFsp） - -## FUSE 错误处理 - -本节仅涵盖 FUSE 特有的错误。其他错误（网络、REST API、权限）已由 Paimon 现有机制处理。 - -### FUSE 特有错误 - -| 错误类型 | 场景 | 原因 | 处理策略 | -|----------|------|------|----------| -| `Transport endpoint is not connected` | 本地文件读写 | FUSE 挂载断开或崩溃 | 立即失败，记录错误并提示检查挂载状态 | -| `Stale file handle` | 文件操作 | 文件被其他进程删除/修改 | 重试一次（重新打开文件） | -| `Device or resource busy` | 删除/重命名操作 | 文件仍被其他进程占用 | 退避重试 | -| `Input/output error` | 任意文件操作 | FUSE 后端故障（远端存储问题） | 失败并给出明确错误信息 | -| `No such file or directory`（意外） | 文件操作 | FUSE 挂载点未就绪 | 检查挂载状态，失败 | - -### 错误处理策略 - -#### FUSE 挂载断开（最关键） - -最关键的 FUSE 特有错误是挂载断开（`Transport endpoint is not connected`）。该错误表示： -- FUSE 进程崩溃 -- 网络问题导致 FUSE 与远端存储断开连接 -- FUSE 挂载被手动卸载 - -**处理方式**：立即失败并给出明确错误信息。不要重试，因为必须先恢复挂载。 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ FUSE 挂载断开处理 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ IOException: "Transport endpoint is │ - │ not connected" ? │ - └───────────────────────────────────────┘ - │ - Yes - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ LOG.error("FUSE 挂载断开，路径: {}。请检查: │ -│ 1. FUSE 进程是否运行 │ -│ 2. 挂载点是否存在: ls -la /mnt/fuse/... │ -│ 3. 如需重新挂载: fusermount -u /mnt/fuse && ...") │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 抛出 IOException 并附带明确信息 │ - └───────────────────────────────────────┘ -``` - -#### Stale File Handle（过期文件句柄） - -当文件在我们持有打开句柄时被其他进程删除或修改，会触发此错误。 - -**处理方式**：重试一次，重新打开文件。 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Stale File Handle 处理 │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ IOException: "Stale file handle" ? │ - └───────────────────────────────────────┘ - │ - Yes - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ LOG.warn("Stale file handle: {}, 重试中...", path) │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ 重试一次: 重新打开文件 │ - └───────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - 成功 失败 - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────┐ - │ 继续操作 │ │ 抛出 IOException │ - │ │ │ 并附带详细信息 │ - └─────────────┘ └─────────────────────┘ -``` - -### 实现示例 - -```java -/** - * FUSE 错误处理器，支持可配置的重试和指数退避。 - */ -public class FuseErrorHandler { - private final int maxAttempts; - private final long initialDelayMs; - private final long maxDelayMs; - - public FuseErrorHandler(int maxAttempts, long initialDelayMs, long maxDelayMs) { - this.maxAttempts = maxAttempts; - this.initialDelayMs = initialDelayMs; - this.maxDelayMs = maxDelayMs; - } - - /** - * 检查是否为 FUSE 挂载断开错误 - */ - public boolean isFuseMountDisconnected(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Transport endpoint is not connected"); - } - - /** - * 检查是否为 Stale file handle 错误 - */ - public boolean isStaleFileHandle(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Stale file handle"); - } - - /** - * 检查是否为 Device or resource busy 错误 - */ - public boolean isDeviceBusy(IOException e) { - String message = e.getMessage(); - return message != null && - message.contains("Device or resource busy"); - } - - /** - * 计算指数退避延迟。 - * 公式: min(initialDelay * 2^attempt, maxDelay) - */ - private long calculateDelay(int attempt) { - long delay = initialDelayMs * (1L << attempt); // 2^attempt - return Math.min(delay, maxDelayMs); - } - - /** - * 执行文件操作，处理 FUSE 特有错误。 - * 对可重试错误使用指数退避策略。 - */ - public T executeWithFuseErrorHandling( - SupplierWithIOException operation, - Path path, - String operationName) throws IOException { - - IOException lastException = null; - - for (int attempt = 0; attempt < maxAttempts; attempt++) { - try { - return operation.get(); - } catch (IOException e) { - lastException = e; - - // FUSE 挂载断开 - 立即失败，不重试 - if (isFuseMountDisconnected(e)) { - LOG.error("FUSE 挂载断开，路径: {}。请检查: 1) FUSE 进程是否运行, " + - "2) 挂载点是否存在, 3) 如需重新挂载", path); - throw new IOException("FUSE 挂载断开: " + path, e); - } - - // 可重试错误: stale file handle, device busy - if (isStaleFileHandle(e) || isDeviceBusy(e)) { - if (attempt < maxAttempts - 1) { - long delay = calculateDelay(attempt); - LOG.warn("FUSE 错误 ({}) 路径: {}, {}ms 后重试 (第 {}/{} 次)", - e.getMessage(), path, delay, attempt + 1, maxAttempts); - try { - Thread.sleep(delay); - } catch (InterruptedException ie) { - Thread.currentThread().interrupt(); - throw new IOException("重试被中断", ie); - } - continue; - } - } - - // 不可重试错误或达到最大重试次数 - throw e; - } - } - - // 不应到达这里，但以防万一 - throw new IOException("FUSE 操作失败，已重试 " + maxAttempts + " 次: " + path, - lastException); - } - - @FunctionalInterface - interface SupplierWithIOException { - T get() throws IOException; - } -} -``` - -### FuseAwareFileIO 包装器 - -```java -/** - * FileIO 包装器，处理 FUSE 特有错误，支持可配置重试。 - * 委托给 LocalFileIO 执行实际文件操作。 - */ -public class FuseAwareFileIO implements FileIO { - private final FileIO delegate; - private final FuseErrorHandler errorHandler; - - public FuseAwareFileIO(Path fusePath, CatalogContext context) { - this.delegate = FileIO.get(fusePath, context); // LocalFileIO - - Options options = context.options(); - this.errorHandler = new FuseErrorHandler( - options.getInteger(FUSE_LOCAL_PATH_RETRY_MAX_ATTEMPTS, 3), - options.getLong(FUSE_LOCAL_PATH_RETRY_INITIAL_DELAY_MS, 100), - options.getLong(FUSE_LOCAL_PATH_RETRY_MAX_DELAY_MS, 5000) - ); - } - - @Override - public SeekableInputStream newInputStream(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.newInputStream(path), path, "newInputStream"); - } - - @Override - public FileStatus getFileStatus(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.getFileStatus(path), path, "getFileStatus"); - } - - @Override - public boolean exists(Path path) throws IOException { - return errorHandler.executeWithFuseErrorHandling( - () -> delegate.exists(path), path, "exists"); - } - - // ... 其他方法类似包装 -} -``` - -### RESTCatalog 集成 - -```java -private FileIO fileIOForData(Path path, Identifier identifier) { - // 1. 尝试解析 FUSE 本地路径 - if (fuseLocalPathEnabled) { - Path localPath = resolveFUSELocalPath(path, identifier); - if (localPath != null) { - // 2. 如果需要，执行校验（参见校验章节） - if (validationMode != ValidationMode.NONE) { - ValidationResult result = validateFUSEPath(localPath, path, identifier); - if (!result.isValid()) { - handleValidationError(result, validationMode); - return createDefaultFileIO(path, identifier); - } - } - - // 3. 返回带错误处理的 FuseAwareFileIO - return new FuseAwareFileIO(localPath, context); - } - } - - // 4. 回退到原有逻辑 - return dataTokenEnabled - ? new RESTTokenFileIO(context, api, identifier, path) - : fileIOFromOptions(path); -} -``` - -### 重试行为示例 - -| 错误类型 | 是否重试 | 延迟模式（默认设置） | -|----------|----------|---------------------| -| `Transport endpoint is not connected` | ❌ 否 | 立即失败 | -| `Stale file handle` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | -| `Device or resource busy` | ✅ 是 | 100ms → 200ms → 400ms → 失败 | -| 其他 IOException | ❌ 否 | 直接抛出 | - -### 日志规范 - -| 日志级别 | 场景 | 示例 | -|----------|------|------| -| ERROR | FUSE 挂载断开 | `FUSE 挂载断开，路径: /mnt/fuse/db/table/snapshot-1` | -| WARN | Stale file handle | `Stale file handle: /mnt/fuse/..., 重试一次...` | -| INFO | 正常 FUSE 操作 | （可选，用于调试） | - -### FUSE 用户最佳实践 - -1. **监控 FUSE 进程**：使用 `ps aux | grep fusermount` 或 FUSE 工具的监控功能 -2. **健康检查**：定期使用 `ls` 或 `stat` 检查挂载点 -3. **自动重启**：考虑使用 systemd 或 supervisor 在崩溃时自动重启 FUSE -4. **日志分析**：查看 `dmesg` 或 FUSE 日志进行根因分析 - diff --git a/designs/fuse-local-path-simplified-design-cn.md b/designs/fuse-local-path-simplified-design-cn.md deleted file mode 100644 index 9e33e35ade86..000000000000 --- a/designs/fuse-local-path-simplified-design-cn.md +++ /dev/null @@ -1,488 +0,0 @@ - - -# 简化版 FUSE 本地路径映射设计 - -## 背景 - -在使用 Paimon RESTCatalog 访问远端对象存储（如 OSS、S3、HDFS）时，数据访问通常通过远端存储 SDK 进行。然而，在远端存储路径通过 FUSE（用户空间文件系统）挂载到本地的场景下，用户可以直接通过本地文件系统路径访问数据，获得更好的性能。 - -本文档描述一个**简化版本**的实现，仅支持 Catalog 级别挂载，通过检查 `default` 数据库实现校验。 - -## 目标 - -1. 仅支持 **Catalog 级别**挂载（单一 `fuse.local-path.root` 配置） -2. 校验通过检查 `default` 数据库的 `location` 与本地路径是否匹配 -3. 保持与现有 RESTCatalog 行为的向后兼容性 - ---- - -## 配置参数 - -所有参数定义在 `pypaimon/common/options/config.py` 中的 `FuseOptions` 类： - -| 参数 | 类型 | 默认值 | 描述 | -|-----|------|--------|------| -| `fuse.local-path.enabled` | Boolean | `false` | 是否启用 FUSE 本地路径映射 | -| `fuse.local-path.root` | String | 无 | FUSE 挂载的本地根路径，如 `/mnt/fuse/warehouse` | -| `fuse.local-path.validation-mode` | String | `strict` | 校验模式：`strict`、`warn` 或 `none` | - -### 使用示例 - -```python -from pypaimon import Catalog - -# 创建 REST Catalog 并启用 FUSE 本地路径 -catalog = Catalog.create({ - 'metastore': 'rest', - 'uri': 'http://rest-server:8080', - 'token': 'xxx', - - # FUSE 本地路径配置 - 'fuse.local-path.enabled': 'true', - 'fuse.local-path.root': '/mnt/fuse/warehouse', - 'fuse.local-path.validation-mode': 'strict' -}) -``` - ---- - -## 校验模式说明 - -**校验模式仅用于校验 FUSE 挂载是否正确**（检查本地路径是否存在），不处理配置错误。 - -| 模式 | 校验失败时行为 | 适用场景 | -|------|---------------|----------| -| `strict` | 抛出异常，阻止操作 | 生产环境，安全优先 | -| `warn` | 记录警告，回退到默认 FileIO | 测试环境，兼容性优先 | -| `none` | 不校验，直接使用 | 信任环境，性能优先 | - -**配置错误**（如 `fuse.local-path.enabled=true` 但 `fuse.local-path.root` 未配置）直接抛异常，不走校验模式。 - ---- - -## 校验流程图 - -``` -┌─────────────────────────────────────────────────────────────┐ -│ FUSE 本地路径校验流程 │ -│ （首次访问数据时触发） │ -└─────────────────────────────────────────────────────────────┘ - │ - ▼ - ┌───────────────────────────────────────┐ - │ validation-mode == NONE ? │ - └───────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - Yes No - │ │ - ▼ ▼ - ┌─────────────┐ ┌─────────────────────────────┐ - │ 跳过校验 │ │ 调用 get_database("default") │ - │ state=True │ └─────────────────────────────┘ - └─────────────┘ │ - ▼ - ┌─────────────────────────────────┐ - │ 获取 default 数据库的 location │ - │ 转换为本地 FUSE 路径 │ - │ 检查本地路径是否存在 │ - └─────────────────────────────────┘ - │ - ┌───────────┴───────────┐ - 存在 不存在 - │ │ - ▼ ▼ - ┌─────────────────┐ ┌─────────────────────────┐ - │ 校验通过 │ │ 根据 validation-mode: │ - │ state=True │ │ - strict: 抛出异常 │ - └─────────────────┘ │ - warn: state=False │ - └─────────────────────────┘ -``` - ---- - -## 路径转换逻辑 - -将远程存储路径转换为本地 FUSE 路径，需要跳过 catalog 层级： - -```python -def _resolve_fuse_local_path(self, original_path: str) -> str: - """ - 将远程存储路径转换为本地 FUSE 路径。 - - 示例: - - 输入: oss://catalog/db1/table1 - - fuse_root: /mnt/fuse/warehouse - - 输出: /mnt/fuse/warehouse/db1/table1 - - 说明: FUSE 挂载点已映射到 catalog 层级，因此需要跳过路径中的 catalog 名称。 - - Args: - original_path: 原始远程存储路径 - - Returns: - 本地 FUSE 路径 - - Raises: - ValueError: 如果 fuse.local-path.root 未配置 - """ - if not self.fuse_local_path_root: - raise ValueError( - "FUSE local path is enabled but fuse.local-path.root is not configured" - ) - - uri = urlparse(original_path) - - # 提取路径部分 - # 有 scheme 时（如 oss://catalog/db/table）： - # - netloc 是 bucket 名（对应 catalog 名） - # - path 是剩余部分（如 /db/table） - # - 跳过 netloc，只保留 path - # 无 scheme 时（如 catalog/db/table）： - # - 跳过第一段（catalog 名） - if uri.scheme: - # 跳过 netloc（bucket/catalog），只保留 path 部分 - path_part = uri.path.lstrip('/') - else: - # 无 scheme：路径格式为 "catalog/db/table"，跳过第一段 - path_part = original_path.lstrip('/') - segments = path_part.split('/') - if len(segments) > 1: - path_part = '/'.join(segments[1:]) - - return f"{self.fuse_local_path_root.rstrip('/')}/{path_part}" -``` - ---- - -## 行为矩阵 - -| 配置 | 校验结果 | 行为 | -|-----|---------|------| -| `enabled=true, mode=strict` | 通过 | 使用 LocalFileIO | -| `enabled=true, mode=strict` | 失败 | 抛出 ValueError | -| `enabled=true, mode=warn` | 通过 | 使用 LocalFileIO | -| `enabled=true, mode=warn` | 失败 | 警告，回退到默认 FileIO | -| `enabled=true, mode=none` | - | 直接使用 LocalFileIO | -| `enabled=false` | - | 使用原有逻辑（data token 或 FileIO.get） | - ---- - -## 代码改动清单 - -### 1. 新增配置类 FuseOptions - -**文件**: `pypaimon/common/options/config.py` - -```python -class FuseOptions: - """FUSE 本地路径配置选项。""" - - FUSE_LOCAL_PATH_ENABLED = ( - ConfigOptions.key("fuse.local-path.enabled") - .boolean_type() - .default_value(False) - .with_description("是否启用 FUSE 本地路径映射") - ) - - FUSE_LOCAL_PATH_ROOT = ( - ConfigOptions.key("fuse.local-path.root") - .string_type() - .no_default_value() - .with_description("FUSE 挂载的本地根路径，如 /mnt/fuse") - ) - - FUSE_LOCAL_PATH_VALIDATION_MODE = ( - ConfigOptions.key("fuse.local-path.validation-mode") - .string_type() - .default_value("strict") - .with_description("校验模式：strict、warn 或 none") - ) -``` - -### 2. 修改 RESTCatalog - -**文件**: `pypaimon/catalog/rest/rest_catalog.py` - -#### 2.1 新增导入 - -```python -import logging -from urllib.parse import urlparse -from pypaimon.common.options.config import FuseOptions -from pypaimon.filesystem.local_file_io import LocalFileIO - -logger = logging.getLogger(__name__) -``` - -#### 2.2 修改 `__init__` 方法 - -```python -def __init__(self, context: CatalogContext, config_required: Optional[bool] = True): - # ... 原有初始化代码 ... - self.data_token_enabled = self.rest_api.options.get(CatalogOptions.DATA_TOKEN_ENABLED) - - # FUSE 本地路径配置 - self.fuse_local_path_enabled = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_ENABLED, False) - self.fuse_local_path_root = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_ROOT) - self.fuse_validation_mode = self.context.options.get( - FuseOptions.FUSE_LOCAL_PATH_VALIDATION_MODE, "strict") - self._fuse_validation_state = None # None=未校验, True=通过, False=失败 -``` - -#### 2.3 修改 `file_io_for_data` 方法 - -```python -def file_io_for_data(self, table_path: str, identifier: Identifier) -> FileIO: - """ - 获取用于数据访问的 FileIO，支持 FUSE 本地路径映射。 - """ - # 尝试使用 FUSE 本地路径 - if self.fuse_local_path_enabled: - # 配置错误直接抛异常 - local_path = self._resolve_fuse_local_path(table_path) - - # 执行校验（仅首次） - if self._fuse_validation_state is None: - self._validate_fuse_path() - - # 校验通过，返回本地 FileIO - if self._fuse_validation_state: - return LocalFileIO(local_path, self.context.options) - - # warn 模式校验失败后回退 - return RESTTokenFileIO(identifier, table_path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(table_path) - - # 回退到原有逻辑 - return RESTTokenFileIO(identifier, table_path, self.context.options) \ - if self.data_token_enabled else self.file_io_from_options(table_path) -``` - -#### 2.4 新增 `_resolve_fuse_local_path` 方法 - -```python -def _resolve_fuse_local_path(self, original_path: str) -> str: - """ - 解析 FUSE 本地路径。 - - FUSE 挂载点已映射到 catalog 层级，需要跳过路径中的 catalog 名称。 - - Returns: - 本地路径 - - Raises: - ValueError: 如果 fuse.local-path.root 未配置 - """ - if not self.fuse_local_path_root: - raise ValueError( - "FUSE local path is enabled but fuse.local-path.root is not configured" - ) - - uri = urlparse(original_path) - - # 提取路径部分 - # 有 scheme 时（如 oss://catalog/db/table）： - # - netloc 是 bucket 名（对应 catalog 名） - # - path 是剩余部分（如 /db/table） - # - 跳过 netloc，只保留 path - # 无 scheme 时（如 catalog/db/table）： - # - 跳过第一段（catalog 名） - if uri.scheme: - # 跳过 netloc（bucket/catalog），只保留 path 部分 - path_part = uri.path.lstrip('/') - else: - # 无 scheme：路径格式为 "catalog/db/table"，跳过第一段 - path_part = original_path.lstrip('/') - segments = path_part.split('/') - if len(segments) > 1: - path_part = '/'.join(segments[1:]) - - return f"{self.fuse_local_path_root.rstrip('/')}/{path_part}" -``` - -#### 2.5 新增 `_validate_fuse_path` 方法 - -```python -def _validate_fuse_path(self) -> None: - """ - 校验 FUSE 本地路径是否正确挂载。 - - 获取 default 数据库的 location，转换为本地路径后检查是否存在。 - """ - if self.fuse_validation_mode == "none": - self._fuse_validation_state = True - return - - # 获取 default 数据库详情，API 调用失败直接抛异常 - db = self.rest_api.get_database("default") - remote_location = db.location - - if not remote_location: - logger.info("Default database has no location, skipping FUSE validation") - self._fuse_validation_state = True - return - - expected_local = self._resolve_fuse_local_path(remote_location) - local_file_io = LocalFileIO(expected_local, self.context.options) - - # 只校验本地路径是否存在，根据 validation mode 处理 - if not local_file_io.exists(expected_local): - error_msg = ( - f"FUSE local path validation failed: " - f"local path '{expected_local}' does not exist " - f"for default database location '{remote_location}'" - ) - self._handle_validation_error(error_msg) - else: - self._fuse_validation_state = True - logger.info("FUSE local path validation passed") -``` - -**说明**： -- 只负责校验 FUSE 是否正确挂载，不处理配置错误 -- API 调用失败等系统异常直接抛出，不走校验模式 -- 直接调用 `get_database("default")` 获取 default 数据库 -- 将远端 location 转换为本地 FUSE 路径后检查是否存在 -- 简单高效，只需一次 API 调用 - -#### 2.6 新增 `_handle_validation_error` 方法 - -```python -def _handle_validation_error(self, error_msg: str) -> None: - """根据校验模式处理错误。""" - if self.fuse_validation_mode == "strict": - raise ValueError(error_msg) - elif self.fuse_validation_mode == "warn": - logger.warning(f"{error_msg}. Falling back to default FileIO.") - self._fuse_validation_state = False # 标记校验失败，回退到默认 FileIO -``` - ---- - -## 文件结构 - -``` -paimon-python/ -├── pypaimon/ -│ ├── catalog/ -│ │ └── rest/ -│ │ └── rest_catalog.py # 修改：添加 FUSE 支持 -│ └── common/ -│ └── options/ -│ └── config.py # 修改：添加 FuseOptions -``` - ---- - -## 与完整版设计的区别 - -| 特性 | 简化版 | 完整版 | -|------|--------|--------| -| 挂载级别 | 仅 Catalog 级别 | Catalog / Database / Table 三级 | -| 配置项 | 3 个 | 7+ 个 | -| 校验方式 | 检查 default 数据库路径存在性 | 双重校验（标识文件 + 数据哈希） | -| 错误处理 | 基本模式 | FUSE 特有错误重试机制 | -| 复杂度 | 低 | 高 | - ---- - -## 测试用例 - -### 1. 基本功能测试 - -```python -def test_fuse_local_path_basic(): - """测试基本 FUSE 本地路径功能""" - options = { - 'metastore': 'rest', - 'uri': 'http://localhost:8080', - 'token': 'xxx', - 'fuse.local-path.enabled': 'true', - 'fuse.local-path.root': '/mnt/fuse/warehouse', - } - catalog = Catalog.create(options) - # 验证 FUSE 配置已加载 - assert catalog.fuse_local_path_enabled == True -``` - -### 2. 校验模式测试 - -```python -def test_validation_mode_strict(): - """测试 strict 模式校验失败抛出异常""" - # 配置不存在的 FUSE 路径 - # 预期抛出 ValueError - -def test_validation_mode_warn(): - """测试 warn 模式校验失败回退""" - # 配置不存在的 FUSE 路径 - # 预期使用默认 FileIO - -def test_validation_mode_none(): - """测试 none 模式跳过校验""" - # 配置不存在的 FUSE 路径 - # 预期直接使用 LocalFileIO -``` - -### 3. 边界条件测试 - -```python -def test_default_db_no_location(): - """测试 default 数据库没有 location 的情况""" - -def test_default_db_not_exist(): - """测试 default 数据库不存在的情况""" - -def test_resolve_fuse_path_missing_root(): - """测试启用 FUSE 但未配置 root 时报错""" - -def test_disabled_fuse(): - """测试 FUSE 未启用时使用默认逻辑""" -``` - -### 4. `_resolve_fuse_local_path` 路径转换测试 - -```python -def test_resolve_fuse_local_path_basic(): - """测试基本路径转换""" - # 输入: oss://catalog/db1/table1 - # fuse_root: /mnt/fuse/warehouse - # 预期输出: /mnt/fuse/warehouse/db1/table1 - -def test_resolve_fuse_local_path_with_trailing_slash(): - """测试 fuse_root 带尾部斜杠""" - # 输入: oss://catalog/db1/table1 - # fuse_root: /mnt/fuse/warehouse/ - # 预期输出: /mnt/fuse/warehouse/db1/table1 - -def test_resolve_fuse_local_path_deep_path(): - """测试深层路径""" - # 输入: oss://catalog/db1/table1/partition1/file.parquet - # fuse_root: /mnt/fuse/warehouse - # 预期输出: /mnt/fuse/warehouse/db1/table1/partition1/file.parquet - -def test_resolve_fuse_local_path_without_scheme(): - """测试路径没有 scheme""" - # 输入: catalog/db1/table1 - # fuse_root: /mnt/fuse/warehouse - # 预期输出: /mnt/fuse/warehouse/db1/table1 -```