feat: add Feishu/Lark image support using doc2markdown by bittoby · Pull Request #6511 · labring/FastGPT

bittoby · 2026-03-05T20:59:42Z

Closes: #5998

Summary

When syncing Feishu/Lark documents into a knowledge base, images were lost because the old code used the /raw_content API which only returns plain text. This PR adds doc2markdown to fetch documents via the Block API, download embedded images, and upload them to S3.

What changed

Image support for Feishu/Lark docs: Documents are now converted to markdown with images preserved. Images are downloaded from Feishu and stored in MinIO/S3.
No duplicate images: S3 filenames are based on the image's resource token, so re-importing the same document won't create duplicates.
Proper cleanup on deletion: Deleting a Feishu collection from a dataset now removes its images from S3.

test.webm

Files changed

packages/service/core/dataset/apiDataset/feishuDataset/api.ts - main implementation
packages/service/core/dataset/collection/controller.ts - S3 cleanup on collection delete
packages/service/package.json - added doc2markdown dependency
.prettierignore - ignore deploy/ (Docker volume permission issues)

How to test

Import a Feishu/Lark doc with images and check that images show up in chunks
Import the same doc again - no new duplicate images in MinIO
Delete the collection - images should be removed from MinIO
Delete the whole dataset - everything cleaned up

cla-assistant · 2026-03-05T20:59:58Z

All committers have signed the CLA.

cla-assistant · 2026-03-05T20:59:59Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions · 2026-03-05T21:01:27Z

Preview sandbox Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_sandbox_68702d4261a6dd3b0d98444ad73f77db5c3ff3fa

github-actions · 2026-03-05T21:02:50Z

Preview mcp_server Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_mcp_server_68702d4261a6dd3b0d98444ad73f77db5c3ff3fa

github-actions · 2026-03-05T21:11:47Z

Preview fastgpt Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_68702d4261a6dd3b0d98444ad73f77db5c3ff3fa

…eishu-image-support # Conflicts: # packages/service/package.json # pnpm-lock.yaml

github-actions · 2026-03-06T11:30:10Z

Docs Preview:

🚀 FastGPT Document Preview Ready!

🔗 👀 Click here to visit preview

bittoby · 2026-03-06T11:46:08Z

@c121914yu @FinleyGe could you please review this PR? I'd appreciate your feedback.

c121914yu · 2026-03-06T11:59:00Z

packages/service/core/dataset/apiDataset/feishuDataset/api.ts

 const feishuBaseUrl = process.env.FEISHU_BASE_URL || 'https://open.feishu.cn';
 const logger = getLogger(LogCategories.MODULE.DATASET.API_DATASET);

+const uploadLocalFileToS3 = async ({


The S3 code requires that the upload and acquisition of the prefix be uniformly carried out within the s3 instance of the corresponding module, and it is not allowed to define the prefix at will

c121914yu · 2026-03-06T12:01:39Z

This design scheme is not appropriate for image processing. The image processing methods of the knowledge base should be fully reused.

bittoby · 2026-03-06T12:02:17Z

Thanks for your feedback. I will consider again

…eishu-image-support

bittoby · 2026-03-06T15:31:54Z

@c121914yu I updated to use existing image process pipeline. And I tested again and confirmed it is working well.
I'd appreciate you review again.

…eishu-image-support

bittoby · 2026-03-13T14:01:27Z

@c121914yu @FinleyGe Please review this. All tests passed and the Feishu/Lark image support function works well. I attached test video in the PR description. I appreciate you review again

packages/service/core/dataset/apiDataset/feishuDataset/api.ts

packages/service/core/dataset/apiDataset/feishuDataset/feishuDocToMarkdown.ts

packages/service/package.json

…ve baseUrl - Upgrade doc2markdown to ^1.3.2 which supports baseUrl natively, removing all monkey-patching of internal methods (199 → 82 lines) - Extract shared uploadMdImagesToS3 helper to eliminate duplicate image upload logic between read.ts and file/read/utils.ts - Add deploy/ to .prettierignore to fix permission denied errors

bittoby · 2026-03-16T14:01:36Z

@AntiMoron Thanks for your feedback. I updated all. Please review again

bittoby · 2026-03-16T17:46:27Z

@c121914yu Hope you had a great weekend!
please review this PR. would appreciate to merge this.
thank you

AntiMoron

LGTM

bittoby · 2026-03-18T13:10:18Z

@c121914yu I'd appreciate your feedback.

bittoby · 2026-03-24T12:50:22Z

@c121914yu I submitted this PR a while ago. what else should I need to update more?

c121914yu · 2026-03-24T13:31:06Z

@c121914yu I submitted this PR a while ago. what else should I need to update more?

We have received your message. We will merge this pr at an appropriate time. You don't need to keep resolving pr conflicts

feat: add Feishu/Lark image support using doc2markdown

5a0fa99

pull-request-size bot added the size/L label Mar 5, 2026

fix: add comment

f1f8dfd

bittoby force-pushed the feat/feishu-image-support branch from 37c53de to f1f8dfd Compare March 5, 2026 21:06

Merge branch 'main' of https://github.com/bittoby/FastGPT into feat/f…

c040ce9

…eishu-image-support # Conflicts: # packages/service/package.json # pnpm-lock.yaml

fix: solve conflcit

e2ad221

c121914yu reviewed Mar 6, 2026

View reviewed changes

bittoby added 3 commits March 6, 2026 13:12

Merge branch 'main' of https://github.com/bittoby/FastGPT into feat/f…

7282cf8

…eishu-image-support

fix: update redesign to reuse original image processing workflow

cfe7b33

update read.ts file to use same image processing pipeline

81ce2c7

bittoby requested a review from c121914yu March 6, 2026 16:39

bittoby added 5 commits March 9, 2026 12:44

fix: solve conflict

9da4d78

fix: solve merge conflicts

3f28d92

fix: solve merge conflicts

c4a0eb0

Merge branch 'main' of https://github.com/bittoby/FastGPT into feat/f…

ba6e23e

…eishu-image-support

feat: add Feishu/Lark document image support in knowledge base

e54f221

AntiMoron reviewed Mar 16, 2026

View reviewed changes

packages/service/core/dataset/apiDataset/feishuDataset/api.ts Outdated Show resolved Hide resolved

packages/service/core/dataset/apiDataset/feishuDataset/feishuDocToMarkdown.ts Show resolved Hide resolved

packages/service/package.json Outdated Show resolved Hide resolved

bittoby added 2 commits March 16, 2026 10:03

fix: solve merge conflicts

68702d4

bittoby requested a review from AntiMoron March 16, 2026 14:01

AntiMoron reviewed Mar 17, 2026

View reviewed changes

bittoby added 3 commits March 19, 2026 04:31

fix: solve conflicts

e3e8340

fix: solve conflicts

c3bd457

fix: solve conflicts

e6c0a1d

Conversation

bittoby commented Mar 5, 2026

Closes: #5998

Summary

What changed

Files changed

How to test

Uh oh!

cla-assistant bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cla-assistant bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview sandbox Image:

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview mcp_server Image:

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview fastgpt Image:

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Docs Preview:

Uh oh!

bittoby commented Mar 6, 2026

Uh oh!

c121914yu Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

c121914yu commented Mar 6, 2026

Uh oh!

bittoby commented Mar 6, 2026

Uh oh!

bittoby commented Mar 6, 2026

Uh oh!

bittoby commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bittoby commented Mar 16, 2026

Uh oh!

bittoby commented Mar 16, 2026

Uh oh!

AntiMoron left a comment

Choose a reason for hiding this comment

Uh oh!

bittoby commented Mar 18, 2026

Uh oh!

bittoby commented Mar 24, 2026

Uh oh!

c121914yu commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cla-assistant bot commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 6, 2026 •

edited

Loading

bittoby commented Mar 13, 2026 •

edited

Loading