feat(feishu): persist imported document images - #3033
Conversation
|
Please add Feishu-specific verification details:
This will make it easier to verify that the Feishu image-token parsing and rewrite flow covers the real exported document format. |
Download Feishu image tokens into local import temp trees so Markdown image references can be ingested alongside the document. Co-authored-by: Cursor <cursoragent@cursor.com>
866b2a1 to
30d3dda
Compare
|
@qin-ctx Hi, I've uploaded the screenshot to the Screenshots (if applicable) section above. Feishu-image ingestion deliberately reuses the existing local-file image pipeline. The only Feishu-specific work happens inside the accessor: it downloads each Path comparison
Concrete before / after for a Feishu document1. Converter output (Markdown as produced from the Feishu doc): 2. After the accessor downloads and rewrites — 3. After This is byte-for-byte the same shape a plain local Notes
|
qin-ctx
left a comment
There was a problem hiding this comment.
I found two blocking issues in the Feishu image import path: the new media downloads run synchronously inside the async accessor, and downloaded images are always named as .png even when the bytes may be JPEG/WebP/GIF. Please address these before merge.
4d30945 to
30d3dda
Compare
qin-ctx
left a comment
There was a problem hiding this comment.
I found one blocking issue in the Feishu image import path: user-token imports can read the document body but the new media download request only allows tenant-token auth, so inline images are not persisted for the same import mode.
Address review feedback on the inline-image import path: - Run the synchronous lark-oapi media download via asyncio.to_thread so a slow Feishu request no longer blocks unrelated async work on the event loop, matching the existing _fetch_document() pattern. - Infer the image file extension from the downloaded bytes (byte-magic sniffing) and fall back to the response Content-Type via the existing mime_types.get_preferred_extension helper, instead of hardcoding .png. This stops JPEG/WebP/GIF bytes from being mislabeled as PNG to downstream consumers (e.g. the data:image/... URI built during multimodal vectorization). - Advertise AccessTokenType.USER on the media download request when a user access token is supplied, so lark-oapi actually injects it. Previously the request only allowed TENANT, so user-token imports read the document body but silently dropped every image. Co-authored-by: Cursor <cursoragent@cursor.com>
1c762b1 to
30d3dda
Compare
|
@qin-ctx Thanks for the thorough review.!All three blocking issues are addressed in the latest commit:
Added unit tests covering extension detection (Content-Type + byte magic), the event-loop offload, and both the tenant- and user-token request paths. One note on the byte-magic sniffing: I'm aware this logic is conceptually duplicated across the codebase — http_accessor._detect_from_magic_bytes and the three VLM backends' _detect_image_format all sniff image magic bytes. I deliberately kept a small local copy here rather than reuse them, because none is directly reusable: http_accessor returns (URLType, ext) tuples tied to its URL-classification flow, and the VLM helpers return format-name strings for API payloads — neither maps cleanly to "bytes → file extension." Factoring these into a shared helper (e.g. mime_types.sniff_image_extension) and migrating all call sites feels out of scope for this PR and touches unrelated modules (VLM backends). I'd suggest doing that consolidation in a dedicated follow-up PR. Happy to file it if you agree. |

Description
Downloads and persists inline Feishu document images during import. Feishu image references such as
feishu://image/{file_token}are resolved into local temporary image files before Markdown parsing, so the existing Markdown image ingestion flow can store them in VikingFS and rewrite references to stableviking://image URIs.Related Issue
Fixes #3030
Type of Change
Changes Made
feishu://image/{file_token}Markdown references.document.md._cleanup_path.feishu.download_imagesconfiguration support.Testing
Test coverage added/updated:
tests/parse/test_feishu_accessor.pyManual validation:
Checklist
Screenshots (if applicable)
Additional Notes
No public API change is required. Image downloading is controlled by the Feishu parser configuration via
download_images.