Summary
Improve Gmail body extraction so text and HTML message parts are handled correctly when their content is attachment-backed and when they declare a non-UTF-8 charset.
Today, the shared _extract_message_bodies helper in gmail/gmail_tools.py only consumes inline body.data and decodes it as UTF-8 with ignored errors. As a result, text/HTML body parts exposed through body.attachmentId can be omitted, and content in declared non-UTF-8 charsets can be corrupted in tools that rely on this helper.
Required changes
- Update the shared Gmail body extraction path to resolve body parts that provide
body.attachmentId through the Gmail attachments API.
- Decode text/plain and text/html payloads using the charset declared by the MIME part when available, with safe fallback behavior for missing or invalid charset declarations.
- Preserve existing inline
body.data extraction behavior.
- Apply the shared behavior consistently to:
get_gmail_message_content
get_gmail_messages_content_batch
get_gmail_thread_content
get_gmail_threads_content_batch
get_gmail_message_full for deliver_as="html" and deliver_as="txt"
Rationale
This is inherited shared-helper behavior rather than a narrow full-export concern. Fixing it in one focused change allows the contract and test coverage to be reviewed consistently across every Gmail body-reading tool.
get_gmail_message_full(deliver_as="eml") remains the byte-exact, lossless path because it exports Gmail's raw RFC 5322 bytes directly. The HTML/TXT modes are convenience representations and should nevertheless correctly retrieve body content and honor declared character encodings.
Acceptance criteria
Backlinks
Summary
Improve Gmail body extraction so text and HTML message parts are handled correctly when their content is attachment-backed and when they declare a non-UTF-8 charset.
Today, the shared
_extract_message_bodieshelper ingmail/gmail_tools.pyonly consumes inlinebody.dataand decodes it as UTF-8 with ignored errors. As a result, text/HTML body parts exposed throughbody.attachmentIdcan be omitted, and content in declared non-UTF-8 charsets can be corrupted in tools that rely on this helper.Required changes
body.attachmentIdthrough the Gmail attachments API.body.dataextraction behavior.get_gmail_message_contentget_gmail_messages_content_batchget_gmail_thread_contentget_gmail_threads_content_batchget_gmail_message_fullfordeliver_as="html"anddeliver_as="txt"Rationale
This is inherited shared-helper behavior rather than a narrow full-export concern. Fixing it in one focused change allows the contract and test coverage to be reviewed consistently across every Gmail body-reading tool.
get_gmail_message_full(deliver_as="eml")remains the byte-exact, lossless path because it exports Gmail's raw RFC 5322 bytes directly. The HTML/TXT modes are convenience representations and should nevertheless correctly retrieve body content and honor declared character encodings.Acceptance criteria
text/plainandtext/htmlbody parts are retrieved and surfaced by all affected tools.body.databehavior remains supported.Backlinks