[POSTMAN-UNSUPPORTED-ATTACHMENT-1] PLU-482: Filter unsupported Postman attachments before sending (#1019)

### TL;DR

Added validation for email attachments in Postman to filter out unsupported file types and notify users.

**Key things to note**:
1. Unsupported attachments: file type(s) that Postman does not accept
1. Password-protected: files that are password-protected and rejected by Postman
1. Postman throws the same "invalid_template" error regardless of whether they are unsupported or password-protected attachments

_Retry to strip attachments will be in the next PR_

### What changed?

- Created a list of accepted file extensions for Postman email attachments (https://postman-v1.guides.gov.sg/email-api-guide/programmatic-email-api/send-email-api/attachments#supported-attachment-file-types)
- Implemented a filtering mechanism to check attachments before sending emails
- Added functionality to send notification emails to users when unsupported attachments are detected
- Enhanced error handling to provide clear guidance when attachments are removed
- Created a form admin link to help users access their original attachments
- Update error message for password-protected files
  - since we use the same list as postman to filter attachments, invalid_template should only throw when users attempt to send password-protected attachments

### Why make this change?

Postman has restrictions on which file types can be sent as email attachments. Previously, users are unable to send emails when attachments include unsupported file types. This change improves the user experience by:

1. Proactively filtering out unsupported attachments
2. Clearly communicating which files were removed and why
3. Providing a way for users to access their original files through the form admin interface



### How to test?
Pre-setup: prepare a form that accepts attachments and connect this form to your pipe + turn on notifications for every failed execution
Make form(s) submission with varying types of attachments:
  1.  supported attachment
  1. unsupported attachment e.g., `svg`
  1. password-protected attachment

There are 3 types of notification emails:
1. Error (failed) execution
1. Blacklisted recipient notification
1. Unsupported attachment notification

**Test cases**
- [ ] Blacklisted recipient (can use `[email protected]`)
  - [ ] Same behaviour as current implementation
  - [ ] Blacklisted recipient notification sent (same as current behaviour)
- [ ] Unsupported attachment
  - [ ] Execution marked as partial success
  - [ ] Email sent without unsupported attachment
  - [ ] Unsupported attachment notification (new) sent with details of unsupported attachment, submission id, execution id, link to form admin
  - [ ] Execution page shows link and opens to form responses
- [ ] Password-protected attachment
  - [ ] Execution fails with "Password-protected" error
  - [ ] Error notification sent
  - [ ] Execution page shows error with "Password-protected attachment(s)"
- [ ] Blacklisted + Unsupported
  - [ ] Execution partial success
  - [ ] Email not sent to blacklisted recipient
  - [ ] Email sent to non-blacklisted recipient without the unsupported attachment
  - [ ] **TWO** notificiation email sent
    - [ ] Blacklisted recipient notification (same as current behaviour)
    - [ ] Unsupported attachment notification (new)
- [ ] Blacklisted + Unsupported + Password-protected
  - [ ] Execution fails
  - [ ] Error notification sent
- [ ] Unsupported + Password-protected
  - [ ] Execution fails
  - [ ] Error notification sent
  - [ ] Execution page shows error with "Password-protected attachment(s)"


### Screenshots
**Execution with unsupported attachment**

[Screen Recording 2025-06-04 at 11.29.32 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/Wrwhm8Mmhv1Z2GwlSb7V/7ff6ca5a-5fe7-45e4-9938-d9420325b56a.mov" />](https://app.graphite.dev/media/video/Wrwhm8Mmhv1Z2GwlSb7V/7ff6ca5a-5fe7-45e4-9938-d9420325b56a.mov)


![Screenshot 2025-06-04 at 11.29.56 AM.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/Wrwhm8Mmhv1Z2GwlSb7V/1108a42f-63c2-4abd-bd32-1cab6c5a4104.png)


![Screenshot 2025-06-04 at 11.30.10 AM.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/Wrwhm8Mmhv1Z2GwlSb7V/79431763-e73b-4566-ac4c-4703b0b3408c.png)

**Execution with unsupported attachment and blacklisted recipient**

[Screen Recording 2025-06-04 at 11.32.02 AM.mov <span class="graphite__hidden">(uploaded via Graphite)</span> <img class="graphite__hidden" src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/Wrwhm8Mmhv1Z2GwlSb7V/e4791c22-085b-4bc9-9847-393bf58a92c1.mov" />](https://app.graphite.dev/media/video/Wrwhm8Mmhv1Z2GwlSb7V/e4791c22-085b-4bc9-9847-393bf58a92c1.mov)


**Password-protected attachments**

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/Wrwhm8Mmhv1Z2GwlSb7V/8e1b8252-bf56-4768-b377-fd4f9dd97e38.png)

Author: kevinkim-ogp

packages/backend/src/apps/postman/__tests__/actions/send-transactional-email.test.ts

@@ -12,7 +12,16 @@ import sendTransactionalEmail from '../../actions/send-transactional-email'
 const mocks = vi.hoisted(() => ({
   getObjectFromS3Id: vi.fn(),
   getDefaultReplyTo: vi.fn(() => '[email protected]'),
+  filterAttachments: vi.fn(() => {
+    return {
+      attachmentFiles: [],
+      invalidAttachments: [],
+      submissionId: null,
+    }
+  }),
   sendBlacklistEmail: vi.fn(),
+  sendInvalidAttachmentsEmail: vi.fn(),
+  createInvalidAttachmentsMessage: vi.fn(() => 'test invalid attachment body'),
 }))
 
 vi.mock('@/helpers/s3', async () => {
@@ -28,13 +37,19 @@ vi.mock('@/helpers/s3', async () => {
 
 vi.mock('../../common/parameters-helper', () => ({
   getDefaultReplyTo: mocks.getDefaultReplyTo,
+  filterAttachments: mocks.filterAttachments,
 }))
 
 vi.mock('../../common/send-blacklist-email', () => ({
   sendBlacklistEmail: mocks.sendBlacklistEmail,
   createRequestBlacklistFormLink: vi.fn(),
 }))
 
+vi.mock('../../common/send-invalid-attachments-email', () => ({
+  sendInvalidAttachmentsEmail: mocks.sendInvalidAttachmentsEmail,
+  createInvalidAttachmentsMessage: mocks.createInvalidAttachmentsMessage,
+}))
+
 describe('send transactional email', () => {
   let $: IGlobalVariable
 
@@ -137,7 +152,7 @@ describe('send transactional email', () => {
       },
       errorStatusCode: 400,
       errorStatusText: 'Bad Request',
-      stepErrorName: 'Unsupported attachment file type',
+      stepErrorName: 'Password-protected attachment(s)',
     },
   ])(
     'should throw step error for different postman errors',
@@ -570,4 +585,112 @@ describe('send transactional email', () => {
       },
     })
   })
+
+  it('should filter out invalid attachments and send notification email', async () => {
+    const recipients = ['[email protected]', '[email protected]']
+    $.step.parameters.destinationEmail = recipients.join(',')
+    $.step.parameters.attachments = [
+      's3:my-bucket:abcd/file 1.txt',
+      's3:my-bucket:wxyz/file-2.svg',
+    ]
+    mocks.filterAttachments.mockResolvedValueOnce({
+      attachmentFiles: [],
+      invalidAttachments: ['file-2.svg'],
+      submissionId: 'abc',
+    })
+    await expect(sendTransactionalEmail.run($)).rejects.toThrowError(
+      PartialStepError,
+    )
+    expect($.http.post).toBeCalledTimes(2)
+    expect(mocks.sendInvalidAttachmentsEmail).toHaveBeenCalledWith({
+      flowName: $.flow.name,
+      flowId: $.flow.id,
+      userEmail: $.user.email,
+      executionId: $.execution.id,
+      submissionId: 'abc',
+      invalidAttachments: ['file-2.svg'],
+      hasInvalidAttachments: true,
+    })
+    expect($.setActionItem).toHaveBeenCalledWith({
+      raw: {
+        status: ['ACCEPTED', 'ACCEPTED'],
+        recipient: recipients,
+        subject: 'test subject',
+        body: 'test body',
+        from: 'jack',
+        reply_to: '[email protected]',
+      },
+    })
+  })
+
+  it('should send two emails if there are blacklisted recipients and invalid attachments', async () => {
+    const recipients = ['[email protected]', '[email protected]']
+    $.step.parameters.destinationEmail = recipients.join(',')
+    $.step.parameters.attachments = [
+      's3:my-bucket:abcd/file 1.txt',
+      's3:my-bucket:wxyz/file-2.svg',
+    ]
+
+    mocks.filterAttachments.mockResolvedValueOnce({
+      attachmentFiles: [],
+      invalidAttachments: ['file-2.svg'],
+      submissionId: 'abc',
+    })
+
+    $.http.post = vi
+      .fn()
+      .mockResolvedValueOnce({
+        data: {
+          params: {
+            body: 'test body',
+            subject: 'test subject',
+            from: 'jack',
+            reply_to: '[email protected]',
+          },
+        },
+      })
+      .mockRejectedValueOnce(
+        new HttpError({
+          response: {
+            data: {
+              code: 'invalid_template',
+              message: 'Recipient email is blacklisted',
+            },
+            status: 400,
+            statusText: 'Bad Request',
+          },
+        } as AxiosError),
+      )
+
+    await expect(sendTransactionalEmail.run($)).rejects.toThrowError(
+      PartialStepError,
+    )
+    expect($.http.post).toBeCalledTimes(2)
+    expect($.setActionItem).toHaveBeenCalledWith({
+      raw: {
+        status: ['ACCEPTED', 'BLACKLISTED'],
+        recipient: recipients,
+        subject: 'test subject',
+        body: 'test body',
+        from: 'jack',
+        reply_to: '[email protected]',
+      },
+    })
+    expect(mocks.sendBlacklistEmail).toHaveBeenCalledWith({
+      flowName: $.flow.name,
+      flowId: $.flow.id,
+      userEmail: $.user.email,
+      executionId: $.execution.id,
+      blacklistedRecipients: [recipients[1]],
+    })
+    expect(mocks.sendInvalidAttachmentsEmail).toHaveBeenCalledWith({
+      flowName: $.flow.name,
+      flowId: $.flow.id,
+      userEmail: $.user.email,
+      executionId: $.execution.id,
+      submissionId: 'abc',
+      invalidAttachments: ['file-2.svg'],
+      hasInvalidAttachments: true,
+    })
+  })
 })

packages/backend/src/apps/postman/actions/send-transactional-email/index.ts

@@ -5,7 +5,6 @@ import { fromZodError } from 'zod-validation-error'
 
 import StepError from '@/errors/step'
 import logger from '@/helpers/logger'
-import { getObjectFromS3Id } from '@/helpers/s3'
 import Step from '@/models/step'
 
 import { dataOutSchema } from '../../common/data-out-validator'
@@ -14,8 +13,12 @@ import {
   transactionalEmailFields,
   transactionalEmailSchema,
 } from '../../common/parameters'
-import { getDefaultReplyTo } from '../../common/parameters-helper'
+import {
+  filterAttachments,
+  getDefaultReplyTo,
+} from '../../common/parameters-helper'
 import { sendBlacklistEmail } from '../../common/send-blacklist-email'
+import { sendInvalidAttachmentsEmail } from '../../common/send-invalid-attachments-email'
 import { throwPostmanStepError } from '../../common/throw-errors'
 
 const action: IRawAction = {
@@ -71,18 +74,8 @@ const action: IRawAction = {
       )
     }
 
-    const attachmentFiles = await Promise.all(
-      result.data.attachments?.map(async (attachment) => {
-        // We verify the flowId here to ensure that the attachment is from the same flow and not
-        // maliciously/ manually injected by another user who does not have access to this attachment
-        const obj = await getObjectFromS3Id(
-          attachment,
-          { flowId: $.flow.id },
-          $,
-        )
-        return { fileName: obj.name, data: obj.data }
-      }),
-    )
+    const { attachmentFiles, invalidAttachments, submissionId } =
+      await filterAttachments(result.data.attachments, $)
 
     let recipientsToSend = result.data.destinationEmail
     /**
@@ -165,16 +158,27 @@ const action: IRawAction = {
       (_, i) => dataOut.status[i] === 'BLACKLISTED',
     )
 
+    const defaultSendEmailParams = {
+      flowId: $.flow.id,
+      flowName: $.flow.name,
+      userEmail: $.user.email,
+      executionId: $.execution.id,
+    }
+    const hasInvalidAttachments = invalidAttachments.length > 0
+    const invalidAttachmentParams = {
+      hasInvalidAttachments,
+      submissionId,
+      invalidAttachments,
+    }
+
     /**
      * Send blacklist notification email if any
+     * If there are any invalid attachments, it will be included in this email
      */
     if (blacklistedRecipients.length > 0 && !$.execution.testRun) {
       try {
         await sendBlacklistEmail({
-          flowId: $.flow.id,
-          flowName: $.flow.name,
-          userEmail: $.user.email,
-          executionId: $.execution.id,
+          ...defaultSendEmailParams,
           blacklistedRecipients,
         })
       } catch (e) {
@@ -188,17 +192,36 @@ const action: IRawAction = {
         })
       }
     }
+
+    /**
+     * Send invalid attachments notification email
+     * Do not send on partial retry as we would have already sent this once with the blacklist email
+     */
+    if (hasInvalidAttachments && !isPartialRetry && !$.execution.testRun) {
+      await sendInvalidAttachmentsEmail({
+        ...defaultSendEmailParams,
+        ...invalidAttachmentParams,
+      })
+
+      logger.warn({
+        message: 'Invalid attachments',
+        flowId: $.flow.id,
+        executionId: $.execution.id,
+        invalidAttachments: invalidAttachments.join(', '),
+      })
+    }
     /**
      * If there's any rate-limit error, we will throw the rate-limit error
      * else we just throw the first error we encounter
      */
-    if (error && errorStatus) {
+    if ((error && errorStatus) || invalidAttachments.length > 0) {
       throwPostmanStepError({
         $,
         status: errorStatus,
         error,
-        isPartialSuccess: hasAtLeastOneSuccess,
+        isPartialSuccess: hasAtLeastOneSuccess || invalidAttachments.length > 0,
         blacklistedRecipients,
+        invalidAttachments,
       })
     }
   },

packages/backend/src/apps/postman/common/constants.ts

@@ -0,0 +1,38 @@
+export const POSTMAN_ACCEPTED_EXTENSIONS = [
+  'txt',
+  'asc',
+  'avi',
+  'bmp',
+  'csv',
+  'dgn',
+  'docx',
+  'dwf',
+  'dwg',
+  'dxf',
+  'ent',
+  'gif',
+  'jpg',
+  'jpeg',
+  'mpeg',
+  'mpg',
+  'mpp',
+  'odb',
+  'odf',
+  'odg',
+  'ods',
+  'pdf',
+  'png',
+  'pptx',
+  'rtf',
+  'sxc',
+  'sxd',
+  'sxi',
+  'sxw',
+  'tif',
+  'tiff',
+  'wmv',
+  'xlsx',
+]
+
+export const POSTMAN_SUPPORTED_ATTACHMENTS_GUIDE_URL =
+  'https://postman-v1.guides.gov.sg/email-api-guide/programmatic-email-api/send-email-api/attachments#list-of-supported-attachment-file-types'

packages/backend/src/apps/postman/common/parameters-helper.ts

@@ -1,9 +1,47 @@
+import { IGlobalVariable } from '@plumber/types'
+
+import { COMMON_S3_BUCKET, getObjectFromS3Id } from '@/helpers/s3'
 import Flow from '@/models/flow'
 
+import { POSTMAN_ACCEPTED_EXTENSIONS } from './constants'
+
 export async function getDefaultReplyTo(flowId: string): Promise<string> {
   const flow = await Flow.query()
     .findById(flowId)
     .withGraphFetched({ user: true })
     .throwIfNotFound()
   return flow.user.email
 }
+
+export async function filterAttachments(
+  attachmentsList: string[],
+  $: IGlobalVariable,
+) {
+  let submissionId: string | null = null
+  const invalidAttachments: string[] = []
+  const attachmentFiles: { fileName: string; data: Uint8Array }[] = []
+
+  await Promise.all(
+    attachmentsList?.map(async (attachment) => {
+      // We verify the flowId here to ensure that the attachment is from the same flow and not
+      // maliciously/ manually injected by another user who does not have access to this attachment
+      const obj = await getObjectFromS3Id(attachment, { flowId: $.flow.id }, $)
+      const fileName = obj.name
+      const fileType = obj.name.split('.').pop()?.toLowerCase()
+      if (!fileType || !POSTMAN_ACCEPTED_EXTENSIONS.includes(fileType)) {
+        invalidAttachments.push(fileName)
+
+        if (submissionId === null) {
+          submissionId = attachment.slice(
+            `s3:${COMMON_S3_BUCKET}:`.length,
+            attachment.indexOf('/', `s3:${COMMON_S3_BUCKET}:`.length),
+          )
+        }
+      } else {
+        attachmentFiles.push({ fileName, data: obj.data })
+      }
+      return
+    }),
+  )
+  return { attachmentFiles, invalidAttachments, submissionId }
+}

packages/backend/src/apps/postman/common/parameters.ts

@@ -6,6 +6,8 @@ import { z } from 'zod'
 
 import { parseS3Id } from '@/helpers/s3'
 
+import { POSTMAN_SUPPORTED_ATTACHMENTS_GUIDE_URL } from './constants'
+
 function recipientStringToArray(value: string) {
   const recipientArray = value
     .split(',')
@@ -81,8 +83,7 @@ export const transactionalEmailFields: IField[] = [
   {
     label: 'Attachments',
     key: 'attachments',
-    description:
-      'Check supported file types [here](https://postman-v1.guides.gov.sg/email-api-guide/programmatic-email-api/send-email-api/attachments#list-of-supported-attachment-file-types).\nPlease note that the maximum file size for each file is 2MB, and the total size of all attachments cannot exceed 10MB.',
+    description: `Check supported file types [here](${POSTMAN_SUPPORTED_ATTACHMENTS_GUIDE_URL}).\nPlease note that the maximum file size for each file is 2MB, and the total size of all attachments cannot exceed 10MB.`,
     type: 'attachment' as const,
     required: false,
     variables: true,