Roadmap overview

Milestone 2: Missing things

2/7. 🔐 Authentication

Add VMAIL_AUTH_MODE config option: "dev" (stub auth) or "header" (trusts reverse proxy headers). In header mode, backend reads Remote-Email/Remote-User headers from Caddy's forward_auth.
Update frontend to redirect to Authelia on 401 and remove hardcoded tokens.
Update WebSocket handler to support header-based auth.

2/8. 🔐 Proper basic functionality

Attachments are not always displayed. Make sure they are displayed correctly.
Sent emails are not part of threads in Inbox. Make sure they are included. I guess same for vice versa. Add backend test to cover this.
Rewrite /scripts/check.sh in Go because the logic is too complex now. Also, make it run E2E tests just once, and log it when failed, AND run gofmt and pnpm lint:fix automatically.
Write docs for how to run the app in dev mode, and in general to run it with a single command, forking to Go and the frontend, with Go having live reload. These things would be nice.
Ensure line-to-line debugging is possible for Go
Add a "Test IMAP and SMTP connections" button to the Settings page.

Milestone 3: Actions

Goal: Be able to manage email.
Tasks: Implement Archive, Star, Trash (both frontend and backend). Implement the search bar UI to call the search API.

This milestone is all about adding actions. We'll build them one by one. The "Star" action is the simplest, so we'll start there. We'll use the action_queue for all of them to make the UI feel instant.

3/1. 🏗️ Backend: Action worker

Before we can queue any actions, we need to build the worker that processes them.

Create the worker service:
- In /backend/internal/sync, create a new worker.go file.
- Create a struct for your worker, e.g., type ActionWorker struct { db *pgxpool.Pool, imapService *imap.Service }. (You'll need to create an imap.Service struct in your /internal/imap folder to hold your IMAP logic).
Create the worker loop:
- In worker.go, create a Start() method. This method should be run as a goroutine from your main.go.
- Inside Start(), use a time.NewTicker (e.g., every 5 seconds) to wake up and poll for jobs.
- In the loop, call a processJobs() function.
Create the job processor:
- In worker.go, create processJobs().
- It should query the DB: SELECT id, action_type, payload FROM action_queue WHERE process_at <= NOW() FOR UPDATE SKIP LOCKED.
- It should loop over the returned rows.
- Use a switch item.action_type to handle different jobs. For now, it will be empty.
- After a job is processed successfully, it must DELETE FROM action_queue WHERE id = $1.
Add unit tests:
- Test the processJobs function in isolation.
- Mock the database: Make your SELECT ... FOR UPDATE mock return a list of 2–3 sample jobs (e.g., one "star", one "move").
- Mock the IMAP service.
- Assert that the correct IMAP service methods (e.g., StarThread, MoveThread) are called with the exact payloads from the mock jobs.
- Assert that the DELETE FROM action_queue command is called for each successfully processed job.
Add integration tests (Go + Testcontainers):
- Start a real Postgres DB using testcontainers-go.
- Mock only the IMAP service.
- Manually INSERT 2-3 jobs into the action_queue table.
- Start your ActionWorker.
- Wait ~100ms.
- Assert your mock IMAP service methods were called.
- Query the database: Assert the processed jobs have been deleted from the action_queue table.

3/2. ⭐️ Feature: Star/Unstar

Backend (Star)

Create the "Star" IMAP logic:
- In /backend/internal/imap, create a new actions.go file.
- Create a function: func (s *Service) StarThread(ctx context.Context, userID string, threadStableID string, starStatus bool) error.
- This function needs to:
  1. Get the user's settings (for credentials) from the DB.
  2. Get an IMAP connection from your pool.
  3. Find all messages in our DB for that thread: SELECT imap_uid, imap_folder_name FROM messages WHERE thread_id = (SELECT id FROM threads WHERE stable_thread_id = $1 AND user_id = $2).
  4. Loop through each message, c.Select(folderName, ...)
  5. Build a seqSet for the imap_uid.
  6. Run the IMAP command: c.UidStore(seqSet, flagOp, []interface{}{imap.FlaggedFlag}) (where flagOp is "+FLAGS.SILENT" or "-FLAGS.SILENT").
  7. Update our own DB to match: UPDATE messages SET is_starred = $1 WHERE ....
Create the actions API endpoint:
- In /backend/internal/api/routes.go, add the route: router.Post("/api/v1/actions", app.actionsHandler).
- Create a new /backend/internal/api/actions_handler.go file.
- The actionsHandler should:
  1. Read the JSON body, e.g., {"action": "star_thread", "payload": {"thread_stable_id": "...", "star_status": true}}.
  2. Check the action type.
  3. Call a new function db.QueueAction(...) to save the job to the action_queue table with process_at = NOW().
  4. Return 202 Accepted to the frontend immediately.
Hook up the worker:
- In worker.go, add a case "star_thread": to your switch statement.
- It should parse the JSON payload, then call your new imapService.StarThread(...).

Frontend (Star)

Create the starThread mutation:
- In /frontend/src/api/ (or a new api/email.api.ts), create a useStarThreadMutation hook using TanStack Query's useMutation.
- This mutation will POST to /api/v1/actions with the star_thread payload.
Add optimistic updates:
- This is key for a fast UI. In your useStarThreadMutation, use the onMutate option to optimistically update the query cache before the API call runs.
- You'll use queryClient.setQueryData(...) to find the thread in the 'threads' query cache and manually set its is_starred status.
- Add onError to roll back the change if the mutation fails.
- Add onSettled to queryClient.invalidateQueries(['threads']) to refetch the real data.
Add the UI button:
- In EmailListItem.tsx, add a star icon button.
- onClick should call starThreadMutation.mutate({ thread_stable_id: ..., star_status: !thread.is_starred }).
- The star's appearance (filled vs. outline) should be based on the thread.is_starred prop.

Testing

Backend Unit (Go):
- actions_handler: Test that a POST /api/v1/actions request with a "star_thread" body results in a new row being inserted into action_queue with the correct action_type and payload.
- imapService: Test the StarThread function. Mock the DB and the IMAP client.
  - Assert it correctly fetches messages from the mock DB.
  - Assert the mock IMAP client's UidStore method is called with +FLAGS.SILENT for starStatus: true.
  - Assert the mock IMAP client's UidStore method is called with -FLAGS.SILENT for starStatus: false.
  - Assert it updates the messages table in the mock DB.
Frontend Unit (RTL):
- Test the EmailListItem component.
- Mock the useStarThreadMutation hook.
- Assert the star icon is "outline" when is_starred is false.
- Click the star button.
- Assert the mutate function from the mock hook was called with { ..., star_status: true }.
Frontend Integration (RTL + msw):
- Test Optimistic Update:
  - Mock the POST /api/v1/actions API to have a 1-second delay.
  - Render the EmailListItem (with is_starred: false) inside a QueryClientProvider.
  - Click the star button.
  - Immediately assert that the star icon changes to "filled" (even before the API mock resolves).
  - Wait for the API to resolve.
  - Assert the star icon remains "filled".

3/3. 🗃️ Feature: Archive & Trash

These are identical, just with a different destination. We'll build them as a generic "move" action.

Backend (Move)

Create the "Move" IMAP logic:
- In /backend/internal/imap/actions.go, create: func (s *Service) MoveThread(ctx context.Context, userID string, threadStableID string, destinationFolder string) error.
- Logic:
  1. Get user settings and IMAP connection.
  2. Get all messages for the thread from our DB (SELECT imap_uid, imap_folder_name FROM messages WHERE ...).
  3. Loop through each message, c.Select(folderName, ...).
  4. Run the IMAP command: c.UidMove(seqSet, destinationFolder).
  5. Update our DB: UPDATE messages SET imap_folder_name = $1 WHERE ....
Update the actionsHandler:
- Add logic to handle a new action_type: "move_thread".
- The payload should be {"thread_stable_id": "...", "destination_folder": "Archive"} (or "Trash").
- It should queue this in the action_queue table.
Hook up the worker:
- In worker.go, add case "move_thread": to your switch.
- It should parse the payload and call imapService.MoveThread(...).

Frontend (Move)

Create the moveThread mutation:
- In email.api.ts, create a useMoveThreadMutation hook.
- It POSTs to /api/v1/actions with the move_thread payload.
Add UI buttons:
- In EmailListItem.tsx (e.g., on hover) and ThreadView.tsx, add "Archive" and "Trash" icon buttons.
- onClick on "Archive" calls moveThreadMutation.mutate({ ..., destination_folder: 'Archive' }).
- onClick on "Trash" calls moveThreadMutation.mutate({ ..., destination_folder: 'Trash' }).
Update cache on success:
- This is not an optimistic update.
- In the onSuccess callback of your useMoveThreadMutation, you must invalidate the cache for the list you're looking at.
- queryClient.invalidateQueries({ queryKey: ['threads', currentFolder] }). This will trigger a refetch, and the item will disappear from the list.

Testing

Backend Unit (Go):
- actions_handler: Test that a POST request with a "move_thread" body (payload destination_folder: 'Archive') queues the correct job.
- imapService: Test the MoveThread function. Mock the DB and IMAP client.
  - Assert it correctly fetches messages.
  - Assert the mock IMAP client's UidMove method is called with the correct seqSet and destination folder.
  - Assert it updates the messages table in the mock DB with the new imap_folder_name.
Frontend Integration (RTL + msw):
- Test Cache Invalidation:
  - Mock the GET /api/v1/threads?folder=INBOX to return three items.
  - Render the Inbox.page.tsx. Assert three items are visible.
  - Mock the POST /api/v1/actions API to succeed.
  - Mock queryClient.invalidateQueries to track calls.
  - Click the "Archive" button on the first item.
  - Assert invalidateQueries was called with the ['threads', 'INBOX'] query key.

3/5. 🧪 Test plan: Milestone 3 (end-to-end)

This Playwright plan tests the full "action" loop. It assumes a pre-populated mock IMAP server.

Test 1: Star a Thread (Full Loop)
- Log in and go to the Inbox.
- Find a thread and assert its star is "outline".
- Click the star.
- Assert the star immediately turns "filled" (optimistic UI).
- Wait 6 seconds (for the worker to run).
- Reload the entire page.
- Log in again (if needed).
- Assert the same thread still shows a "filled" star (proves the backend IMAP & DB update worked).
Test 2: Archive a Thread (Full Loop)
- Log in and go to the Inbox.
- Find a thread and note its subject (e.g., "Project Budget").
- Click the "Archive" button.
- Assert the "Project Budget" thread disappears from the Inbox (tests cache invalidation).
- Click the "Archive" folder in the sidebar.
- Assert the "Project Budget" thread appears in the Archive folder list.
Test 3: Search (Full Loop)
- Log in. (Assume a message with the subject "Special Report Q3" exists on the mock IMAP server).
- Type "Special Report" into the search bar and press Enter.
- Assert the URL changes to /search?q=Special%20Report.
- Assert the thread "Special Report Q3" is visible in the search results.

Milestone 4: Composing

Goal: Be able to send email.
Tasks: Build composer UI. Implement SMTP logic on the backend. Implement Reply/Forward. Implement "Undo Send."

This milestone is built around the action_queue you created in M3. The "Send" button doesn't send an email; it queues a "send" job. This is what makes "Undo Send" possible.

Breakdown:

4/1. 🏗️ Backend: "Send" & "Undo" API endpoints

We'll start by creating the two API endpoints the frontend needs: one to queue a send action, and one to cancel it.

Create the "Send" API endpoint:
- Add the route: router.Post("/api/v1/send", app.sendHandler).
- Create a new /backend/internal/api/send_handler.go.
- Handler Logic:
  - Read the JSON body (to, subject, body, etc.).
  - Get the user's undo_send_delay_seconds from their user_settings in the DB.
  - Calculate process_at = NOW() + (delay * time.Second).
  - Create the payload (a JSONB object of the full email).
  - INSERT the job into action_queue (action_type = 'send_email', payload, process_at).
  - Crucially: Use RETURNING id on your INSERT to get the new job's UUID.
  - Return 202 Accepted with a JSON body: {"job_id": "..."}. The frontend needs this ID for the "Undo" button.
Create the "Undo" API endpoint:
- Add the route: router.Delete("/api/v1/send/undo/:jobID", app.undoHandler).
- Create a new /backend/internal/api/undo_handler.go.
- Handler Logic:
  - Get jobID from the URL parameters.
  - Get userID from the auth middleware.
  - Run DELETE FROM action_queue WHERE id = $1 AND user_id = $2. (Checking user_id is a critical security step).
  - Return 200 OK.
Backend Unit Tests:
- send_handler:
  - Test that POST /api/v1/send with a valid body inserts a row into action_queue.
  - Assert the action_type is send_email.
  - Assert the process_at time is correct (e.g., now + 20s).
  - Assert the handler returns a 202 status and the job_id.
- undo_handler:
  - INSERT a test job for user_a.
  - Test that DELETE /api/v1/send/undo/:jobID (as user_a) deletes the job.
  - Test that DELETE /api/v1/send/undo/:jobID (as user_b) does not delete the job (and returns an error or 404).

4/2. ⚙️ Backend: SMTP & "Sent" folder worker

Now, we'll teach our M3 ActionWorker how to process the send_email jobs.

Create SMTP logic:
- Create a new folder /backend/internal/smtp.
- In smtp/client.go, create a SendEmail function: func SendEmail(settings *UserSettings, payload *EmailPayload) error.
- This function should:
  - Use github.com/go-mail/mail to build the email message (set "From", "To", "Subject", "HTMLBody").
  - Use net/smtp to connect: smtp.Dial(settings.smtp_server_hostname + ":587").
  - Authenticate: smtp.PlainAuth(...).
  - Send: mail.Send(client, msg).
Create IMAP "Append" logic:
- In /backend/internal/imap/actions.go, create a new function: func (s *Service) AppendToSent(ctx context.Context, userID string, rawEmailBytes []byte) error.
- This function should:
  - Get user settings (for credentials and sent_folder_name).
  - Get an IMAP connection.
  - c.Append(settings.sent_folder_name, ...) to upload the raw email bytes.
Hook up the worker:
- In /backend/internal/sync/worker.go, add case "send_email": to your switch.
- Logic:
  - Parse the payload.
  - Call smtp.SendEmail(...).
  - If SMTP fails, log the error and don't delete the job (it will retry).
  - If SMTP succeeds, call imap.AppendToSent(...) to save a copy.
  - If both succeed, DELETE the job from action_queue.
Backend Unit Tests:
- smtp.SendEmail: Mock smtp.Dial and smtp.Auth. Test that the function attempts to connect and send.
- imap.AppendToSent: Mock the IMAP client. Test that c.Append is called with the correct sent_folder_name.
- worker: Test the case "send_email" logic.
  - Mock the SMTP and IMAP services.
  - Assert that on SMTP success, both SMTP and IMAP methods are called.
  - Assert that on SMTP failure, the IMAP method is not called.

4/3. 🎨 Frontend: "Compose" UI & "Undo" snackbar

Now, the frontend to create and cancel jobs.

Create composer store:
- In /frontend/src/store, create composer.store.ts (using Zustand).
- It should hold state: isOpen (boolean), to (string[]), subject (string), body (string), inReplyTo (Message, optional).
- Add actions: openCompose(), openReply(msg), openForward(msg), close().
Create Composer component:
- Create /frontend/src/components/composer/Composer.tsx.
- It renders as a modal or pop-up (fixed to the bottom corner) only if composer.store.isOpen is true.
- It should have input fields for "To" and "Subject," and a rich text editor (or textarea) for "Body," all bound to the Zustand store.
Create "Send" mutation:
- In api/email.api.ts, create useSendEmailMutation.
- It POSTs to /api/v1/send with the data from the composer.store.
- onClick for the "Send" button calls sendEmailMutation.mutate().
Create "Undo" snackbar/toast:
- Create a store/ui.store.ts (Zustand) to hold undoJobId (string | null) and showUndo (boolean).
- In the onSuccess callback of useSendEmailMutation:
  1. Call composer.store.close().
  2. Set ui.store.showUndo = true and ui.store.undoJobId = response.job_id.
  3. Start a setTimeout for undo_send_delay_seconds. When it fires, set showUndo = false.
- Create components/UndoSnackbar.tsx that renders if showUndo is true.
- Create useUndoSendMutation that DELETEs /api/v1/send/undo/:jobID.
- The "Undo" button in the snackbar calls this mutation and hides the snackbar.
Frontend Integration Tests (RTL + msw):
- Mock POST /api/v1/send to return {"job_id": "123"}.
- Click the "Compose" button. Assert composer is visible.
- Fill inputs, click "Send". Assert the API was called with the correct data.
- Assert the composer closes.
- Assert the "Undo Snackbar" is now visible.
- Mock DELETE /api/v1/send/undo/123.
- Click the "Undo" button. Assert the DELETE API was called and the snackbar hides.

4/4. ↩️ Frontend: "Reply/Forward" logic

This part just pre-fills the composer you just built.

Add "Reply/Forward" buttons:
- In Message.tsx (inside ThreadView.tsx), add "Reply," "Reply All," and "Forward" buttons.
Implement store actions:
- composer.store.openReply(msg):
  - Sets isOpen = true.
  - Sets to = [msg.from_address].
  - Sets subject = "Re: " + msg.subject.
  - Sets body = "On [date], [sender] wrote:\n\n> [quoted body]".
  - Sets inReplyTo = msg.
- composer.store.openReplyAll(msg):
  - Same, but to = [msg.from_address, ...msg.to_addresses, ...msg.cc_addresses] (filtering out the user's own email).
- composer.store.openForward(msg):
  - Same, but to = [], subject = "Fwd: " + msg.subject, and body = "Forwarded message:\n\n...".
Frontend Integration Tests (RTL):
- Render ThreadView.tsx with a mock message.
- Click "Reply". Assert the Composer component opens.
- Assert the "To" and "Subject" fields are pre-filled correctly with "Re: ...".

4/5. 🧪 Test plan: Milestone 4 (end-to-end)

This Playwright plan tests the full send-and-undo loop.

Test 1: Compose and Send (Full Loop)
- Log in.
- Click "Compose."
- Fill in "To" (with an external email you can check), "Subject" ("E2E Test Send"), and "Body".
- Click "Send".
- Assert the composer closes.
- Assert the "Undo" snackbar appears.
- Do not click Undo. Wait for the undo_send_delay_seconds (default: 20 sec) plus the worker poll time (default: 5 sec).
- Check external email: Assert the email ("E2E Test Send") was received.
- Check V-Mail UI: Click the "Sent" folder. Assert the "E2E Test Send" email now appears in the "Sent" list (proves the IMAP APPEND worked).
Test 2: Compose and Undo (Full Loop)
- Log in.
- Click "Compose."
- Fill in "To" (with an external email), "Subject" ("E2E Test Undo"), and "Body".
- Click "Send".
- Assert the "Undo" snackbar appears.
- Immediately click "Undo".
- Assert the snackbar disappears.
- Wait 30 seconds (longer than your undo delay).
- Check external email: Assert the email ("E2E Test Undo") was NOT received.
- Check V-Mail UI: Click the "Sent" folder. Assert the email does NOT appear in the "Sent" list.

Milestone 5: Quality of life

Goal: Polish the MVP.
Tasks: Auto-save drafts. Add keyboard shortcuts. Add pagination. Add IDLE and WebSocket connection.

Breakdown:

5/1. 💾 Feature: Auto-save drafts

This plan implements a fast, reliable auto-save that saves to your Postgres DB first, and then (as a bonus) syncs to your IMAP server in the background.

Backend

Update drafts table:
- Add a new column: imap_uid BIGINT NOT NULL DEFAULT 0. This will store the UID of the draft on the IMAP server once it's synced.
Create POST /api/v1/drafts endpoint:
- This will be your auto-save endpoint. It needs to be fast.
- Create a SaveDraft(ctx, draft) function in /backend/internal/db/drafts.go.
- This function should use INSERT ... ON CONFLICT (id) DO UPDATE ... to create or update the draft in your Postgres drafts table.
- The actions_handler for this route should:
  1. Read the draft payload (to, subject, body, etc.) from the JSON body.
  2. Call db.SaveDraft(...).
  3. Return the draft.id to the frontend: {"draft_id": "..."}.
(Optional/Bonus) Create background IMAP sync:
- After saving to the DB, have the POST /api/v1/drafts handler queue a new job: db.QueueAction(..., "sync_draft", payload{"draft_id": "..."})
- In your /backend/internal/sync/worker.go, add a case "sync_draft":
- In /backend/internal/imap/actions.go, create func (s *Service) SyncDraft(ctx, draftID string) error.
- This function should:
  1. Fetch the draft from your Postgres drafts table.
  2. Get the imap_uid and drafts_folder_name (from user_settings).
  3. Get an IMAP connection.
  4. If imap_uid == 0: APPEND the draft to the drafts_folder_name and save the new imap_uid (from the APPEND response) back to your Postgres drafts table.
  5. (Harder/v2) If imap_uid > 0: DELETE the old message (STORE +FLAGS.SILENT \Deleted) and APPEND the new one (and update the imap_uid). For now, just APPENDing is fine.
Create GET /api/v1/drafts endpoint:
- This route should query your Postgres drafts table (not IMAP) and return all saved drafts for the user.
Update send_email worker:
- The payload for send_email now needs an optional draft_id_to_delete.
- After the send_email job successfully sends (SMTP) and appends to "Sent" (IMAP), it must also:
  1. DELETE FROM drafts WHERE id = $1 (the Postgres draft).
  2. (If you did the bonus sync) Queue a new job to delete the draft from the IMAP Drafts folder.

Frontend

Create useAutoSave hook:
- Create hooks/useAutoSave.ts.
- This hook takes the current composer state (to, subject, body) as an argument.
- It uses useEffect and setTimeout to create a "debounce" (e.g., trigger 2 seconds after the user stops typing).
- When the timeout fires, it calls saveDraftMutation.mutate(...).
Create useSaveDraftMutation:
- In api/email.api.ts, create this mutation. It POSTs to /api/v1/drafts.
- In its onSuccess callback, it must save the returned draft_id into your composer.store (which needs a new draft_id field).
Integrate with Composer:
- Call useAutoSave(composerState) from your Composer.tsx component.
Update "Send" button:
- When the "Send" button is clicked, the useSendEmailMutation must now include the draft_id from the composer.store in its payload, so the backend knows which draft to delete.
Create "Drafts" page:
- Add a "Drafts" link to your Sidebar.tsx (it should point to /drafts).
- Create pages/Drafts.page.tsx.
- This page uses useQuery to GET /api/v1/drafts.
- It renders a list of drafts (you can reuse EmailListItem.tsx).
- When a draft is clicked, it calls composer.store.openDraft(draftData) (a new action you'll create) to open the composer and pre-fill it.

Testing

Backend Unit: Test the POST /api/v1/drafts handler (asserts DB INSERT/UPDATE and returns an id). Test the GET /api/v1/drafts handler.
Frontend Unit: Test useAutoSave hook. Mock setTimeout and the mutation. Assert mutate is (or is not) called based on typing/pausing.
Frontend Integration: Mock POST /api/v1/drafts. Type in the composer, pause for 3 seconds. Assert the POST API was called.
E2E:
1. Click "Compose," type "test subject."
2. Reload the page.
3. Go to the "Drafts" page. Assert "test subject" is in the list.
4. Click it. Assert the composer opens with "test subject".
5. Add a recipient and click "Send."
6. Wait 30 seconds. Assert the draft is now gone from the "Drafts" page.

5/2. ⌨️ Feature: Add keyboard shortcuts

This is a frontend-only task that expands on the hook you built in M2.

Expand useKeyboardShortcuts.ts:
- Add logic to listen for key presses.
- c: Call composer.store.openCompose().
- /: Find the search bar ref and call ref.current.focus().
- g then i: Maps('/?folder=INBOX') (this needs a simple state machine in the hook).
- g then s: Maps('/?folder=Sent') (or your starred view).
- g then d: Maps('/drafts').
Add shortcuts for selected items:
- This requires a "selected item" state (e.g., ui.store.selectedThreadId). Your j/k keys should update this ID.
- e: If selectedThreadId exists, call moveThreadMutation.mutate({ ..., destination_folder: 'Archive' }).
- s: If selectedThreadId exists, call starThreadMutation.mutate(...).
- # (Shift+3): If selectedThreadId exists, call moveThreadMutation.mutate({ ..., destination_folder: 'Trash' }).
Add shortcuts for the thread view:
- When on a /thread/:threadId page:
- r: Call composer.store.openReply(currentThreadData).
- a: Call composer.store.openReplyAll(currentThreadData).
- f: Call composer.store.openForward(currentThreadData).
Add "Undo" shortcut:
- z: If ui.store.showUndo is true, call undoSendMutation.mutate().

Testing

Frontend Unit:
- Write extensive tests for useKeyboardShortcuts.ts.
- Simulate keydown events (c, /, g+i, e, r, etc.).
- Mock all the store actions and mutations.
- Assert that the correct mock function is called for each key press.

5/4. ⚡ Feature: Add real-time updates

This is the most complex but most rewarding "quality of life" feature.

Backend

Add WebSocket library:
- go get github.com/gorilla/websocket
Create WebSocket Hub:
- Created /backend/internal/websocket/hub.go.
- The Hub struct manages active connections: userID -> set of *websocket.Conn (via a Client wrapper).
- It exposes methods: Register(userID, conn), Unregister(userID, client), and Send(userID, message []byte).
- It supports multiple connections per user with a per-user limit (currently 10).
Create GET /api/v1/ws endpoint:
- Added the route in cmd/server/main.go.
- The handler (WebSocketHandler) upgrades the HTTP connection to a WebSocket.
- It gets the userID from the auth context using GetUserIDFromContext.
- It calls hub.Register(userID, conn) and starts a read loop to detect disconnects.
- On disconnect, it calls hub.Unregister(userID, client) and stops the IMAP IDLE listener when there are no more active connections for that user.
Create IMAP IDLE listener:
- Implemented in /backend/internal/imap/idle.go as func (s *Service) StartIdleListener(ctx context.Context, userID string, hub *websocket.Hub).
- Launch: When a user successfully connects to the WebSocket (hub.Register), WebSocketHandler starts this function in a new goroutine for that userID (if not already running).
- Logic:
  1. Get a dedicated IMAP listener connection from the pool.
  2. Run SELECT INBOX.
  3. Start an IDLE loop (with fallback) using go-imap-idle.
  4. Listen for updates via the client's Updates channel. When an update indicates new messages in INBOX, call SyncThreadsForFolder for INBOX immediately.
  5. After syncing, call hub.Send(userID, []byte('{"type":"new_email","folder":"INBOX"}')).
  6. On errors (e.g., timeout), log, remove the listener connection, and retry after a short sleep.

Frontend

Create useWebSocket hook:
- Created hooks/useWebSocket.ts.
- It is called once from the main Layout.tsx.
- useEffect on mount:
  1. Opens new WebSocket(VITE_WS_URL || '<origin>/api/v1/ws').
  2. Sets status in a connection.store.ts (Zustand) to connecting/connected/disconnected.
  3. Handles onmessage and onclose to update connection state.
- The onmessage handler parses event.data and, when message.type === 'new_email', invalidates queries for that folder.
Invalidate cache on message:
- Inside the onmessage handler:
- Gets the queryClient using useQueryClient().
- Calls queryClient.invalidateQueries({ queryKey: ['threads', message.folder] }).
- This automatically makes TanStack Query refetch the thread list, and the new email appears.
Connection status banner and manual reconnect:
- Added a ConnectionStatusBanner component, shown when the WebSocket status is disconnected.
- The banner displays a Gmail-style "Connection lost. New emails may be delayed." message with a "Try now" link that triggers a reconnect of the WebSocket.

Testing

Frontend Integration (RTL + WebSocket mocking via MSW):
- Uses msw's WebSocket support instead of mock-socket.
- Renders a component that uses useWebSocket under a QueryClientProvider.
- Simulates a message from the mock socket: server.send('{"type": "new_email", "folder": "INBOX"}').
- Asserts that queryClient.invalidateQueries was called with { queryKey: ['threads', 'INBOX'] }.
E2E:
- Adds a new E2E test in e2e/tests/inbox.spec.ts.
- With the Inbox page open, the test calls /test/add-imap-message (a test-only backend endpoint) to append a message to INBOX on the IMAP server.
- Asserts the new email appears in the V-Mail inbox without a page reload.

Milestone 6: Offline

Goal: Basic offline support.
Tasks: Implement IndexedDB caching for recently viewed emails. Build the sync logic.

Breakdown:

6/1. 💾 Frontend: Set up the local database

First, we need a place to store the emails in the browser. We'll use dexie.js, which is a powerful and easy-to-use wrapper for IndexedDB.

**Install Dexie: pnpm install dexie dexie-react-hooks
Define the local DB schema:
- Create a new file: /frontend/src/lib/db.ts.
- In this file, define your Dexie database. The schema should mirror your Postgres tables, as this will make syncing much easier.

import Dexie, { Table } from 'dexie'

// Define interfaces for your tables
// (You can move these to a /types file)
export interface IThread {
  id: string // This is your stable_thread_id
  subject?: string
  // ... other thread properties
}

export interface IMessage {
  id: string // This is your message_id_header
  thread_id: string
  imap_folder_name: string
  from_address?: string
  subject?: string
  unsafe_body_html?: string
  body_text?: string
  is_read: boolean
  is_starred: boolean
  // ... other message properties
}

export class VMailDB extends Dexie {
  threads!: Table&lt;IThread>
  messages!: Table&lt;IMessage>

  constructor() {
    super('vmailDB')
    this.version(1).stores({
      // 'id' is the primary key
      // 'thread_id' and 'imap_folder_name' are indexes
      threads: 'id',
      messages: 'id, thread_id, imap_folder_name',
    })
  }
}

export const db = new VMailDB()

6/2. 📥 Frontend: Cache viewed emails

This part implements "caching for recently viewed emails." The logic is simple: anything you successfully fetch from the API, you save a copy of in IndexedDB.

Cache thread list:
- In your Inbox.page.tsx (or wherever you fetch threads), find your useQuery for GET /api/v1/threads.
- Add an onSuccess callback to the useQuery options.
- In onSuccess(data), call db.threads.bulkPut(data.threads). This will "upsert" (insert or update) all the threads you just fetched.
Cache full thread:
- In your Thread.page.tsx, find your useQuery for GET /api/v1/thread/:threadId.
- Add an onSuccess callback.
- In onSuccess(data), save both the thread and its messages:
  TypeScript
  db.threads.put(data.thread)
- db.messages.bulkPut(data.thread.messages)

6/3. 🔌 Frontend: Read from cache when offline

Now, we'll change your queries to always read from the local cache first. This gives an "offline-first" feel.

Modify GET /api/v1/threads query:
- In your useQuery for GET /api/v1/threads, change the queryFn.
- The queryFn should first try to get data from Dexie:

queryFn: async () => {
  // 1. Try to get data from the local cache
  const cachedThreads = await db.threads
    .where('imap_folder_name') // Assuming you add this to the threads table
    .equals(folder)
    .toArray()

  // 2. If online, fetch from API in the background
  if (navigator.onLine) {
    try {
      const freshData = await api.getThreads(folder) 
      // The onSuccess (from 6/2) will auto-cache this
      return freshData.threads
    } catch (error) {
      // If API fails, return cached data so app still works
      return cachedThreads
    }
  }

  // 3. If offline, just return cached data
  return cachedThreads
}

Modify GET /api/v1/thread/:threadId query:
- Apply the same logic. The queryFn should first await db.messages.where('thread_id').equals(threadId).toArray() and return that if the user is offline.

6/4. 🔄 Backend: Create "delta" sync endpoints

Your action_queue already handles syncing actions (writes). This is for syncing reads (changes from other clients, like your phone).

To do this efficiently, we need a "give me what's changed" endpoint.

Create a new migration:
- migrate create -ext sql -dir backend/migrations -seq add_timestamps
Modify schema (.up.sql):
- We need updated_at on our main tables.

-- Create a trigger function to auto-update timestamps
CREATE OR REPLACE FUNCTION trigger_set_timestamp()
RETURNS TRIGGER AS $$
BEGIN
  NEW.updated_at = NOW();
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

-- Add the column and trigger to 'threads'
ALTER TABLE "threads" ADD COLUMN "updated_at" TIMESTAMPTZ NOT NULL DEFAULT now();
CREATE TRIGGER set_timestamp
BEFORE UPDATE ON "threads"
FOR EACH ROW
EXECUTE PROCEDURE trigger_set_timestamp();

-- Add the column and trigger to 'messages'
ALTER TABLE "messages" ADD COLUMN "updated_at" TIMESTAMPTZ NOT NULL DEFAULT now();
CREATE TRIGGER set_timestamp
BEFORE UPDATE ON "messages"
FOR EACH ROW
EXECUTE PROCEDURE trigger_set_timestamp();

Create new "sync" API endpoints:
- GET /api/v1/sync/threads?since=<timestamp>
- GET /api/v1/sync/messages?since=<timestamp>
- These handlers read the since query param.
- They query Postgres: SELECT * FROM threads WHERE user_id = $1 AND updated_at > $2.
- They return a list of updated/new threads and messages.

6/5. ⚡ Frontend: Build the sync logic

This logic will run in the background to keep the local database fresh.

Create useSync hook:
- Create hooks/useSync.ts.
- This hook will be called once from your main Layout.tsx.
Implement the sync function:
- Inside the hook, create a sync() function.
- It should:
  1. Get lastSyncTimestamp from localStorage.
  2. fetch('/api/v1/sync/threads?since=' + lastSyncTimestamp).
  3. fetch('/api/v1/sync/messages?since=' + lastSyncTimestamp).
  4. Take the results and bulkPut them into your Dexie db.threads and db.messages tables.
  5. Save new Date().toISOString() into localStorage as the new lastSyncTimestamp.
Trigger the sync:
- Use useEffect to call sync() once on app load.
- Use setInterval to call sync() every 5 minutes.
- Add a window.addEventListener('online', sync) to trigger an immediate sync when the user's connection returns.

6/6. 🧪 Test plan: Milestone 6 (end-to-end)

Offline mode is notoriously hard to test. Use Playwright for this.

Test 1: Offline read (cache population)
- Log in while online.
- Open the "Inbox".
- Click the first thread (Subject: "Meeting Notes").
- Click back to the "Inbox".
- Turn network offline using Playwright's context.setOffline(true).
- Reload the page.
- Assert the "Inbox" list still loads (from Dexie).
- Assert you can click "Meeting Notes" and read the full email (from Dexie).
- Assert that clicking the second thread (which you never opened) shows a loading spinner or an "Offline" message (because its body isn't cached).
Test 2: Offline action (action_queue)
- Log in while online.
- Open the "Inbox".
- Turn network offline.
- Find a thread ("Meeting Notes") and click the "Star" button.
- Assert the star optimistically turns "filled".
- Reload the page (still offline).
- Assert the star is still "filled" (this tests that your optimistic UI state is also saved, or that you're reading the action_queue).
- Turn network online (context.setOffline(false)).
- Wait 10–15 seconds (for the worker and sync logic to run).
- Reload the page (now online).
- Assert the "Meeting Notes" email is still starred (proving the action was synced to the server).
Test 3: Background sync (delta sync)
- Log in to V-Mail in Playwright. Have the "Inbox" open.
- Use your spike script (or another email client) to send a new email to your account.
- Do not reload the Playwright browser.
- Wait for your sync interval (or manually trigger sync via a debug button).
- Assert the new email appears at the top of the list in V-Mail without a page reload.

Later

Write a doc for how to create a daily DB backup, e.g., via a pg_dump cron job.

FilesExpand file tree

ROADMAP.md

Latest commit

History

ROADMAP.md

File metadata and controls

Roadmap overview

Milestone 2: Missing things

2/7. 🔐 Authentication

2/8. 🔐 Proper basic functionality

Milestone 3: Actions

3/1. 🏗️ Backend: Action worker

3/2. ⭐️ Feature: Star/Unstar

Backend (Star)

Frontend (Star)

Testing

3/3. 🗃️ Feature: Archive & Trash

Backend (Move)

Frontend (Move)

Testing

3/5. 🧪 Test plan: Milestone 3 (end-to-end)

Milestone 4: Composing

4/1. 🏗️ Backend: "Send" & "Undo" API endpoints

4/2. ⚙️ Backend: SMTP & "Sent" folder worker

4/3. 🎨 Frontend: "Compose" UI & "Undo" snackbar

4/4. ↩️ Frontend: "Reply/Forward" logic

4/5. 🧪 Test plan: Milestone 4 (end-to-end)

Milestone 5: Quality of life

5/1. 💾 Feature: Auto-save drafts

Backend

Frontend

Testing

5/2. ⌨️ Feature: Add keyboard shortcuts

Testing

5/4. ⚡ Feature: Add real-time updates

Backend

Frontend

Testing

Milestone 6: Offline

6/1. 💾 Frontend: Set up the local database

6/2. 📥 Frontend: Cache viewed emails

6/3. 🔌 Frontend: Read from cache when offline

6/4. 🔄 Backend: Create "delta" sync endpoints

6/5. ⚡ Frontend: Build the sync logic

6/6. 🧪 Test plan: Milestone 6 (end-to-end)

Later

Archive

Milestone 1

Milestone 2

2/1. 🏗️ Backend: Server foundation

2/2. 🛡️ Backend: Auth and onboarding

2/3. 🎨 Frontend: Skeleton and settings page

2/4. 📨 Backend: Read-only email API

2/5. 🖥️ Frontend: Read-only email UI

Testing: Unit Tests (Jest + React Testing Library)

Testing: Integration Tests (React Testing Library + msw)

2/6. 🧪 Test Plan: Milestone 2 (End-to-End)

Front end cleanup

Milestone 3

3/4. 🔎 Feature: Search

Phase 1: Basic Text Search

Backend (Basic Search)

Frontend (Basic Search)

Testing (Phase 1)

Phase 2: Gmail-like Syntax Parsing

Backend (Query Parser)

Frontend (Query Validation)

Testing (Phase 2)

Implementation Notes

Future Enhancements (Not in this milestone)

Milestone 5

5/3. 📄 Feature: Add pagination

Backend

Frontend

Testing

Testing: Integration Tests (React Testing Library + `msw`)