fix(store): tolerate duplicate conversation parts#5
Merged
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR updates the store import/sync path to tolerate duplicate provider conversation-part IDs (seen in Intercom archive imports) by switching from “fail on duplicate” behavior to “upsert + last occurrence wins”, and ensures search/FTS stays consistent when parts move between conversations.
Changes:
- Upsert
conversation_partson(provider, provider_id)conflicts and dedupe duplicate parts within a conversation before writing. - Rebuild
conversation_ftsfor the current conversation and any conversation that lost a moved duplicate part. - Normalize blank part
ProviderIDvalues (and synthesize stable IDs when needed), plus add regression tests covering duplicates and blank IDs.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
internal/store/fixture.go |
Implements part dedupe + upsert-by-provider-ID and adds targeted FTS rebuild logic. |
internal/store/fixture_test.go |
Adds synthetic regression tests for duplicate provider part IDs and blank provider IDs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+424
to
+430
| movedPartConversationIDs := map[string]struct{}{} | ||
| for _, part := range dedupeParts(conversation.ID, conversation.Parts) { | ||
| previousConversationID, err := existingPartConversationID(ctx, tx, conversation.Provider, part.ProviderID, conversation.ID) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| if previousConversationID != "" { |
Comment on lines
+478
to
+486
| func rebuildConversationFTS(ctx context.Context, tx *sql.Tx, conversationID string) error { | ||
| if _, err := tx.ExecContext(ctx, `delete from conversation_fts where conversation_id = ?`, conversationID); err != nil { | ||
| return fmt.Errorf("clear fts: %w", err) | ||
| } | ||
| if _, err := tx.ExecContext(ctx, `insert into conversation_fts(conversation_id, subject, body, tags, participants, assignee, state, rating, fin_status) values(?, ?, ?, ?, ?, ?, ?, ?, ?)`, | ||
| conversation.ID, conversation.Subject, strings.Join(bodies, "\n"), strings.Join(conversation.Tags, " "), strings.Join(conversation.Participants, " "), | ||
| conversation.Assignee, conversation.State, conversation.Rating, conversation.FinStatus); err != nil { | ||
| if _, err := tx.ExecContext(ctx, `insert into conversation_fts(conversation_id, subject, body, tags, participants, assignee, state, rating, fin_status) | ||
| select c.id, | ||
| c.subject, | ||
| coalesce((select group_concat(body, char(10)) from (select p.body as body from conversation_parts p where p.conversation_id = c.id order by p.updated_at, p.provider_id)), ''), | ||
| coalesce((select group_concat(name, ' ') from (select t.name as name from conversation_tags ct join tags t on t.id = ct.tag_id where ct.conversation_id = c.id order by t.name)), ''), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix Intercom archive imports when a provider conversation part ID appears more than once in a sync window.
Changed
Risks
Verification
Summary by cubic
Fixes Intercom archive imports that failed when a conversation part provider ID appears more than once. Parts are now deduped and upserted by provider ID, and FTS is rebuilt for any conversation that loses a moved duplicate.
conversation_partson(provider, provider_id); last occurrence wins.conversation_ftsfor the current conversation and any previous owner of a moved part.Written for commit 50a48b1. Summary will update on new commits. Review in cubic