Fix: Resolve chat export truncation (Issue #1299)#1314
Open
SyncWithRaj wants to merge 1 commit into
Open
Conversation
Author
|
Hi! This is my first PR here. I fixed the pagination issue for the chat export. Let me know if you need me to make any changes! |
Member
|
Hi @SyncWithRaj, thanks for the PR. Can you please clean the changes to just include the fix without the formating changes (single to double quote, whitespacing etc.)? It'll make it easier to review the PR and keep it on point |
- Changed ConversationAdapters.get_all_conversations_for_export to use offset/limit pagination and added ordering by updated_at. - Updated /api/chat/export route to accept explicit offset and limit parameters instead of page index. - Updated frontend exportChats function to pass offset and limit to the backend.
5b55760 to
06a6a92
Compare
Author
|
Hi @debanjum Cleaned up the formatting-only changes (quote style, indentation, line wrapping, etc.) from the frontend file and squashed into a single commit. The diff now only contains the functional changes related to offset/limit pagination. Thanks for the feedback! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes the data truncation issue reported in #1299, where the "Export Chats" feature would download an incomplete
conversations.jsonfile, missing recent conversations and getting stuck in a loop of duplicate records.Root Cause
get_all_conversations_for_exportused thepageargument directly in the Django queryset slice ([page : page + 10]). This meantpage=0fetched[0:10],page=1fetched[1:11], causing massive overlaps and preventing it from ever reaching all records..order_by(), which could lead to inconsistent pagination results across different databases.pageindex, while the backend expected an explicit offset value.Changes Made
adapters/__init__.py): Updatedget_all_conversations_for_exportto acceptoffsetandlimitinstead ofpage. Added.order_by("-updated_at")to ensure a consistent, predictable order. Fixed the slice to[offset : offset + limit].api_chat.py): Updated the/api/chat/exportroute signature to acceptoffsetandlimitquery parameters.settings/page.tsx): Updated theexportChatsloop to explicitly pass?offset=${page * 10}&limit=10to the backend.How to Test
conversations.jsonfile to verify that the total count of unique conversations matches the database count exactly, with zero duplicates.Before:
After:
Fixes #1299