FIX migration task chunking by sorting links by id.#370
Merged
GuySartorelli merged 1 commit intosilverstripe:4.1from Apr 8, 2025
Merged
FIX migration task chunking by sorting links by id.#370GuySartorelli merged 1 commit intosilverstripe:4.1from
GuySartorelli merged 1 commit intosilverstripe:4.1from
Conversation
Previously links were sorted by the default “sort”. When the number of links exceeded the default chunk limit this resulted in some links not being published by the task due to result set order change between chunks.
Closed
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
After running the linkable migration task, not all links were published. This occurs if the number of links exceeds the default chunk limit of 1000 records.
Looking in the database at the LinkField_Link table I could see that some links were published multiple times (2,3) and some links were not published at all, no record in the LinkField_Link_Live table.
The default sort order for links is the "Sort" column which is not unique. I think that when the database was queried for the second chunk the order of results was different. MySQL doesn't guarantee the same order if the sort column isn't unique.
To fix the issue I sorted the results by ID rather than the default sort column. This fixed the issue and all links were published once.
Manual testing steps
Run the link migration task against a database with over 1000 linkable links that need migrating. Afterwards check the LinkField_Link table and the LinkField_Link_Live table to make sure that all links were published once.
Issues
Pull request checklist