Skip to content

FIX migration task chunking by sorting links by id.#370

Merged
GuySartorelli merged 1 commit intosilverstripe:4.1from
timezoneone:4.1
Apr 8, 2025
Merged

FIX migration task chunking by sorting links by id.#370
GuySartorelli merged 1 commit intosilverstripe:4.1from
timezoneone:4.1

Conversation

@jcop007
Copy link
Copy Markdown

@jcop007 jcop007 commented Mar 7, 2025

Description

After running the linkable migration task, not all links were published. This occurs if the number of links exceeds the default chunk limit of 1000 records.

Looking in the database at the LinkField_Link table I could see that some links were published multiple times (2,3) and some links were not published at all, no record in the LinkField_Link_Live table.

The default sort order for links is the "Sort" column which is not unique. I think that when the database was queried for the second chunk the order of results was different. MySQL doesn't guarantee the same order if the sort column isn't unique.

To fix the issue I sorted the results by ID rather than the default sort column. This fixed the issue and all links were published once.

Manual testing steps

Run the link migration task against a database with over 1000 linkable links that need migrating. Afterwards check the LinkField_Link table and the LinkField_Link_Live table to make sure that all links were published once.

Issues

Pull request checklist

  • The target branch is correct
  • All commits are relevant to the purpose of the PR (e.g. no debug statements, unrelated refactoring, or arbitrary linting)
    • Small amounts of additional linting are usually okay, but if it makes it hard to concentrate on the relevant changes, ask for the unrelated changes to be reverted, and submitted as a separate PR.
  • The commit messages follow our commit message guidelines
  • The PR follows our contribution guidelines
  • Code changes follow our coding conventions
  • This change is covered with tests (or tests aren't necessary for this change)
  • Any relevant User Help/Developer documentation is updated; for impactful changes, information is added to the changelog for the intended release
  • CI is green

Previously links were sorted by the default “sort”.  When the number of links exceeded the default chunk limit this resulted in some links not being published by the task due to result set order change between chunks.
Copy link
Copy Markdown
Member

@GuySartorelli GuySartorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@GuySartorelli GuySartorelli merged commit 45f12df into silverstripe:4.1 Apr 8, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants