Skip to content

Conversation

@roni-estein
Copy link

Ok, So this IS NOT a PR to merge but more of a sample that you can pick apart and use whatever you like or toss completely.

To test it out, make a database with some data and populate it with 1-100 million rows of random data and relationships. Anything that will strain the computer that you are on so queries actually take a few seconds. 2-5 seconds is a good benchmark for unoptimized versions. Try and transform with chunk and normally it will take long, really long like possibly minutes or hours before it either comes through or fails based on your memory cpu limits.

Transforms using this strategy will still take a bit of time but I cut this migration down by 80%, enough to keep it complex but basically easy enough that most Laravel beginners could follow it without any real questions. This should complete on 1 million users in about 30 - 60 seconds. You could probably speed it up or slow it down by changing saveEvery to 100 or 10 or even 1 and playing with RAM etc...

Also in this example I didn't use the screen annotation function but I normally using it when purging key mismatches or looking for dead data in a function

$this->updateStepWithMessage("Updating all the matched {$foreignKey}s in {$inTable}");

will read something like

  1. Updating all the matched user_ids in posts

Hope you find this useful. If you find better faster ways to do this please let me know :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant