Skip to content

GH-46: faster syncdb transfers, new flags, safer db-post-import hook#47

Open
tormi wants to merge 26 commits into
mainfrom
GH-46
Open

GH-46: faster syncdb transfers, new flags, safer db-post-import hook#47
tormi wants to merge 26 commits into
mainfrom
GH-46

Conversation

@tormi
Copy link
Copy Markdown
Member

@tormi tormi commented Mar 5, 2026

Ticket GH-46

Changes

syncdb (faster + new features)

  • Gzip compressed transfers--gzip on drush sql-dump reduces transfer from ~18 min to ~2 min for a 629MB database
  • SSH -C compression as additional fallback
  • Single ddev yq call instead of three (~4.8s → ~1.6s)
  • Upfront validation of drush/sites/self.site.yml and alias — instant error with available aliases listed
  • --backup flag — creates local DB backup before overwriting
  • Warning about local db overwrite
  • Renamed prod_ssh_*remote_ssh_* (env-agnostic)
  • Post-sync hints for ddev drush deploy and ddev drush uli

db-post-import hook

  • Replaced separate db commands with drush deploy

Developers now run ddev syncdb prod --no-deploy if they want fast import and skip drush deploy.

Docs

  • Updated syncdb command docs with new flags and examples
  • Fixed database management section to reflect sanitize-only hook
  • Added copyable markdown snippet for project READMEs
  • Condensed Performance Optimization section
  • Fixed period consistency, casing, and typos throughout

Closes #46

zHelmet and others added 5 commits February 25, 2026 09:08
Remove dangerous cache:rebuild before config:import that causes fatal
errors on breaking schema changes (see wunderio/charts#514).
Remove redundant double cache:rebuild (~15-20s saved).
Remove full deploy pipeline from hook — developers run ddev drush deploy
deliberately after import.
Keep only drush sqlsan for all ddev import-db use cases.
Add --gzip to drush sql-dump for ~5.5x faster transfers.
Add SSH -C compression as fallback.
Consolidate three ddev yq calls into one (~4.8s to ~1.6s).
Validate drush/sites/self.site.yml and alias upfront.
Add --backup flag to create local DB backup before overwriting.
Add --force flag to skip confirmation prompt.
Add confirmation prompt before overwriting local database.
Rename prod_ssh_* variables to remote_ssh_* (env-agnostic).
Show ddev drush deploy and ddev drush uli hints after sync.
Update description to reflect any remote environment, not just production.
Update examples to show new flags.
Document new syncdb flags (--backup, --force, --keep-dump).
Update database management section to reflect sanitize-only hook.
Add copyable markdown snippet for project READMEs.
Use version-agnostic drupal project type in setup instructions.
Condense Performance Optimization section (DRY/KISS).
Fix period consistency, casing (PHPCS, PHPCBF), and typos.
zHelmet added 2 commits March 9, 2026 08:09
Remove unnecessary confirmation prompts since the command is deliberately invoked by the user. Build the import-db command as an array instead of using eval, and pass --skip-hooks flag through to ddev import-db when requested. Remove post-run hints that are redundant when hooks handle deployment steps.
@tormi tormi requested a review from zHelmet March 10, 2026 07:16
@tormi tormi mentioned this pull request Mar 10, 2026
Comment thread wunderio/core/tooling/syncdb.sh Outdated
@ragnarkurmwunder
Copy link
Copy Markdown

Note: I did a visual review only.

@hkirsman
Copy link
Copy Markdown
Collaborator

Gzip compressed transfers — --gzip on drush sql-dump reduces transfer from ~18 min to ~2 min for a 629MB database

Note that this is case for very slow internet eg in Alicante. But compression is of course must have and makes thing still better!

Comment thread README.md Outdated
- update setup command to use --project-type=drupalN
- add note that N is the Drupal version number

Refs: README.md
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR targets faster and safer database synchronization/import workflows for the DDEV Drupal add-on, primarily by compressing sync transfers, adding new sync flags, and revising post-import behavior/documentation.

Changes:

  • Updated syncdb to use gzip-compressed drush sql-dump, add --backup / --keep-dump / --no-deploy, and do earlier alias validation.
  • Adjusted db-post-import behavior to sanitize and (currently) run drush deploy unless a marker file is present.
  • Refreshed README documentation and command descriptions to match the new syncdb usage.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
wunderio/core/wdr-core.sh Attempts to ensure DDEV_APPROOT is set when missing (host-side execution context).
wunderio/core/tooling/syncdb.sh Adds gzip dump transfer, new flags, upfront alias checks, backup option, and marker-file-based deploy skipping.
wunderio/core/hooks/db-post-import.sh Changes post-import hook to sanitize and conditionally run drush deploy.
README.md Updates installation steps, syncdb docs/examples, and performance notes.
commands/host/wunderio-core-syncdb.sh Updates command description and PATH initialization for host-side syncdb.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread wunderio/core/wdr-core.sh Outdated
Comment thread wunderio/core/tooling/syncdb.sh Outdated
Comment thread wunderio/core/tooling/syncdb.sh
Comment thread wunderio/core/hooks/db-post-import.sh
Comment thread README.md
Comment on lines +21 to +24
ddev config --project-type=drupalN --docroot=web --project-name=example.com
```

where `N` is the major version of Drupal (e.g., `10` or `11`).
hkirsman added 2 commits May 21, 2026 11:29
When DDEV_APPROOT is derived from git or PWD, export it so child scripts
inherit the variable outside the DDEV container.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

syncdb: uncompressed transfer ~5.5x slower than necessary + dangerous cache:rebuild in db-post-import hook

5 participants