diff --git a/README.md b/README.md index d3dc8fd..f778b24 100644 --- a/README.md +++ b/README.md @@ -12,15 +12,17 @@ scripts to enhance your Drupal development workflow. ### Steps -1. Initialize your Drupal 10 project. Project name parameter is optional, but -it's advisable to use domain name as your project name as that's used for -the subdomain of ddev.site eg if project name is example.com, then localhost -URL will become example.com.ddev.site. +1. Initialize your Drupal project. Project name parameter is optional, but + it's advisable to use domain name as your project name as that's used for + the subdomain of ddev.site eg if project name is example.com, then localhost + URL will become example.com.ddev.site. ```bash - ddev config --project-type=drupal10 --docroot=web --project-name=example.com + ddev config --project-type=drupalN --docroot=web --project-name=example.com ``` + where `N` is the major version of Drupal (e.g., `10` or `11`). + 2. Install Wunderio DDEV Drupal as a DDEV add-on and restart DDEV: ```bash @@ -59,66 +61,100 @@ URL will become example.com.ddev.site. ## Features +For a quick reference in your project's README, add: + +```markdown +This project uses [ddev-wunderio-drupal](https://github.com/wunderio/ddev-wunderio-drupal) DDEV add-on. Run `ddev -h` to see all available commands. +``` + ### Custom DDEV Commands - `pmu`: Runs drush pmu commands and creates dummy module folders if they don't exist. This helps to uninstall module that has gone missing for example during branch switching. + ```bash ddev pmu module1 module2 ... ``` + - `twig-debug`: Toggles Drupal Twig debugging on/off. Useful for template development. + ```bash ddev twig-debug # Enable Twig debugging ddev twig-debug off # Disable Twig debugging ``` -- `grumphp`: Runs GrumPHP commands + +- `grumphp`: Runs GrumPHP commands. + ```bash ddev grumphp ``` -- `phpunit`: Runs PHPUnit commands + +- `phpunit`: Runs PHPUnit commands. + ```bash ddev phpunit ``` + - `regenerate-phpunit-config`: Regenerates fresh PHPUnit configuration. Run this if you don't have phpunit configured in your project. + ```bash ddev regenerate-phpunit-config ``` + - `codecept`: Runs codeception commands. + ```bash ddev codecept ``` -- `phpcbf`: Runs phpcbf commands + +- `phpcbf`: Runs PHPCBF commands. + ```bash ddev phpcbf ``` -- `phpcs`: Runs PHPcs commands + +- `phpcs`: Runs PHPCS commands. + ```bash ddev phpcs ``` -- `phpstan`: Runs PHPStan commands. Usually, the directory to be scanned is web/modules/custom or a module in the said directory. + +- `phpstan`: Runs PHPStan commands. Usually, the directory to be scanned is `web/modules/custom` or a specific module within that directory. + ```bash ddev phpstan analyze ``` -- `syncdb`: Synchronizes local database from desired environment. - You should have aliases set in drush/sites/self.site.yml + +- `syncdb`: Synchronizes local database from a remote environment. + Requires aliases in `drush/sites/self.site.yml`. + ```bash - ddev syncdb # Will give error as environment is not set - ddev syncdb prod # Will fetch database from production. + ddev syncdb # e.g. ddev syncdb prod + ddev syncdb prod --backup # Back up local DB before overwriting + ddev syncdb prod --keep-dump # Keep the downloaded dump file + ddev syncdb prod --no-deploy # Skip running drush deploy after import + ddev syncdb prod --backup --keep-dump # Combine flags ``` -- `yq`: Runs [yq](https://mikefarah.gitbook.io/yq) commands (YAML processor) + +- `yq`: Runs [yq](https://mikefarah.gitbook.io/yq) commands (YAML processor). It's available inside DDEV, but we expose it to host because why not :). It's required in syncdb script, but it could prove useful in day to day work. + ```bash ddev yq ``` + - `commit`: Generates AI-powered commit messages from staged changes using the configured API. The command analyzes your staged changes and branch name to generate commit messages following the project's commit message format (ticket/issue ID prefix, present tense, imperative mood). Requires `OPENAI_API_URL` and `OPENAI_API_KEY` environment variables to be configured via DDEV global config. + ```bash ddev commit ``` + **Setup:** Configure the API credentials in DDEV global config (applies to all projects) and restart your DDEV project: + ```bash ddev config global --web-environment-add="OPENAI_API_URL=https://your-api-url" ddev config global --web-environment-add="OPENAI_API_KEY=your-api-key" @@ -130,25 +166,25 @@ URL will become example.com.ddev.site. ### Enhanced Configuration 1. **Custom DDEV Configuration** - - Post-start scripts for both host and web containers - by default it gives you uli link. - - Automatic update checks for this package + - Post-start scripts for both host and web containers — by default it gives you a `uli` link. + - Automatic update checks for this package. 2. **Performance Optimizations** - - Special `database_dumps/` directory for Mac users not to mount db dumps + - Special `database_dumps/` directory for Mac users not to mount db dumps. ### Automated Workflows The project includes several automated workflows: 1. **Database Management** - - Post-import database hooks (clears cache, sanitizes database) - - Post-restore snapshot hooks (clears cache, sanitizes database) - - Database synchronization from production + - Post-import and post-restore-snapshot hooks (sanitize database). + - Database synchronization from remote environments via `ddev syncdb`. + - Run `ddev drush deploy` after import to apply updates, import config, and rebuild caches. 2. **Development Environment Setup** - - Automatic composer installation on first start - - Post-start hook that run drush uli - - Integration with Wunderio's development tools eg grumphp, phpunit + - Automatic Composer installation on first start. + - Post-start hook that runs `drush uli`. + - Integration with Wunderio's development tools, e.g. GrumPHP, PHPUnit. Both custom commands and hooks are scripts under `~/.ddev/wunderio/core/` folder (note it's your host home folder) and you can extend them if you copy particular @@ -184,48 +220,21 @@ Previously, this package was installed as a Composer plugin and deployed files i ## Performance Optimization -### Database Operations for Mac Users +### Database Dumps Directory (macOS) -**Important for Mac users:** When working with database imports and exports on macOS, you should store your database -dump files in the `database_dumps` directory at the project root. This directory is specially configured in this -template to provide specific performance benefits. +Store database dump files in the `database_dumps/` directory at the project root. On macOS this directory is excluded from Mutagen sync, avoiding slow DDEV startups and disk bloat. -``` -project-root/ -├── database_dumps/ <- Place your .sql or .sql.gz files here -├── web/ -├── .ddev/ -└── ... -``` - -**Key benefits:** - -1. **Faster DDEV startup times:** When large database files are stored in the standard project directories, -they can significantly slow down DDEV startup as Mutagen indexes and syncs these files. Using the `database_dumps` -directory avoids this overhead. - -2. **Reduced virtual disk usage:** By excluding database dumps from Mutagen synchronization, your virtual disk -requires less space, preventing potential disk space issues. - -This optimization is configured via `upload_dirs` in `.ddev/config.wunderio.yaml`: +Configured via `upload_dirs` in `.ddev/config.wunderio.yaml`: ```yaml upload_dirs: - ../database_dumps ``` -**Example usage:** -```bash -# Save your database dumps to the database_dumps directory -cp ~/Downloads/my-database-backup.sql.gz ./database_dumps/ +`ddev syncdb` uses this directory automatically. For manual imports: -# Then import using the path relative to your project +```bash ddev import-db --file=database_dumps/my-database-backup.sql.gz ``` -This improvement is particularly noticeable in projects with multiple or large database dumps, where -startup times can be reduced from minutes to seconds. - -**Note for Linux users:** While this configuration doesn't provide performance improvements on Linux -systems (which don't use Mutagen), it's still good practice to store database dumps in the -dedicated `database_dumps` folder for consistent organization across team environments. +**Note:** On Linux (no Mutagen) this has no performance effect, but keeps dumps organized consistently across teams. diff --git a/commands/host/wunderio-core-syncdb.sh b/commands/host/wunderio-core-syncdb.sh index fd596a2..d652652 100755 --- a/commands/host/wunderio-core-syncdb.sh +++ b/commands/host/wunderio-core-syncdb.sh @@ -2,12 +2,12 @@ #ddev-generated -## Description: Synchronise local database with production. +## Description: Synchronise local database with a remote environment. ## Usage: syncdb -## Example: "ddev syncdb" +## Example: "ddev syncdb prod" "ddev syncdb prod --backup --no-deploy" ## ExecRaw: true ## ProjectTypes: drupal9,drupal10,drupal11 -export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:$HOME/.ddev/wunderio/core/ +export PATH="$HOME/.ddev/wunderio/core/${PATH:+:$PATH}" wdr-core.sh tooling syncdb.sh "$@" diff --git a/wunderio/core/hooks/db-post-import.sh b/wunderio/core/hooks/db-post-import.sh index 0b33cdd..27f1427 100755 --- a/wunderio/core/hooks/db-post-import.sh +++ b/wunderio/core/hooks/db-post-import.sh @@ -2,8 +2,11 @@ #ddev-generated # -# Helper script to run post-import db hook. +# Post-import database hook. # +# Sanitizes the database after import and executes full deployment. +# Deployment can be skipped by placing a marker file at /mnt/wdr-hooks/.no-deploy +# (e.g. via `ddev syncdb --no-deploy`). set -eu if [[ -n "${WUNDERIO_DEBUG:-}" ]]; then @@ -19,14 +22,16 @@ if [[ -n "${WUNDERIO_DEBUG:-}" ]]; then exit 0 fi -# Every import is treated as deployment -# Unified based on https://www.drush.org/12.x/deploycommand/. -drush updatedb --no-cache-clear -y || { display_error_message "Database update failed"; exit 1; } +# Sanitize imported database (remove sensitive data). drush sqlsan -y || { display_error_message "Database sanitization failed"; exit 1; } -drush cache:rebuild || { display_error_message "Cache rebuild failed"; exit 1; } -drush config:import -y || { display_error_message "Config import failed"; exit 1; } -drush cache:rebuild || { display_error_message "Final cache rebuild failed"; exit 1; } -drush deploy:hook || { display_error_message "Deploy hook failed"; exit 1; } + +if [[ -f /mnt/wdr-hooks/.no-deploy ]]; then + sudo rm -f /mnt/wdr-hooks/.no-deploy + display_status_message "Skipping drush deploy (--no-deploy)." +else + drush deploy -y || { display_error_message "Drupal deploy failed"; exit 1; } +fi uli_link=$(drush uli) -display_status_message "Drupal is working, running drush uli: $uli_link" +display_status_message "Database imported and sanitized." +display_status_message "One-time login link: $uli_link" diff --git a/wunderio/core/tooling/syncdb.sh b/wunderio/core/tooling/syncdb.sh index 981b0b4..5b66f65 100755 --- a/wunderio/core/tooling/syncdb.sh +++ b/wunderio/core/tooling/syncdb.sh @@ -4,8 +4,6 @@ # # Synchronise local database with a remote environment. # -# Based on https://github.com/wunderio/unisport/blob/master/.lando/syncdb.sh -# set -eu @@ -20,8 +18,11 @@ source "$WUNDERIO_GLOBAL_SCRIPT_ROOT/_helpers.sh" # Check if an alias was provided as an argument. if [[ -z "${1:-}" ]]; then display_error_message "Error: No site alias name provided." - display_warning_message "Usage: ddev syncdb your_alias_name" + display_warning_message "Usage: ddev syncdb [--keep-dump] [--backup] [--no-deploy]" display_warning_message "Example: ddev syncdb prod" + display_warning_message " --keep-dump Keep the downloaded dump file after import" + display_warning_message " --backup Create a local database backup before overwriting" + display_warning_message " --no-deploy Skip running drush deploy after import" exit 1 fi @@ -32,90 +33,126 @@ SITE_ALIAS="@${ALIAS_KEY}" shift 1 -# If --keep-dump is passed, keep the dump file +# Parse flags KEEP_DUMP=false +BACKUP=false +NO_DEPLOY=false for arg in "$@"; do - if [[ "$arg" == "--keep-dump" ]]; then - KEEP_DUMP=true - break - fi + case "$arg" in + --keep-dump) KEEP_DUMP=true ;; + --backup) BACKUP=true ;; + --no-deploy) NO_DEPLOY=true ;; + esac done -# Define the dumps directory at the project root. +# --- 2. Validate drush/sites/self.site.yml and alias upfront --- +SITE_YML="$PROJECT_ROOT/drush/sites/self.site.yml" +if [[ ! -f "$SITE_YML" ]]; then + display_error_message "Missing drush/sites/self.site.yml" + display_warning_message "This file defines SSH aliases for remote environments." + display_warning_message "See https://www.drush.org/13.x/site-aliases/ for format." + exit 1 +fi + +if ! grep -Fq -- "${ALIAS_KEY}:" "$SITE_YML"; then + display_error_message "Alias '${ALIAS_KEY}' not found in drush/sites/self.site.yml" + display_warning_message "Available aliases:" + grep -E '^[a-zA-Z]' "$SITE_YML" | sed 's/:$//' | while read -r alias; do + display_warning_message " - $alias" + done + exit 1 +fi + +# --- 3. Prepare dumps directory and warn about overwrite --- +display_warning_message "This will overwrite your local database with data from '$SITE_ALIAS'." + DUMPS_DIR="$PROJECT_ROOT/database_dumps" -# Check if the directory does not exist if [ ! -d "$DUMPS_DIR" ]; then mkdir -p "$DUMPS_DIR" - # Ignore everything we never want this folder or its contents to end up into git. + # Ignore everything — we never want this folder or its contents to end up in git. echo "*" > "$DUMPS_DIR/.gitignore" display_status_message "Directory '$DUMPS_DIR' and .gitignore file created." fi -# Define file name for the dump. -sql_file="$DUMPS_DIR/${ALIAS_KEY}-syncdb-$(date +'%Y-%m-%d').sql" +# Use .sql.gz extension for compressed dump. +sql_file="$DUMPS_DIR/${ALIAS_KEY}-syncdb-$(date +'%Y-%m-%d').sql.gz" -# Read the remote alias details from drush configuration. -# We pass the full alias (@prod) to Drush. -if ! alias_details=$(ddev drush sa "$SITE_ALIAS" 2>&1); then - # The command failed. The error message is now in the 'alias_details' variable. - display_error_message "Drush command failed." +# --- 4. Create local backup (if --backup) --- +if [[ "$BACKUP" == "true" ]]; then + backup_file="$DUMPS_DIR/backup-$(date +'%Y-%m-%d-%H%M%S').sql.gz" + display_status_message "Creating local database backup: $backup_file" + ddev export-db --gzip --file="$backup_file" + display_status_message "Backup saved." +fi - # Print the actual error message from Drush for better debugging. +# --- 5. Read remote alias details from Drush --- +if ! alias_details=$(ddev drush sa "$SITE_ALIAS" --format=yaml 2>&1); then + display_error_message "Drush command failed." echo "--------------------------------------------------" echo "$alias_details" echo "--------------------------------------------------" - display_error_message "Please ensure the alias '$ALIAS_KEY' is defined in your project's drush/sites/self.site.yml file." - exit 1 # Exit with a non-zero status code to indicate failure + exit 1 fi -# Parse Alias Details using yq -# DDEV might be injecting some messages to output so clean the output +# Parse Alias Details using a single yq call. +# DDEV might be injecting some messages to output so clean the output. alias_details_clean=$(echo "$alias_details" | sed -n '/@self/,$p') -# Dynamically build the yq query path using the alias key ("prod"). -prod_ssh_user=$(ddev yq '."@self.'"$ALIAS_KEY"'".user' <<< "$alias_details_clean") -prod_ssh_host=$(ddev yq '."@self.'"$ALIAS_KEY"'".host' <<< "$alias_details_clean") -prod_ssh_options=$(ddev yq '."@self.'"$ALIAS_KEY"'".ssh.options' <<< "$alias_details_clean") +# Dynamically build the yq query path using the alias key ("main"). +alias_full="@self.${ALIAS_KEY}" +read -r remote_ssh_user remote_ssh_host remote_ssh_options < <( + ddev exec -- yq -r ".\"$alias_full\" | [.user, .host, .ssh.options] | @tsv" <<< "$alias_details_clean" +) -# --- Validate parsed SSH details --- -if [[ -z "$prod_ssh_user" || "$prod_ssh_user" == "null" ]]; then +# --- 6. Validate parsed SSH details --- +if [[ -z "$remote_ssh_user" || "$remote_ssh_user" == "null" ]]; then display_error_message "Missing or invalid SSH user for alias '$ALIAS_KEY'." display_warning_message "Check your drush/sites/self.site.yml configuration for the 'user' field." exit 1 fi -if [[ -z "$prod_ssh_host" || "$prod_ssh_host" == "null" ]]; then +if [[ -z "$remote_ssh_host" || "$remote_ssh_host" == "null" ]]; then display_error_message "Missing or invalid SSH host for alias '$ALIAS_KEY'." display_warning_message "Check your drush/sites/self.site.yml configuration for the 'host' field." exit 1 fi -if [[ -z "$prod_ssh_options" || "$prod_ssh_options" == "null" ]]; then +if [[ -z "$remote_ssh_options" || "$remote_ssh_options" == "null" ]]; then display_error_message "Missing or invalid SSH options for alias '$ALIAS_KEY'." display_warning_message "Check your drush/sites/self.site.yml configuration for the 'ssh.options' field." exit 1 fi -# Test SSH connection - allow passphrase prompt to be visible -ssh_command=(ssh $prod_ssh_options "$prod_ssh_user@$prod_ssh_host") +# Test SSH connection - allow passphrase prompt to be visible. +# Use SSH -C (compression) as a fallback for transfer speed. +ssh_command=(ssh -C $remote_ssh_options "$remote_ssh_user@$remote_ssh_host") if ! "${ssh_command[@]}" "true"; then display_error_message "Failed to establish SSH connection to $ALIAS_KEY. Check your network and credentials." exit 1 fi -# --- Perform the Database Dump and Import --- -display_status_message "Dumping database from '$SITE_ALIAS'..." +# --- 7. Perform the Database Dump and Import --- +display_status_message "Dumping database from '$SITE_ALIAS' (gzip compressed)..." -"${ssh_command[@]}" "drush sql-dump --structure-tables-list=cache,cache_*,history,search_*,sessions" > "$sql_file" +"${ssh_command[@]}" "drush sql-dump --gzip --structure-tables-list=cache,cache_*,history,search_*,sessions" > "$sql_file" display_status_message "Dump complete, starting import!" +# Place marker file in the named volume so the post-import hook can read it. +if [[ "$NO_DEPLOY" == "true" ]]; then + ddev exec sudo touch /mnt/wdr-hooks/.no-deploy + trap 'ddev exec sudo rm -f /mnt/wdr-hooks/.no-deploy' EXIT +fi + +# ddev import-db natively handles .gz files. ddev import-db --file="$sql_file" + if [[ "$KEEP_DUMP" != "true" ]]; then rm "$sql_file" fi + { set +x; } 2>/dev/null display_status_message "Sync with '$SITE_ALIAS' complete!" diff --git a/wunderio/core/wdr-core.sh b/wunderio/core/wdr-core.sh index 2eb6884..8e28f08 100755 --- a/wunderio/core/wdr-core.sh +++ b/wunderio/core/wdr-core.sh @@ -38,7 +38,10 @@ else else PROJECT_ROOT="$PWD" fi + # Ensure DDEV_APPROOT is set for any downstream consumers. + export DDEV_APPROOT="$PROJECT_ROOT" fi + export PROJECT_ROOT # Remove the first argument (the method)