Add mydumper/myloader DB backup and restore support

[stefannica]

Describe changes

Implements a mydumper/myloader DB backup strategy.

The mydumper/myloader DB backup strategy is far superior in terms of speed and used storage space:

• 30x faster database backup times
• 2.5x faster database restore times
• 10x lower storage space required

Example:

• the file dump strategy:

$ time zenml backup-database -s dump-file
Database was backed up to /home/silent-assassin/.config/zenml/database_backup/zenml-backup.json.

real	3m38.208s
user	0m22.391s
sys	0m1.246s

$ du -sh /home/silent-assassin/.config/zenml/database_backup/zenml-backup.json
794M	/home/silent-assassin/.config/zenml/database_backup/zenml-backup.json

$ time zenml restore-database -s dump-file
Dropping database 'zenml'
Creating database 'zenml'
Database was restored from /home/silent-assassin/.config/zenml/database_backup/zenml-backup.json.

real	2m0.481s
user	1m22.547s
sys	0m0.997s

• vs. the new mydumper strategy (with compression enabled):

$ time zenml backup-database -s mydumper
Starting mydumper backup of database zenml to /home/silent-assassin/.config/zenml/database_backup/zenml-backup
Database zenml successfully backed up to /home/silent-assassin/.config/zenml/database_backup/zenml-backup
Database was backed up to /home/silent-assassin/.config/zenml/database_backup/zenml-backup.

real	0m7.041s
user	0m13.646s
sys	0m0.395s

$ du -sh /home/silent-assassin/.config/zenml/database_backup/zenml-backup
82M	/home/silent-assassin/.config/zenml/database_backup/zenml-backup

$ time zenml restore-database -s mydumper
Dropping database 'zenml'
Creating database 'zenml'
Starting myloader restore of database zenml from /home/silent-assassin/.config/zenml/database_backup/zenml-backup
Database zenml successfully restored from /home/silent-assassin/.config/zenml/database_backup/zenml-backup
Database was restored from /home/silent-assassin/.config/zenml/database_backup/zenml-backup.

real	0m48.295s
user	0m4.312s
sys	0m0.475s

Other related changes:

• refactored the existing database backup code to be cleaner, extensible and maintainable
• allows implementing and configuring a custom database backup/restore strategy
• extends the helm chart to support the mydumper/myloader and custom backup strategies

Implementation details

The previous MigrationUtils class that held all the code required for all the available database backup strategies has been reorganized into a hierarchy of classes:

• the BaseDatabaseBackupEngine class defines the database backup/restore engine abstraction. It contains several common utilities such as database creation/deletion and the backup_database , restore_database and cleanup_database_backup abstract methods that every backup engine has to implement.
• the SQLAlchemyDatabaseBackupEngine is a base class that uses SQLAlchemy to dump/load data from/to the database. It is used as a base class to implement the existing DB strategies: in-memory, file-dump and database cloning. These are implemented in the InMemoryDatabaseBackupEngine, FileDatabaseBackupEngine and DBCloneDatabaseBackupEngine classes respectively.
• the new mydumper/myloader backup strategy is implemented in its own MyDumperDatabaseBackupEngine class that extends directly from BaseDatabaseBackupEngine

A lot of code that was spread across the SQLZenStore implementation has been moved to the base backup engine class and its implementations, so that the SQLZenStore class is no longer concerned with the backup/restore functionality.

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• I have added tests to cover my changes.
• I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
• IMPORTANT: I made sure that my changes are reflected properly in the following resources:
  • ZenML Docs
  • Dashboard: Needs to be communicated to the frontend team.
  • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
  • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[github-actions]

Documentation Link Check Results

✅ Absolute links check passed
✅ Relative links check passed
_{Last checked: 2025-12-17 19:08:50 UTC}

[Json-Andriopoulos]

We do those validations repeatedly (both on dump and restore). Some validations we do on the property level (e.g. on def backup_location. It would be good for code clarity to group those validations and execute them once at instantiation? Maybe we could pass them as a pydantic params object and handle any validations there?

[stefannica]

This is in fact guaranteed to be not null, but had to do something like this to please mypy.

[schustmi]

Love this Pr! Only left a few small questions

[Json-Andriopoulos]

I like the solution and the refactoring a lot. Left some comments - non blocking - so feel free to address them at your convenience.

[github-actions]

Classification template updates in examples/mlops_starter have been pushed.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[github-actions]

E2E template updates in examples/e2e have been pushed.