Update DockerHub flavor Container Registry documentation for using open source Quay remote registry. #4036
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: ZenML GmbH <[email protected]> (cherry picked from commit be4f3ed)
(cherry picked from commit 91bf1a3638d959bde62d0dcd94fc51ec306f4351) (cherry picked from commit e93c6a6)
(cherry picked from commit 7e4f61b)
* Document pipeline deployments in the concepts section * Add more information about deployments * Add deployers base docs * Add more information around deployers * Move deployment section top level * Fix the TOC * Update docs/book/how-to/deployment/deployment.md Co-authored-by: Alex Strick van Linschoten <[email protected]> * Fix URLs * Update docs/book/how-to/deployment/deployment.md Co-authored-by: Alex Strick van Linschoten <[email protected]> * Add GCP and Docker deployer documentation. * Add AWS App Runner deployer docs and fix CLI indentation * Fix TOC * Apply suggestion from @strickvl Co-authored-by: Alex Strick van Linschoten <[email protected]> * Apply code review suggestions * Update docs/book/component-guide/deployers/aws-app-runner.md Co-authored-by: Alex Strick van Linschoten <[email protected]> * Fix broken link --------- Co-authored-by: Alex Strick van Linschoten <[email protected]> (cherry picked from commit 08b6e45)
* Snapshot docs * Replace all mentions of run templates with snapshots * Update docs/book/how-to/snapshots/snapshots.md Co-authored-by: Alex Strick van Linschoten <[email protected]> * Some PR comment fixes * Change pics * Optimised images with calibre/image-actions * Apply suggestions from code review Co-authored-by: Alex Strick van Linschoten <[email protected]> * Core concept + code block fixes * Better snapshot description * Revert some links back to absolute --------- Co-authored-by: Alex Strick van Linschoten <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> (cherry picked from commit 7ce5488)
* add mlops starter and readme
* update minimal_agent
* quickstart
* format
* Auto-update of Starter template
* New quickstart
* Update deployer and stack registration for Docker
* Update agent serving pipeline to remove confidence threshold
* Add missing comma in classify_intent function parameters
* Add ZenML Quickstart example notebook
* Update README with ZenML pipeline modes and OpenAI key.- Add support for OpenAI in agent.yaml.- Integrate LLM responses and classifier upgrades in agent pipeline.- Log metadata for training and response generation
* Update quickstart README with key ZenML features
* Upgrade to structured, targeted agent responses
* lint and docstring
* Update pipeline docstrings with Args and Returns
* Refactor typing to remove Optional for boolean inputs
* Rename agent_serving_pipeline.py to support_agent.py<commit message> Rename agent_serving_pipeline to support_agent
* Update import statement in visualization module
* Update quickstart README with additional details
* lint and docstring
* Update payload structure for ZenML deployment API
* Remove unnecessary code and display analysis results in a structured layout
* Add DocumentAnalysis to pipelines/__init__.py & refactor code
* Remove unused static files and associated styles
* New quickstart
* Code review
* README updates
* README updates
* Removed future imports
* Removed future imports
* README updates
* README updates
* Update import order and use requirements file for installation
* Update README.md for direct agent pipeline run explanation
* Add weather agent example
* Update examples/minimal_agent_production/streamlit_app.py
* Update examples/quickstart/utils.py
* Update examples/quickstart/steps/evaluate.py
* Update examples/quickstart/steps/data.py
* Update examples/minimal_agent_production/README.md
* Update examples/minimal_agent_production/steps/analyze.py
* Update examples/minimal_agent_production/steps/ingest.py
* Update examples/minimal_agent_production/requirements.txt
* Update examples/quickstart/steps/train.py
* Update examples/minimal_agent_production/pipelines/__init__.py
* Update README for minimal agent production examples
This commit updates the README.md file for the minimal agent production examples to reflect the current structure and content of the example files. The changes ensure that the descriptions for each script are accurate and up-to-date.
No functional changes were made to the code; this is purely a documentation update.
* Extract HTML and CSS into template files for render step
Refactors the render_analysis_report_step to use external template files
instead of inline strings for better maintainability and learning patterns.
- Created templates/report.css with all report styling
- Created templates/report.html with HTML structure and format placeholders
- Updated render.py to load and populate templates using Path-based resolution
- Reduced render.py from 302 to 89 lines for improved readability
This makes the templates easier to customize and provides a clearer
separation between presentation logic and data processing.
* Fix model label to reflect actual analysis method used
Previously, the model field was always set to "openai-gpt-4o-mini" even
when the deterministic fallback was used, which doesn't use OpenAI at all.
Now correctly sets:
- "openai-gpt-4o-mini (llm)" when LLM analysis succeeds
- "rule-based (deterministic)" when fallback is used
This ensures accurate metadata for lineage tracking, cost attribution,
and quality assessment of pipeline runs.
* Remove deleted functions from __all__ export list
Cleaned up __all__ to only include functions that are actually imported
and available in the module. Removed references to:
- aggregate_evaluation_results_step
- annotate_analyses_step
- load_recent_analyses
- render_evaluation_report_step
These functions were previously deleted but their names remained in the
export list, which could cause import errors.
* Add Literal types for sentiment and readability values
Introduces type-safe constraints for sentiment and readability fields:
- SentimentType: Literal["positive", "negative", "neutral"]
- ReadabilityType: Literal["easy", "medium", "hard"]
Applied to:
- AnalysisResponse.sentiment and .readability fields
- DocumentAnalysis.sentiment field
This provides IDE autocomplete, type checking at development time, and
Pydantic validation at runtime to prevent invalid values from being
assigned to these fields throughout the pipeline.
* Remove duplicate stop words from get_common_stop_words
Removed duplicate entries in the stop words set:
- 'will' (was listed twice)
- 'were' (was listed twice)
Sets automatically deduplicate in Python, but having duplicates in the
source code is confusing and suggests copy-paste errors.
* Add code fences
* Fix model label tracking and deduplicate constants
Implements compliance fixes to ensure model labels reflect actual models used
and eliminate duplicate constant definitions across the codebase.
Key changes:
- Return used_model from perform_llm_analysis to track actual model
- Use actual model from analysis result for label formatting
- Deduplicate extension→type mapping in Streamlit using shared constants
- Update Streamlit banner to show actual model from analysis object
- Fix pipelines/__init__.py by removing unimported DocumentAnalysis export
These changes ensure runtime accuracy over build-time constants, preventing
scenarios where displayed model names don't match what was actually used for
analysis (e.g., when users override default model parameters).
* Update default LLM model to gpt-5-mini
* Update the README so it's accurate
* cleanup
* Formatting
* Update production notes in README for clarity
Enhanced the production notes section in the README to provide clearer instructions on deploying with Docker settings. Updated the example configuration file name and deployment command to reflect optional user-defined configurations.
This change aims to improve user experience by ensuring accurate and helpful deployment guidance.
* Update README examples for consistency in deployment commands
Replaced hyphens with underscores in the deployment command examples within the README to ensure consistency and accuracy. This change aligns with the updated naming conventions for deployment identifiers, enhancing clarity for users.
No functional changes were made; this is purely a documentation update to improve user experience.
* Extract get_winner_info to module level
Makes the function independently testable and reusable rather than
being nested inside render_evaluation_template. The function now
takes explicit parameters instead of relying on closure scope.
* Extract LLM prompts to dedicated prompts module
Move all LLM prompt content from utils.py into a new prompts.py module
to improve code organization and maintainability.
Changes:
- Create examples/quickstart/prompts.py with:
- INTENT_CLASSIFICATION_PROMPT constant
- INTENT_CONTEXTS mapping
- TEMPLATE_RESPONSES mapping
- RESPONSE_PROMPT_TEMPLATE constant
- build_intent_classification_prompt(text) builder
- build_response_prompt(original_text, intent) builder
- Update examples/quickstart/utils.py:
- Add dual-path imports to support both local and orchestrated execution
- Remove inline prompt constants
- Replace manual prompt formatting with builder functions
- call_llm_for_intent now uses build_intent_classification_prompt()
- generate_llm_response now uses build_response_prompt()
The prompts module is dependency-free (no OpenAI/ZenML imports) and
centralizes all prompt text in one place for easier maintenance.
* Remove unused call_llm_for_intent function
The function was never imported or called anywhere in the codebase.
The ruff formatter also removed the now-unused build_intent_classification_prompt import.
* Remove unused shebang from run.py
The shebang was incorrectly placed after the license header (line 17)
where it has no effect. Shebangs must be on line 1 to work. The file
is always invoked as 'python run.py' per the documentation.
* Escape HTML template variables to prevent rendering issues
Add html.escape() to string variables before inserting them into HTML
templates. This prevents issues with special characters like <, >, &,
and quotes. CSS styles are explicitly excluded from escaping since they
need to be raw CSS.
* Update "Your First AI Pipeline" documentation
Revised the "Your First AI Pipeline" guide to reflect changes in terminology and functionality. The section now emphasizes deploying a ZenML pipeline as a managed HTTP endpoint, with options for invoking via CLI or curl. Additional updates include clarifying the architecture, prerequisites, and deployment instructions, as well as enhancing the descriptions of tracked artifacts and metadata.
This update aims to improve clarity and user experience for new users starting with ZenML.
* Update quickstart README with agent pipeline execution instructions
Added instructions for running the agent pipeline directly in batch mode within the quickstart README. This update aims to enhance user understanding of executing the pipeline without deployment, providing a clear example command for users to follow.
No functional changes were made; this is a documentation improvement to support user experience.
* Fix minimal agent README instructions and add image
* Update "Your First AI Pipeline" documentation for clarity
Revised the "Your First AI Pipeline" guide to enhance clarity and user experience. Changes include improved phrasing for invoking the deployment via the ZenML CLI and curl, as well as the addition of a Streamlit web interface section. This update aims to provide clearer instructions for new users on building and deploying a ZenML pipeline as a managed HTTP endpoint.
No functional changes were made; this is a documentation improvement to support user understanding.
* Add cloud training configs and automated release workflow for quickstart
Enhances the quickstart example with cloud orchestrator support and automates
the release validation workflow for AWS, Azure, and GCP deployments.
Changes:
- Add cloud-specific training configs (training_{aws,azure,gcp,default}.yaml)
- Generate cloud-specific requirements files with integration dependencies
- Support --config flag in run.py for both training and evaluation steps
- Create generate_cloud_requirements.sh to automate requirements generation
- Create qs_run_release_flow.sh to orchestrate release validation flow
- Update release workflow to use new automated scripts
- Add Docker availability checks in release workflow
- Document that cloud configs are for CI validation only
The cloud configs enable testing of the quickstart example on production cloud
orchestrators during the release process while keeping the local development
experience simple.
* Add featured companies logos to README.md
* Update company logos with border-radius and padding
* Update architecture overview link to documentation
* Update featured company logos in README.md
* Update company logos in README.md
* Update featured company logos in README.md
* Update featured company links in README.md
* Add streamlined quickstart testing workflow
Create testing.yml workflow to validate new quickstart implementation on PR.
Simplified from release_prepare.yml by removing all release automation
(version bumping, PR creation, Discord notifications, file updates).
Focus solely on testing quickstart pipelines across AWS/Azure/GCP:
- Extract version from VERSION file (works with any branch)
- Build test Docker images
- Setup release tenant
- Run quickstart on all three cloud providers
This allows us to validate the new sklearn-based quickstart before merging.
* Remove OPENAI_API_KEY from training pipeline configs
The sklearn-based training pipeline doesn't use OpenAI, so remove
OPENAI_API_KEY from:
- Training pipeline definition (intent_training_pipeline.py)
- All training config files (training_*.yaml)
Agent pipeline correctly retains OPENAI_API_KEY in agent.yaml.
This fixes CI failures where OPENAI_API_KEY environment variable
substitution was failing during training pipeline compilation.
* Add OPENAI_API_KEY secret to quickstart pipeline workflows
The agent deployment phase requires OPENAI_API_KEY to deploy the
OpenAI-powered support agent pipeline. Add the secret to both:
- testing.yml (test workflow)
- release_prepare.yml (release workflow)
The repository already has this secret configured (used in
weekly-agent-pipelines-test.yml), it just wasn't being passed
to the run_quickstart_pipelines job.
This fixes agent deployment failures with:
"Unable to substitute environment variable placeholder 'OPENAI_API_KEY'"
* Fix
* Delete test workflow
---------
Co-authored-by: GitHub Actions <[email protected]>
Co-authored-by: Hamza Tahir <[email protected]>
Co-authored-by: Hamza Tahir <[email protected]>
Co-authored-by: Alex Strick van Linschoten <[email protected]>
Co-authored-by: Stefan Nica <[email protected]>
Co-authored-by: Alex Strick van Linschoten <[email protected]>
(cherry picked from commit 118c799)
|
|
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
schustmi
requested changes
Oct 6, 2025
There was a problem hiding this comment.
Question for discussion: The "dockerhub" flavour name is misleading as it implies that it only works with dockerhub and not other remote container registries. A better flavour name could be "remote_docker", which would be consistent with the existing "local_docker" flavour.
There is a default flavor for container registries, which is meant to be used for any container registry which doesn't fit any of the other flavors. In reality however, the default and dockerhub container registries are currently functionally equivalent, as both of them only use the python docker client to push images.
However: In the future, the dockerhub flavor implementation might actually implement logic that only works for dockerhub container registries. So while I think all your documentation makes sense, I think it should instead be for the default flavor.
|
Also, regarding the Kaniko changes: I wasn't aware of this, but that's great news. Adding it to the docs however is a little misleading I think, as our current implementation still uses the unmaintained container images. Would you mind removing that from the docs here, and instead creating a GH issue for us to update the integration to this new fork? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe changes
Updated the DockerHub documentation to show how to use open source docker image registry, Quay. I've also left space for other FOSS remote image registries, such as Harbor.
Question for discussion: The "dockerhub" flavour name is misleading as it implies that it only works with dockerhub and not other remote container registries. A better flavour name could be "remote_docker", which would be consistent with the existing "local_docker" flavour.
Also minor update to Kaniko documentation.
Pre-requisites
Please ensure you have done the following:
developand the open PR is targetingdevelop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes