Agent guide docs

[htahir1]

Describe changes

Added an agent guide

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• I have added tests to cover my changes.
• I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
• IMPORTANT: I made sure that my changes are reflected properly in the following resources:
  • ZenML Docs
  • Dashboard: Needs to be communicated to the frontend team.
  • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
  • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing Touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch doc/agentframeworkexamplesiwthguide

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[strickvl]

High-level comments

• somewhere you should make the distinction / mention what's the difference between LLMOps guide + agent guide. (maybe also at the top of both of those you should say 'oh, there's also this other guide which covers x and y and its purpose is z which you might want to look at also… and also link to it at the end of the respective guides)
• I think at a high level everything checks out (apart from the conditional routing etc in chapter 1), but as is this will raise many many questions, so IMO it needs to be fully expanded. (like the other guides)

chapter 1: dev workflow

• the dev workflow where you compare two different agents IMO is not how people are developing. Rather people are comparing two different versions of the same agents, but with different prompts. I think that's something that def needs changing in the 'development & experimentation' section.
• the complete_agent_development_pipeline doesn't make sense to me. I'd just delete that whole "Complete Development Pipeline" section. I don't know that anyone would ever (want to) do it that way. Also this guide doesn't need that.
• For my taste, that 'development and experimentation' section is sort of the real meat of the guide and we don't really get into the details of the 'why' you might want to do one vs the other etc. These guides were always sort of educational as well as practical, so somehow I'd expect more prose amidst the code snippets. After all, this section is sort of the big sell… + must answer the question why should I use ZenML when experimenting on my agents instead of just doing it outside ZenML.

Chapter 2: production & deployment

• we sort of miss a step here where we're logging things to the model control plane, since in the 'deploying your agent' it assumes that we've already done that?
• again here I don't think it's a realistic pattern that we're deciding in a conditional which config to use ('direct' or 'framework' etc.) That would have happened much earlier in the process.
• I think I'd want to see more of a story around 'versions' here. You're doing it in code, but it's somehow a bit implicit / not foregrounded?
• The bit about a/b testing + traffic_split was maybe an interesting idea, but it feels a bit half-baked since the exact implementation of that is hidden away.
• At the end you say 'the production logs + traces you're collecting are useful' etc, but AFAICS we didn't talk about that yet at all…

Chapter 3: evaluation

• For me, the high-level code examples make sense but for many people I can see them as prompting more questions than they answer. I.e. people will want the details. "how do you write an llm as judge eval" / 'what model to use' / 'how to create a test set' etc..
• I think maybe this page is fine for now, but going forward I think we're going to have to write up all the details of our take on this stuff as we'll be asked these questions repeatedly. (or point them to somewhere else that does answer them)

Language use

• I think we can be a bit more deliberate about how / when we use agents vs workflows. e.g. in very first sentence of the guide, I read "Transform your agent development from experimental scripts into systematic, production-ready workflows using ZenML as your agent development platform." which somehow muddies the water a bit. (unless that's the point? if so, not sure that's the best way to handle that. IMO we should just lean into the word 'agent')
• Not sure 'chapter' is the right word, since the sections aren't labelled with numbers…
• The section "What ZenML tracks automatically" is maybe a bit misleading. I mean we don't track any of that 'automatically'. We only track it if you hook it up to a step output or input, right?

Nits

• just start with python 3.10 instead of 3.9+ since 3.9 is not long for this world (deprecated next month)
• In the 'What You'll Learn' in the README.md, the 'After chapter 1' etc stuff doesn't get properly laid out / rendered in gitbook. Needs fixing.
• It's not "the OpenAI SDK" but "the OpenAI Agents SDK"
• update 'gpt-4' anywhere / everywhere to 'gpt-5'
• obviously the images need zurification