Skip to content

Docs: Add praisonai validate CLI page and update YAML validation behaviour (PR #2128) #725

Description

@MervinPraison

Context

Upstream PR: MervinPraison/PraisonAI#2128fix: add fail-fast YAML schema validation with actionable errors (merged, fixes #2125).

The PR ships a brand-new top-level CLI command praisonai validate (with validate, validate check, validate schema subcommands), backed by a new Pydantic schema (praisonai/config/schema.py) and a comprehensive validator (praisonai/config/validator.py). It also changes runtime behaviour: invalid agents.yaml / workflow.yaml configs now fail fast with aggregated errors instead of emitting non-blocking warnings.

Both effects need documentation. This issue is for the docs agent that will create / update the pages — every fact below was extracted from PR #2128's diff so the writer can build the pages without re-reading the SDK.


What changed in PR #2128

1. New CLI command — praisonai validate

Registered in src/praisonai/praisonai/cli/app.py:

"validate": (".commands.validate", "app", "Validate YAML configuration files"),

Implemented in src/praisonai/praisonai/cli/commands/validate.py (Typer app, no_args_is_help=True). Three subcommands:

Command Purpose
praisonai validate <file> Validate a single YAML file
praisonai validate check [directory] Validate every YAML file in a directory
praisonai validate schema Print the YAML config schema (fields, types, required flags)

praisonai validate <file> — flags

Flag Type Default Description
file (positional) str Path to YAML config to validate
--strict bool False Treat warnings as errors
--quiet / -q bool False Only show errors, suppress success messages
--json bool False Emit results as JSON (for CI)

JSON output shape:

{
  "file": "agents.yaml",
  "valid": true,
  "errors": [],
  "warnings": [],
  "strict_mode": false
}

Exit code is 0 on valid, 1 on invalid or missing file.

praisonai validate check [directory] — flags

Flag Type Default Description
directory (positional) str "." Directory to search
--pattern / -p str "*.yaml" Glob pattern (also auto-includes *.yml when the pattern is the default)
--strict bool False Treat warnings as errors
--stop-on-error bool False Stop on first invalid file

Renders a Rich table with File | Status | Errors | Warnings columns plus a summary block. Exit code is 0 if every file is valid, else 1.

praisonai validate schema

Prints, in this order:

  • Main configuration sections (roles/agents, tasks, workflow, tools/toolsets, config)
  • Agent/Role fields (from AgentConfig.model_json_schema()) with required markers and types
  • Task fields (from TaskConfig.model_json_schema()) with required markers and types

2. Runtime behaviour change — fail-fast validation

In src/praisonai/praisonai/agents_generator.py, _validate_agents_config was rewritten to call the new ConfigValidator. Previously: emitted logger.warning(...) for unknown fields and bad models. Now:

  • Warnings are still logged via self.logger.warning
  • If any error is found, the run aborts with raise ValueError(...) containing an aggregated multi-line message:
    Configuration validation failed with N error(s):
      1. <error>
      2. <error>
    
    Additionally, there are M warning(s):
      1. <warning>
    

Strict mode is also reachable globally via the env var PRAISONAI_VALIDATE_STRICT=true.

3. New module praisonai/config/

Exposes (via praisonai/config/__init__.py):

  • YAMLConfig, AgentConfig, TaskConfig, WorkflowConfig, WorkflowStep
  • ValidationResult
  • ProcessType (sequential | hierarchical | consensual | workflow)
  • HandoffPolicy (any | all | round_robin | least_busy)
  • ToolRetryPolicy, HandoffConfig, ApprovalConfig, RuntimeConfig, CliBackendConfig, GlobalConfig
  • ConfigValidator

Required vs optional fields enforced by the schema

AgentConfig required: role, goal, backstory.
TaskConfig required: description, agent (also validated against regex ^[a-zA-Z0-9_-]+$).
YAMLConfig requires at least one of roles / agents; if process == workflow, requires steps or workflow.

Backward-compatible aliases the validator normalises

Alias Canonical
agents: roles:
topic: input:
stream: streaming:

WorkflowStep type rules

  • Allowed type: task | parallel | loop | route
  • task requires agent and task
  • parallel / loop require steps; loop additionally requires count
  • route requires routes

4. Cross-reference validation (new)

YAMLConfig.validate_cross_references() checks:

  • Every tasks[i].agent exists in roles/agents
  • Every workflow task step agent exists (recurses through nested steps and routes)
  • Every agent.handoff.to target resolves to a known role

ConfigValidator._validate_tools additionally resolves every tool name through ToolResolver:

  • Unknown tool ⇒ error: "Unknown tool 'X'. Ensure it's properly installed or defined in your configuration."
  • Known-optional tool (curated list incl. PostgreSQLTool, SlackTool, AWSTool, BrowserTool, PandasTool, KubernetesTool, …) ⇒ warning: "Tool 'X' requires additional dependencies. Install with: pip install 'praisonai[tools]' …"

5. Unknown-field detection (warning)

Top-level unknown keys and unknown agent/role fields are reported as warnings: "Unknown … field 'X'. This field will be ignored." The previous "did you mean …?" fuzzy-match suggestions are gone — the docs page must not promise them anymore.

6. New dependency

src/praisonai/pyproject.toml adds pydantic>=2.0.0.


Required docs work

Place all new pages in docs/features/ (or docs/cli/). Per AGENTS.md, do not touch docs/concepts/.

A. NEW page — docs/cli/validate.mdx

Document the praisonai validate command end-to-end. Suggested frontmatter:

---
title: "Validate"
sidebarTitle: "Validate"
description: "Fail-fast YAML configuration validation with aggregated, actionable errors"
icon: "circle-check"
---

Sections to include (follow the standard template in AGENTS.md):

  1. One-line intro + hero Mermaid diagram — show: YAML → Schema check → Cross-ref check → Tool check → ✓ Valid / ✗ Errors.
  2. Quick Start<Steps>:
    • Step 1: praisonai validate agents.yaml (success path)
    • Step 2: praisonai validate agents.yaml (failure path, show aggregated error output)
    • Step 3: praisonai validate agents.yaml --strict --json (CI usage)
  3. Subcommands — table:
    • validate <file> — single file
    • validate check [dir] — directory scan
    • validate schema — print schema
  4. Flags table — every flag from §1 above, with type / default / description.
  5. What gets validated — Mermaid flowchart of the four checks:
    • Schema (required fields, types, Pydantic models)
    • Cross-references (task→agent, workflow step→agent, handoff→role)
    • Tool references (resolved via ToolResolver; lists curated optional tools)
    • Unknown fields (warning, not error)
  6. Output formats — show plain (Rich) output, --quiet output, and --json output. Include the JSON shape from §1.
  7. CI integration example — a short GitHub Actions snippet using praisonai validate check . --strict --json.
  8. Strict mode — explain --strict flag + PRAISONAI_VALIDATE_STRICT=true env var (global strict mode).
  9. Exit codes0 valid, 1 invalid / file not found.
  10. Related — link to yaml-configuration-reference, cli, doctor.

B. UPDATE — docs/features/yaml-configuration-reference.mdx

The current "Automatic Field Validation" section (around the "PraisonAI automatically checks every field name … and warns you about typos" copy) is stale after this PR. Required changes:

  1. Behaviour rewrite. Replace the "Warnings are non-blocking. Your workflow still runs" Note with a <Warning> that says configuration errors now fail fast with ValueError, listing every error at once. Keep the "warnings are still non-blocking" framing only for unknown-field warnings.
  2. Drop the "Did you mean …?" claim. The new validator no longer emits difflib-based suggestions. The "How Suggestions Work" subsection and the matching sample output line should be removed or rewritten.
  3. Add a "Fail-Fast Errors" subsection with:
    • An example of a missing-required-field error (e.g. agent without goal) and the resulting aggregated error block.
    • An example of a bad cross-reference (task references undefined agent) and its error message.
    • An example of an unknown tool error vs an optional-tool warning.
  4. Update the bottom "Validation" section. Currently it points only at praisonai workflow validate. Add praisonai validate <file> as the new canonical command and link to the new page from §A. Keep workflow validate mentioned as the workflow-specific variant.
  5. Mention PRAISONAI_VALIDATE_STRICT in either Validation or Best Practices.
  6. Required fields callout. Add a small note that role, goal, backstory are now schema-enforced as required for every agent.
  7. Add backward-compat aliases callout so users see that agents↔roles, topic↔input, stream↔streaming are still accepted and normalised by the validator.

C. UPDATE — docs/features/cli.mdx

The page currently lists praisonai workflow validate in the commands table. Add a new row for praisonai validate <file> (and praisonai validate check, praisonai validate schema) and link to the new page from §A.

D. Sidebar / docs.json

Add the new cli/validate page under the appropriate CLI group in docs.json. Do not add it under any "Concepts" group (see AGENTS.md §1.8).


Style requirements (from AGENTS.md)

  • Agent-centric, beginner-friendly tone. Lead each section with one short sentence.
  • Use Mintlify components: <Steps>, <AccordionGroup>, <CardGroup>, <Tabs>, callouts.
  • Hero Mermaid diagram using the standard palette (#8B0000, #189AB4, #10B981, #F59E0B, #6366F1, white text, #7C90A0 stroke).
  • All code examples must run as-is — use realistic but minimal YAML (no your-key-here placeholders).
  • Don't claim "Did you mean …?" suggestions — PR #2128 removed them.
  • Don't document docs/concepts/.

Verification checklist for the writer

  • docs/cli/validate.mdx created and added to docs.json
  • docs/features/yaml-configuration-reference.mdx updated (fail-fast behaviour, suggestions removed, new validation section)
  • docs/features/cli.mdx lists the new top-level command
  • All flag tables match the source in src/praisonai/praisonai/cli/commands/validate.py
  • JSON output shape matches the one above
  • PRAISONAI_VALIDATE_STRICT env var mentioned
  • No references to removed difflib "Did you mean" suggestions remain anywhere in the docs
  • Sidebar entry NOT placed under "Concepts"

Source of truth (read before writing)

File in MervinPraison/PraisonAI @ commit 9d2b354 What it defines
src/praisonai/praisonai/cli/commands/validate.py CLI surface: subcommands, flags, output formats
src/praisonai/praisonai/config/schema.py Pydantic models, required fields, enums, step type rules
src/praisonai/praisonai/config/validator.py Validation flow, tool resolution, optional-tool list, error/warning wording
src/praisonai/praisonai/config/__init__.py Public exports
src/praisonai/praisonai/agents_generator.py (_validate_agents_config) Runtime fail-fast behaviour, PRAISONAI_VALIDATE_STRICT env var
src/praisonai/praisonai/cli/app.py Command registration

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingclaudeTrigger Claude Code analysisdocumentationImprovements or additions to documentation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions