databrickslabs · anshuroy-db · Sep 26, 2025 · Sep 26, 2025 · Oct 2, 2025 · Oct 2, 2025
diff --git a/README.md b/README.md
@@ -35,7 +35,7 @@ Quick setup for testing and development - requires Python 3.9+ and Node.js.
 
 ## See It in Action
 
-![Kasal UI Screenshot](src/docs/images/kasal-ui-screenshot.png)
+![Kasal UI Screenshot](src/images/kasal-ui-screenshot.png)
 *Visual workflow designer for creating AI agent collaborations*
 
 Create your first agent workflow in under 2 minutes:
@@ -50,20 +50,15 @@ Create your first agent workflow in under 2 minutes:
 
 | Topic | Description |
 |-------|-------------|
-| **[Getting Started](src/docs/GETTING_STARTED.md)** | Complete setup guide for development and deployment |
-| **[Deployment Guide](src/docs/DEPLOYMENT_GUIDE.md)** | Build, deployment, and OAuth configuration |
-| **[Architecture](src/docs/ARCHITECTURE.md)** | System architecture and design patterns |
-| **[CrewAI Engine](src/docs/CREWAI_ENGINE.md)** | AI agent orchestration engine documentation |
-| **[Database Migrations](src/docs/DATABASE_MIGRATIONS.md)** | Database schema management with Alembic |
-| **[Models & Schemas](src/docs/MODELS.md)** | SQLAlchemy models and Pydantic schemas |
-| **[Repository Pattern](src/docs/REPOSITORY_PATTERN.md)** | Data access layer implementation |
-| **[LLM Manager](src/docs/LLM_MANAGER.md)** | Multi-provider LLM configuration and management |
-| **[Embeddings](src/docs/EMBEDDINGS.md)** | Documentation embeddings for enhanced crew generation |
+| **[Why Kasal](src/docs/WHY_KASAL.md)** | What problems it solves and who it's for |
+| **[Solution Architecture](src/docs/ARCHITECTURE_GUIDE.md)** | Layers, lifecycles, and platform integration |
+| **[Code Structure](src/docs/CODE_STRUCTURE_GUIDE.md)** | Where things live and how to navigate the repo |
+| **[Developer Guide](src/docs/DEVELOPER_GUIDE.md)** | Local setup, config, and extension patterns |
+| **[API Reference](src/docs/API_REFERENCE.md)** | REST endpoints, payloads, and errors |
 
 ### More Documentation
-- **[API Documentation](src/docs/)** - Complete API reference
-- **[Best Practices](src/docs/BEST_PRACTICES.md)** - Development guidelines
-- **[Security Model](src/docs/SECURITY_MODEL.md)** - Authentication and authorization
+- **[Docs Hub](src/docs/README.md)** - Documentation index
+- **[End‑User Tutorial Catalog](src/docs/END_USER_TUTORIAL_CATALOG.md)** - Screenshot-ready walkthroughs
 - **[Testing Guide](src/backend/tests/README.md)** - Testing strategy and coverage
 
 ---

diff --git a/src/docs/API_REFERENCE.md b/src/docs/API_REFERENCE.md
@@ -5,6 +5,7 @@
 ---
 
 ## Getting Started
+Base URLs, authentication, and rate limits you need before calling endpoints.
 
 ### Base URL
 ```
@@ -35,6 +36,7 @@ curl -X GET https://api.example.com/v1/crews \
 ---
 
 ## Authentication Endpoints
+Login, refresh, and logout flows to manage tokens.
 
 ### POST /auth/login
 **Login with credentials**
@@ -77,6 +79,7 @@ Response: 204 No Content
 ---
 
 ## Crew Management
+Create and manage multi-agent crews and their configurations.
 
 ### GET /crews
 **List all crews**
@@ -160,6 +163,7 @@ Response: 204 No Content
 ---
 
 ## Agent Management
+Create and list individual agents with roles, models, and tools.
 
 ### GET /agents
 **List all agents**
@@ -201,6 +205,7 @@ Response: 201 Created
 ---
 
 ## Execution Management
+Start executions, get status, retrieve traces, and stop runs.
 
 ### POST /executions
 **Start crew execution**
@@ -264,6 +269,7 @@ Response: 200 OK
 ---
 
 ## Task Management
+Create and list tasks assigned to agents.
 
 ### GET /tasks
 **List tasks**
@@ -302,6 +308,7 @@ Response: 201 Created
 ---
 
 ## Tool Management
+Discover built-in tools and register custom tools.
 
 ### GET /tools
 **List available tools**
@@ -344,6 +351,7 @@ Response: 201 Created
 ---
 
 ## Memory Management
+Fetch and clear short/long-term memory for a crew.
 
 ### GET /memory/{crew_id}
 **Get crew memory**
@@ -378,40 +386,10 @@ Response: 204 No Content
 
 ---
 
-## Analytics
 
-### GET /analytics/usage
-**Get usage statistics**
-```json
-Response: 200 OK
-{
-  "period": "2024-01",
-  "executions": 1543,
-  "tokens_used": 2450000,
-  "success_rate": 0.98,
-  "avg_duration": 245
-}
-```
 
-### GET /analytics/performance
-**Get performance metrics**
-```json
-Response: 200 OK
-{
-  "crews": [
-    {
-      "crew_id": "crew_abc123",
-      "executions": 500,
-      "success_rate": 0.99,
-      "avg_duration": 180
-    }
-  ]
-}
-```
-
----
-
-## WebSocket Events
+## 🔵 WebSocket Events
+Real-time updates for task lifecycle, errors, and progress.
 
 ### Connection
 ```javascript
@@ -454,7 +432,8 @@ ws.onopen = () => {
 
 ---
 
-## Error Codes
+## 🔷 Error Codes
+Standardized error responses and meanings.
 
 | Code | Message | Description |
 |------|---------|-------------|
@@ -482,7 +461,8 @@ ws.onopen = () => {
 
 ---
 
-## Testing
+## 🔹 Testing
+Sandbox, Postman collection, and OpenAPI spec.
 
 ### Sandbox Environment
 ```bash
@@ -493,55 +473,6 @@ curl -X POST https://sandbox-api.kasal.ai/v1/crews \
   -d @crew.json
 ```
 
-### Postman Collection
-[Download Collection](https://api.kasal.ai/postman/kasal-api-v1.json)
-
-### OpenAPI Spec
-[View OpenAPI 3.0 Spec](https://api.kasal.ai/openapi.json)
-
----
-
-## SDKs & Libraries
-
-### Python
-```python
-pip install kasal-sdk
-
-from kasal import KasalClient
-
-client = KasalClient(api_key="your_key")
-crew = client.crews.create(name="My Crew")
-result = client.execute(crew.id, inputs={})
-```
-
-### JavaScript/TypeScript
-```javascript
-npm install @kasal/sdk
-
-import { Kasal } from '@kasal/sdk';
-
-const kasal = new Kasal({ apiKey: 'your_key' });
-const crew = await kasal.crews.create({ name: 'My Crew' });
-const result = await kasal.execute(crew.id, {});
-```
-
-### Go
-```go
-import "github.com/kasal/kasal-go"
-
-client := kasal.NewClient("your_key")
-crew, _ := client.Crews.Create("My Crew")
-result, _ := client.Execute(crew.ID, inputs)
-```
-
----
-
-## API Support
-
-- **Status Page**: [status.kasal.ai](https://status.kasal.ai)
-- **API Console**: [console.kasal.ai](https://console.kasal.ai)
-- **Developer Forum**: [forum.kasal.ai](https://forum.kasal.ai)
-- **Email**: [email protected]
 
 ---
 

diff --git a/src/docs/ARCHITECTURE_GUIDE.md b/src/docs/ARCHITECTURE_GUIDE.md
@@ -1,11 +1,12 @@
-# Kasal Solution Architecture
+# 🔵 Kasal Solution Architecture
 
 > **Enterprise AI Orchestration Platform** - Scalable, secure, cloud-native
 ![Kasal Overvew](././/images/Kasal-overview.png)
 *Visual workflow designer for creating AI agent collaborations*
 ---
 
 ## System Overview
+What the platform aims to achieve and core design principles.
 
 ### Platform Vision
 **Transform business workflows with autonomous AI agents** - Zero infrastructure complexity
@@ -22,6 +23,7 @@
 ---
 
 ## High-Level Architecture
+A big-picture view of the client, application, AI, and data layers.
 
 ```mermaid
 graph TB
@@ -71,81 +73,87 @@ graph TB
 ---
 
 ## Architecture Pattern
+The layered approach and how requests flow through components.
 
 ### High-level
 - Layered architecture:
   - Frontend (React SPA) → API (FastAPI) → Services → Repositories → Database
 - Async-first (async SQLAlchemy, background tasks, queues)
-- Config via environment (`src/backend/src/config/settings.py`)
-- Pluggable orchestration engine (`src/backend/src/engines/` with CrewAI)
+- Config via environment (src/backend/src/config/settings.py)
+- Pluggable orchestration engine (src/backend/src/engines/ with CrewAI)
 
 ### Request lifecycle (CRUD path)
-1) Router in `api/` receives request, validates using `schemas/`
-2) Router calls `services/` for business logic
-3) Service uses `repositories/` for DB/external I/O
-4) Data persisted via `db/session.py`
+From HTTP request to response: validation, business logic, and persistence.
+1) Router in api/ receives request, validates using schemas/
+2) Router calls services/ for business logic
+3) Service uses repositories/ for DB/external I/O
+4) Data persisted via db/session.py
 5) Response serialized with Pydantic schemas
 
 ### Orchestration lifecycle (AI execution)
-- Entry via `executions_router.py` → `execution_service.py`
-- Service prepares agents/tools/memory and selects engine (`engines/engine_factory.py`)
+How executions are prepared, run, and observed using the engine.
+- Entry via executions_router.py → execution_service.py
+- Service prepares agents/tools/memory and selects engine (engines/engine_factory.py)
 - CrewAI path:
-  - Prep: `engines/crewai/crew_preparation.py` and `flow_preparation.py`
-  - Run: `engines/crewai/execution_runner.py` with callbacks/guardrails
-  - Observability: `execution_logs_service.py`, `execution_trace_service.py`
-- Persist status/history: `execution_repository.py`, `execution_history_repository.py`
+  - Prep: engines/crewai/crew_preparation.py and flow_preparation.py
+  - Run: engines/crewai/execution_runner.py with callbacks/guardrails
+  - Observability: execution_logs_service.py, execution_trace_service.py
+- Persist status/history: execution_repository.py, execution_history_repository.py
 
 ### Background processing
-- Scheduler at startup: `scheduler_service.py`
-- Embedding queue (SQLite): `embedding_queue_service.py` (batches writes)
-- Startup/shutdown cleanup: `execution_cleanup_service.py`
+Schedulers and queues for recurring and long-running tasks.
+- Scheduler at startup: scheduler_service.py
+- Embedding queue (SQLite): embedding_queue_service.py (batches writes)
+- Startup/shutdown cleanup: execution_cleanup_service.py
 
 ### Data modeling
-- ORM in `models/*` mirrors `schemas/*`
+ORM entities, Pydantic schemas, and repository boundaries.
+- ORM in models/* mirrors schemas/*
 - Repositories encapsulate all SQL/external calls (Databricks APIs, Vector Search, MLflow)
-- `db/session.py`:
+- db/session.py:
   - Async engine and session factory
   - SQLite lock retry w/ backoff
-  - Optional SQL logging via `SQL_DEBUG=true`
+  - Optional SQL logging via SQL_DEBUG=true
 
 ### Auth, identity, and tenancy
-- Databricks Apps headers parsed by `utils/user_context.py`
+User context, group isolation, and authorization controls.
+- Databricks Apps headers parsed by utils/user_context.py
 - Group-aware multi-tenant context propagated via middleware
-- JWT/basic auth routes in `auth_router.py`, users in `users_router.py`
-- Authorization checks in `core/permissions.py`
+- JWT/basic auth routes in auth_router.py, users in users_router.py
+- Authorization checks in core/permissions.py
 
 
 ### Security Controls
+Defense-in-depth across network, API, data, secrets, and compliance.
 | Layer | Control | Implementation |
 |-------|---------|----------------|
 | **Network** | TLS 1.3 | End-to-end encryption |
 | **API** | OAuth 2.0 | Databricks SSO |
 | **Data** | AES-256 | Encryption at rest |
-| **Secrets** | Vault | HashiCorp Vault |
-| **Compliance** | SOC2 | Audit trails |
 
 ---
 
 ### Storage Strategy
+Where different data types live and why.
 | Data Type | Storage | Purpose |
 |-----------|---------|---------|
 | **Transactional** | PostgreSQL | ACID compliance |
-| **Session** | Redis | Fast cache |
 | **Vectors** | Databricks Vector | Semantic search |
-| **Files** | S3/Azure Blob | Document storage |
-| **Logs** | CloudWatch/Datadog | Observability |
+| **Logs** | MLFlow traces | Observability |
 
 ---
 
 ### Observability
-- Central log manager: `core/logger.py` (writes to `LOG_DIR`)
-- API/SQL logging toggles (`LOG_LEVEL`, `SQL_DEBUG`)
+Logs, traces, metrics, and how to access them.
+- Central log manager: core/logger.py (writes to LOG_DIR)
+- API/SQL logging toggles (LOG_LEVEL, SQL_DEBUG)
 - Execution logs/traces persisted and queryable via dedicated routes/services
 
 ### Configuration flags (selected)
-- `DOCS_ENABLED`: enables `/api-docs`, `/api-redoc`, `/api-openapi.json`
-- `AUTO_SEED_DATABASE`: async background seeders post DB init
-- `DATABASE_TYPE`: `postgres` (default) or `sqlite` with `SQLITE_DB_PATH`
+Important toggles that affect developer and runtime experience.
+-  DOCS_ENABLED: enables /docs
+- AUTO_SEED_DATABASE: async background seeders post DB init
+- DATABASE_TYPE: sqlite with SQLITE_DB_PATH