Skip to content

README.md

Latest commit

 

History

History
374 lines (285 loc) · 21.3 KB

README.md

File metadata and controls

374 lines (285 loc) · 21.3 KB
Raw

XafDynamicAssemblies

Architecture Overview

AI-powered dynamic entity system for DevExpress XAF. Create new business object types, properties, and relationships at runtime — no recompilation, no redeployment. Uses Roslyn for in-process C# compilation and AssemblyLoadContext for hot-loading.

This is not EAV. The system generates real CLR types backed by real SQL columns and foreign key constraints. The result is full XAF framework support — list views, detail views, validation, reporting — for entities that never existed at compile time.

What It Does

Define a new entity through the UI:

  1. Create a class — give it a name, navigation group, and description
  2. Add fields — string, int, decimal, bool, DateTime, Guid, or references to other entities
  3. Click Deploy — Roslyn compiles the class, DDL creates the table, the server restarts, and your new entity appears in the navigation with full CRUD views

The entire cycle takes seconds. No developer intervention required.

Key Features

  • Runtime entity creation via metadata-driven Roslyn compilation
  • Hot-load deploy with automatic process restart and SignalR client reconnection
  • Entity relationships — runtime entities can reference other runtime entities or compiled entities, with real SQL foreign keys
  • Graduation path — promote runtime entities to compiled C# source code for inclusion in the main codebase
  • Degraded mode — if compilation fails at startup, compiled entities still work normally
  • Web API (OData) — expose runtime entities as REST endpoints with full CRUD and OData query support
  • AI Schema Assistant — conversational AI for entity CRUD and metadata actions via natural language (LLMTornado + Claude Sonnet)
  • Schema export/import — download schema definitions as JSON files, upload to restore or migrate between environments
  • Error recovery — fix bad metadata, redeploy, and the system recovers without manual intervention
  • Full validation — class names, field names, type names, and reserved words are validated before save
  • Metadata-driven actions — add buttons to a DetailView without writing code: SetField/ShowMessage/OpenView steps defined as metadata, live the next time the view opens (no deploy, no restart, no compilation); up to 10 actions per entity
  • 168-test regression suite — Playwright E2E across 12 phases plus unit tests, all passing (.NET/Playwright/xUnit)

Tech Stack

Layer Technology
Runtime .NET 10, C# 14
Framework DevExpress XAF 26.1, EF Core 10
Compilation Roslyn (Microsoft.CodeAnalysis.CSharp 5.0)
Database PostgreSQL 17 (via Npgsql)
UI Blazor Server
Real-time SignalR for schema change notifications
Web API DevExpress XAF Web API (OData v4), Swashbuckle (Swagger)
Testing Playwright (.NET) + xUnit, 156 E2E tests across 12 phases (+ 7 mock-server self-tests + 10 step-value-converter unit tests)
Infrastructure Docker Compose (PostgreSQL)

Prerequisites

Quick Start

# 1. Start PostgreSQL
docker compose up -d

# 2. Build the solution
dotnet build XafDynamicAssemblies.slnx

# 3. Initialize the database
dotnet run --project XafDynamicAssemblies/XafDynamicAssemblies.Blazor.Server \
  -- --updateDatabase --forceUpdate --silent

# 4. Run the server with auto-restart support
./run-server.bat          # Windows
./run-server.sh           # Linux/macOS

Open https://localhost:5001 in your browser.

Process Restart

Deploying schema changes requires a full process restart — XAF's TypesInfo and application model are process-static singletons that cannot be reset. The app exits with code 42 after each deploy, and something external must relaunch it:

Hosting Restart mechanism
Local dev run-server.bat / run-server.sh — loops on exit code 42
IIS ANCM out-of-process hosting — auto-restarts Kestrel on any exit
Docker / systemd restart: unless-stopped or Restart=on-failure

Do not use dotnet run directly — it will exit after deploy and not come back.

IIS Deployment

The included web.config uses out-of-process hosting. ANCM runs Kestrel as a child process and automatically restarts it when the app exits — this handles exit code 42 without a wrapper script.

Key settings:

  • hostingModel="OutOfProcess" — required. In-process hosting runs inside w3wp.exe, and Environment.Exit(42) would crash the worker process instead of triggering a clean restart.
  • startupTimeLimit="120" — allows time for Roslyn compilation on cold start (default 30s is too short for large schemas)

To deploy:

  1. dotnet publish -c Release the Blazor.Server project
  2. Create an IIS site pointing to the publish folder
  3. Ensure the app pool uses No Managed Code (.NET CLR version)
  4. The web.config in the publish output handles the rest

For internals on the restart mechanism, see UNDER_THE_HOOD.md — Hot-Load and Process Restart.

Usage

Creating a Runtime Entity

  1. Navigate to Schema Management > Custom Class
  2. Click New, enter a class name (e.g. Invoice), navigation group (e.g. Billing)
  3. Save, then navigate to Schema Management > Custom Field
  4. Add fields — each needs a name, type, and parent class
  5. Return to Custom Class and click Deploy Schema
  6. The server restarts — your entity appears in the nav with full CRUD

Supported Field Types

Type C# Type PostgreSQL
String string text
Integer int? integer
Long long? bigint
Decimal decimal? numeric(18,6)
Double double? double precision
Float float? real
Boolean bool? boolean
DateTime DateTime? timestamp
Guid Guid? uuid
Reference Navigation + FK uuid (FK constraint)

Entity Relationships

Runtime entities can reference:

  • Other runtime entities — compiled in the same Roslyn assembly
  • Compiled entities — e.g., a runtime EmployeeInfo referencing the compiled Company entity

All references create real SQL foreign key constraints.

Exposing via Web API (OData)

Runtime entities can be exposed as OData REST endpoints:

  1. Open a Custom Class in detail view
  2. Check Is Api Exposed
  3. Save and click Deploy Schema
  4. After restart, full CRUD endpoints are live at /api/odata/{ClassName}

What you get:

  • GET /api/odata/{ClassName} — list with OData query support ($filter, $select, $orderby, $top, $skip, $count)
  • GET /api/odata/{ClassName}({key}) — single record by ID
  • POST /api/odata/{ClassName} — create
  • PATCH /api/odata/{ClassName}({key}) — update
  • DELETE /api/odata/{ClassName}({key}) — delete
  • Swagger UI at /swagger (development mode)

Metadata entities (CustomClass, CustomField) are always exposed. Runtime entities are opt-in via the IsApiExposed flag.

AI Schema Assistant

Talk to the system in plain English to create, modify, or delete runtime entities:

  1. Navigate to Schema Management > AI Chat
  2. Type a natural language request, e.g. "Create an Invoice entity with fields Amount (decimal), DueDate (datetime), and IsPaid (boolean)"
  3. The AI creates the metadata — review and deploy as usual

The AI assistant uses LLMTornado with Claude Sonnet (configurable) and has access to 14 schema management tools (list/describe/create/modify/delete entities, validate, pending changes, roles, and metadata actions: list_actions/create_action/delete_action/set_action_active). It maintains conversation context for multi-turn workflows and asks clarifying questions for ambiguous requests. Configuration is in appsettings.json under the AI section.

The assistant can also manage metadata actions (the codeless DetailView buttons described below) — and unlike entity changes, these are live without a deploy or restart:

  • "Add an 'Approve' button to Invoice that sets Status to Approved and shows a confirmation message" → create_action; the button appears the next time an Invoice DetailView opens
  • "What actions are defined?" → list_actions (caption, target, steps, criteria, active state)
  • "Disable the Approve action on Invoice" / "…enable it again" → set_action_active
  • "Delete the Approve action on Invoice" → delete_action

The assistant validates the same rules as the Custom Action editor (required per-step fields, unique caption per entity, at most one OpenView step) and warns — without blocking — on unparseable criteria, a target entity that doesn't exist yet, or the 10-actions-per-entity render ceiling.

Metadata Actions (Codeless DetailView Buttons)

Add a button to any entity's DetailView without writing a controller — pure metadata, live the next time the view opens. No deploy, no restart, no compilation.

  1. Navigate to Schema Management > Custom Action
  2. Create an action: Caption (button text), Target Entity (simple class name, e.g. Invoice), optional Criteria (XAF criteria string — button is disabled when the current object doesn't match), optional Confirmation Message
  3. Add Steps, executed in SortOrder:
Step kind What it does Fields
SetField Sets a property on the current object (value converted to the member's type) FieldName, Value
ShowMessage Displays a toast (Info/Success/Warning/Error) MessageText, MessageType
OpenView Opens another entity's ListView after the steps complete TargetEntityName
  1. Save — the button appears the next time a DetailView of the target entity opens

If at least one SetField ran, changes are committed in a single save. Validation runs on save: required fields per step kind, unique (TargetEntity, Caption), at most one OpenView per action. Works on both runtime and compiled entities; limit is 10 actions per entity type.

Under the hood: XAF Blazor only renders actions declared in a controller's constructor, so MetadataActionDispatcherController maintains a fixed pool of 10 slot actions and assigns metadata to them on each view activation.

Prefer chat? The AI Schema Assistant can create, list, toggle, and delete these actions through natural language — see the example prompts above.

Schema Export / Import

From the Custom Class list view, use the toolbar actions:

  • Export Schema — downloads all runtime entity definitions as a .json file (browser file download)
  • Import Schema — opens a file picker to upload a .json schema file, then creates or updates entities to match

Export/import history is tracked in Schema Management > Schema History with timestamps, user, and the full JSON payload.

Graduating to Compiled Code

When a runtime entity is stable:

  1. Open it in Custom Class detail view
  2. Click Graduate — a confirmation dialog explains what will happen
  3. The system generates production C# source, a DbContext snippet, and a migration note
  4. Copy the code into your project and deploy
  5. The graduated entity takes over the existing SQL table — zero data migration

Partial class option: If GenerateAsPartial is checked on a CustomClass, Graduate generates public partial class Foo : BaseObject without class-level attributes ([DefaultClassOptions], [NavigationItem], [DefaultProperty]), so you can provide those on a hand-written partial in your project.

Visual warnings: Graduated entities appear gray and italic in the ListView; Graduating entities appear in orange and italic. A warning banner is shown on the ListView when non-Runtime entities exist, and the DetailView displays a warning message when viewing graduated entities.

Solution Structure

XafDynamicAssemblies/
├── XafDynamicAssemblies.Module/          # Shared module — all business logic
│   ├── BusinessObjects/                  # EF Core entities + DbContext
│   │   ├── CustomClass.cs               # Runtime entity metadata
│   │   ├── CustomField.cs               # Runtime field definitions
│   │   ├── CustomAction.cs              # Metadata action definitions
│   │   ├── CustomActionStep.cs          # Action steps (SetField/ShowMessage/OpenView)
│   │   └── XafDynamicAssembliesDbContext.cs
│   ├── Services/                         # Core engine
│   │   ├── RuntimeAssemblyBuilder.cs     # Roslyn C# generation + compilation
│   │   ├── AssemblyGenerationManager.cs  # ALC lifecycle management
│   │   ├── SchemaSynchronizer.cs         # DDL via Npgsql
│   │   ├── SchemaChangeOrchestrator.cs   # Hot-load orchestration
│   │   ├── DynamicModelCacheKeyFactory.cs# EF Core model invalidation
│   │   ├── GraduationService.cs          # Source code export
│   │   ├── SchemaExportImportService.cs  # JSON schema export/import
│   │   ├── SupportedTypes.cs             # Type mapping
│   │   ├── AIChatService.cs              # LLMTornado integration + tool loop
│   │   ├── SchemaAIToolsProvider.cs      # 14 AI tools for schema CRUD and metadata actions
│   │   ├── SchemaDiscoveryService.cs     # ITypesInfo reflection for AI prompt
│   │   └── StepValueConverter.cs         # SetField literal → member type conversion
│   ├── Controllers/                      # XAF actions
│   │   ├── SchemaChangeController.cs     # Deploy Schema
│   │   ├── GraduateController.cs         # Graduate
│   │   ├── GraduationWarningController.cs # Visual warnings for graduated/graduating entities
│   │   ├── SchemaExportImportController.cs # Export/Import Schema (file download/upload)
│   │   ├── TestCompileController.cs      # Test Compile All (ListView action)
│   │   └── MetadataActionDispatcherController.cs # Live CustomAction slot-pool dispatcher (DetailView)
│   ├── Validation/                       # Name validation rules
│   └── Module.cs                         # Bootstrap, metadata query
│
├── XafDynamicAssemblies.Blazor.Server/   # Blazor Server host
│   ├── Program.cs                        # Exit-code-42 restart mechanism
│   ├── Startup.cs                        # DI, XAF, SignalR wiring
│   ├── Services/RestartService.cs        # Restart request tracking
│   └── Hubs/SchemaUpdateHub.cs           # Client notifications
│
├── XafDynamicAssemblies.Tests/           # Playwright E2E tests (.NET / xUnit)
│   ├── Fixtures/                         # Browser + mock-LLM fixtures
│   ├── Pages/                            # Page object models
│   │   ├── NavigationPage.cs             # XAF accordion nav
│   │   ├── ListViewPage.cs               # Grid interactions
│   │   └── DetailViewPage.cs             # Form interactions
│   ├── MockLlm/                          # In-process mock LLM server (port 5555)
│   └── Tests/                            # 12 phases, 156 E2E tests + 7 mock-server self-tests + 10 converter unit tests
│       ├── Phase01_MetadataCrudTests.cs
│       ├── Phase02_RuntimeEntityTests.cs
│       ├── Phase03_ValidationTests.cs
│       ├── Phase04_HotLoadTests.cs
│       ├── Phase05_RelationshipTests.cs
│       ├── Phase06_GraduationTests.cs
│       ├── Phase07_ErrorHandlingTests.cs
│       ├── Phase08_PerformanceTests.cs
│       ├── Phase09_ReviewFixesTests.cs
│       ├── Phase10_WebApiTests.cs
│       ├── Phase11_AIChatMockedTests.cs
│       ├── Phase11_AIChatLiveTests.cs
│       ├── Phase12_ActionBuilderTests.cs
│       ├── SchemaSyncCaseSensitivityTests.cs
│       └── StepValueConverterTests.cs
│
├── docker-compose.yml                    # PostgreSQL 17
├── run-server.bat / run-server.sh        # Windows / Linux restart wrapper
└── run-server-mock.bat                   # Windows restart wrapper + mock LLM routing

Running Tests

The server must be running via run-server.bat / run-server.sh (not dotnet run directly) because tests trigger deploy+restart cycles. Phase 11 mocked AI-chat tests additionally require the server started via run-server-mock.bat — it routes the app's LLM calls to the in-process mock server on port 5555; without it, Phase 11 fails with generic chat errors rather than a clear "wrong server" message.

# Install Playwright browsers (first time only)
pwsh XafDynamicAssemblies/XafDynamicAssemblies.Tests/bin/Debug/net8.0/playwright.ps1 install chromium

# Full regression — start the server via run-server-mock.bat first, then:
dotnet test XafDynamicAssemblies/XafDynamicAssemblies.Tests --filter "Category!=LiveAI" -v normal

# Single phase
dotnet test XafDynamicAssemblies/XafDynamicAssemblies.Tests --filter "FullyQualifiedName~Phase04" -v normal

Live AI tests are opt-in — they call a real LLM provider and are excluded from the filters above. Set AI_TEST_API_KEY and run with --filter "Category=LiveAI":

AI_TEST_API_KEY=sk-... dotnet test XafDynamicAssemblies/XafDynamicAssemblies.Tests --filter "Category=LiveAI"

Test Phases

Phase Tests What It Covers
1 — Metadata CRUD 11 Create, read, update, delete CustomClass and CustomField
2 — Runtime Entities 13 Roslyn compilation, entity setup, full CRUD on runtime types
3 — Validation 9 Invalid names, reserved words, type dropdown, Test Compile All (ListView action)
4 — Hot-Load 7 Deploy action, navigation updates, field addition, data survival across restarts
5 — Relationships 8 Entity references, FK constraints, cross-entity navigation
6 — Graduation 9 Source generation, status transition, data preservation post-graduation
7 — Error Handling 7 Degraded mode, compilation failure recovery, empty metadata, restart resilience
8 — Performance 4 Bulk 10-class compilation, concurrent page access
9 — Review Fixes 19 Cross-references, required refs, field attributes, graduation escaping
10 — Web API 36 Swagger, OData CRUD, query features, IsApiExposed toggle, API↔UI consistency
11 — AI Chat (Mocked) 18 Chat panel, prompt suggestions, entity/field proposals, roles, multi-turn, chat-created metadata actions (create/toggle/delete via chat with DB-effect asserts, live button render) (requires run-server-mock.bat)
11 — AI Chat (Live) 5 Live AI entity creation, modification, ambiguity resolution, multi-turn (opt-in, requires AI_TEST_API_KEY)
12 — Action Builder 9 CustomAction/CustomActionStep UI, live activation without restart, SetField/ShowMessage/OpenView execution, criteria-based enablement, validation
— Schema Sync Case Sensitivity 1 Regression: column-existence check is case-sensitive (differently-cased field rename creates a new column)
— Mock LLM Server 7 Self-tests for the in-process mock LLM server/script matcher (incl. action-verb tool_use wire shapes)
— Step Value Converter 10 Unit tests converting CustomActionStep literals to member types (SetField)

Test Environment

Variable Default Description
BASE_URL https://localhost:5001 App URL
HEADLESS true Headless browser mode
SLOW_MO 0 Slow down for debugging (ms)
MOCK_LLM_PORT 5555 Port the mock LLM server listens on (must match run-server-mock.bat)
AI_TEST_API_KEY (none) API key for live AI tests (Phase 11 Live, Category=LiveAI); tests skipped if unset

Database

PostgreSQL 17 runs via Docker on a non-standard port:

Setting Value
Host localhost
Port 5434
Database XafDynamicAssemblies
Username xafdynamic
Password xafdynamic 
# Start the database
docker compose up -d postgres

# Manual schema update
dotnet run --project XafDynamicAssemblies/XafDynamicAssemblies.Blazor.Server \
  -- --updateDatabase --forceUpdate --silent

Architecture Deep Dive

For internals — the Roslyn compilation pipeline, hot-load sequence, type identity management, process restart mechanism, and graduation workflow — see UNDER_THE_HOOD.md.

Known Limitations

  • Non-collectible ALC — runtime types persist in memory. Hot-load works via process restart, not in-process unload.
  • No inverse navigation — compiled entities cannot have navigation properties pointing back to runtime entities.
  • Process restart required — XAF's TypesInfo and SharedApplicationModelManagerContainer are process-static and cannot be reset in-process after recompilation.
  • DevExpress license required — XAF is commercial software.

License

This project uses DevExpress XAF, which requires a commercial license. The project code is provided as-is for educational and reference purposes.