Skip to content

Commit dcf1bbe

Browse files
s0xattakdefands0xattakdefand
committed
Description of changes
1 parent e52de0f commit dcf1bbe

2 files changed

Lines changed: 461 additions & 0 deletions

File tree

‎wiki/Architecture.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Architecture
2+
3+
This document provides a detailed overview of the Core Web architecture, including design patterns, components, and data flow.
4+
5+
## 🏗️ High-Level Architecture
6+
7+
Core Web follows a modular, service-oriented architecture designed for scalability and maintainability. The system is composed of several interconnected components:
8+
9+
```
10+
┌─────────────────────────────────────────────────────────────┐
11+
│ Load Balancer/Proxy │
12+
└─────────────────────┬───────────────────────────────────────┘
13+
│
14+
┌─────────────────────┴───────────────────────────────────────┐
15+
│ Core Web Server (8080) │
16+
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
17+
│ │ REST API │ │ GraphQL │ │ gRPC │ │
18+
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
19+
│ ┌─────────────────────────────────────────────────────────┐ │
20+
│ │ Authentication & AuthZ │ │
21+
│ └─────────────────────────────────────────────────────────┘ │
22+
└─────────────────────┬───────────────────────────────────────┘
23+
│
24+
┌─────────────┼─────────────┐ ┌────────────┐
25+
│ │ │ │ │
26+
┌───────▼──┐ ┌──────▼──┐ ┌───────▼──┐ ┌──────▼──┐ ┌─────▼──────┐
27+
│ MySQL │ │ Redis │ │ MongoDB │ │ClickHouse│ │ WASM Host │
28+
│ (Primary)│ │(Caching)│ │(Documents)│ │(Analytics)│ │ (Plugins) │
29+
└──────────┘ └─────────┘ └──────────┘ └──────────┘ └────────────┘
30+
│ │ │ │ │
31+
└─────────────┼─────────────┘ │ │
32+
│ │ │
33+
┌───────▼───────┐ ┌───────▼──────┐ │
34+
│ Job Workers │ │ Events │ │
35+
│ (Background) │ │ (Streaming) │ │
36+
└───────────────┘ └──────────────┘ │
37+
│ │ │
38+
┌───────▼───────┐ ┌───────▼──────┐ │
39+
│ Outbox Pattern│ │ Redis Streams│ │
40+
│ Publisher │ │ Consumer │ │
41+
└───────────────┘ └──────────────┘ │
42+
│
43+
┌───────▼───────┐
44+
│ Admin Dashboard│
45+
│ (Yew/WASM) │
46+
└───────────────┘
47+
```
48+
49+
## 🧩 Core Components
50+
51+
### 1. Main Web Server
52+
53+
The main web server is the entry point for all client requests. It's built with:
54+
55+
- **Axum/Tower/Hyper** - For HTTP handling
56+
- **Multiple Protocol Support** - REST, GraphQL, gRPC, WebSocket
57+
- **Authentication & Authorization** - JWT, OIDC, RBAC/ABAC
58+
- **Observability** - Tracing, metrics, logging
59+
60+
### 2. Data Layer
61+
62+
Core Web supports multiple databases for different use cases:
63+
64+
- **MySQL** - Primary transactional database with SQLx
65+
- **Redis** - Caching, session storage, pub/sub messaging
66+
- **MongoDB** - Document storage for flexible schemas
67+
- **ClickHouse** - Analytics and time-series data warehouse
68+
69+
### 3. Background Processing
70+
71+
The job worker system handles background tasks:
72+
73+
- **Job Workers** - Dedicated background job processing
74+
- **Scheduled Tasks** - Cron-like recurring task execution
75+
- **Event Streaming** - Redis Streams with durable consumer groups
76+
- **Outbox Pattern** - Reliable event publishing from MySQL to Redis and ClickHouse
77+
78+
### 4. WebAssembly Integration
79+
80+
Core Web includes a WASM plugin system:
81+
82+
- **WASM Plugins Host** - Wasmtime-based plugin system
83+
- **Admin Dashboard** - Yew-based WASM administration interface
84+
- **Feature Flags UI** - Toggle runtime features through web interface
85+
86+
## 🔁 Data Flow Patterns
87+
88+
### Request Processing Flow
89+
90+
1. **Incoming Request** - Client request arrives at the web server
91+
2. **Authentication** - Request is authenticated using JWT/OIDC
92+
3. **Authorization** - Access control checks are performed
93+
4. **Business Logic** - Request is processed by the appropriate handler
94+
5. **Data Access** - Data is retrieved/modified in the database
95+
6. **Response** - Response is formatted and sent back to the client
96+
97+
### Outbox Pattern
98+
99+
For reliable event publishing:
100+
101+
1. **Transaction** - Business operation and outbox record are saved in the same transaction
102+
2. **Polling** - Job worker polls the outbox table for new records
103+
3. **Publishing** - Events are published to Redis and ClickHouse
104+
4. **Cleanup** - Processed outbox records are cleaned up
105+
106+
### Read-Through Caching
107+
108+
For improved read performance:
109+
110+
1. **Cache Check** - Check if data exists in Redis cache
111+
2. **Cache Hit** - Return cached data if available
112+
3. **Cache Miss** - Retrieve data from MySQL/MongoDB
113+
4. **Cache Update** - Store data in Redis with expiration
114+
115+
## 🛡️ Security Architecture
116+
117+
Core Web implements multiple layers of security:
118+
119+
### Authentication
120+
- **JWT Tokens** - Secure token-based authentication
121+
- **OIDC Integration** - Single sign-on support
122+
- **API Keys** - Service-to-service authentication
123+
124+
### Authorization
125+
- **RBAC** - Role-based access control
126+
- **ABAC** - Attribute-based access control with Casbin/Cedar
127+
- **Scope-based Access** - Fine-grained permission control
128+
129+
### Security Headers
130+
- **HSTS** - HTTP Strict Transport Security
131+
- **CSP** - Content Security Policy
132+
- **CORS** - Cross-Origin Resource Sharing protection
133+
- **CSRF** - Cross-Site Request Forgery protection
134+
135+
## 📈 Observability Architecture
136+
137+
Comprehensive observability is built into Core Web:
138+
139+
### Tracing
140+
- **Distributed Tracing** with OpenTelemetry
141+
- **Span Context Propagation** across services
142+
- **Trace Sampling** for performance optimization
143+
144+
### Metrics
145+
- **Application Metrics** - Custom business metrics
146+
- **System Metrics** - Resource utilization metrics
147+
- **Prometheus Integration** - Metrics collection and querying
148+
149+
### Logging
150+
- **Structured Logging** - JSON-formatted logs
151+
- **Log Levels** - Configurable logging levels
152+
- **Log Aggregation** - Centralized log collection
153+
154+
## 🔄 Resilience Patterns
155+
156+
Core Web implements several resilience patterns:
157+
158+
### Retry Mechanisms
159+
- **Exponential Backoff** - Increasing delays between retries
160+
- **Jitter** - Randomization to prevent thundering herds
161+
- **Circuit Breaker** - Fail-fast mechanism for failing services
162+
163+
### Resource Management
164+
- **Bulkhead Pattern** - Resource isolation
165+
- **Timeout Handling** - Prevent hanging requests
166+
- **Rate Limiting** - Token bucket algorithm with Redis backend
167+
168+
## 📦 Deployment Architecture
169+
170+
Core Web is designed for modern deployment environments:
171+
172+
### Containerization
173+
- **Multi-stage Docker Builds** - Optimized container images
174+
- **Distroless Runtime** - Minimal attack surface
175+
- **Health Checks** - Kubernetes-ready endpoints
176+
177+
### Orchestration
178+
- **Kubernetes Support** - Deployment manifests included
179+
- **Horizontal Scaling** - Stateless service design
180+
- **Service Discovery** - Dynamic service registration
181+
182+
## 🧪 Testing Architecture
183+
184+
Core Web includes a comprehensive testing strategy:
185+
186+
### Unit Testing
187+
- **Component-level Tests** - Individual function testing
188+
- **Mock Dependencies** - Isolated test environments
189+
- **Property-based Testing** - Proptest for complex scenarios
190+
191+
### Integration Testing
192+
- **Database Integration** - Testcontainers for real database testing
193+
- **Service Integration** - End-to-end service testing
194+
- **Contract Testing** - API contract verification
195+
196+
### Performance Testing
197+
- **Load Testing** - High-concurrency scenario testing
198+
- **Benchmarking** - Performance regression detection
199+
- **Profiling** - CPU and memory usage analysis
200+
201+
## 📚 Next Steps
202+
203+
To learn more about specific aspects of the architecture:
204+
205+
1. Review the [Crate Stack](Crate-Stack.md) documentation
206+
2. Explore the [API Reference](API-Reference.md)
207+
3. Check out the [Deployment](Deployment.md) guides
208+
4. Learn about [Security](Security.md) implementation details

0 commit comments

Comments
 (0)