Skip to content

Commit f4f4a43

Browse files
chaenchaen
authored
Merge pull request #165 from chrisburr/doc-chart-structure-comprehensive
docs: Add comprehensive chart structure documentation
2 parents 1616993 + 0756d26 commit f4f4a43

File tree

3 files changed

+458
-1
lines changed

3 files changed

+458
-1
lines changed

‎docs/admin/explanations/chart-structure.md‎

Lines changed: 233 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,233 @@
1-
* The Chart structure: modes of the charts operations (how things bootstrap, secret generations, etc)
1+
# Chart Structure
2+
3+
The DiracX Helm chart is designed to deploy a complete DiracX environment with all necessary dependencies and services. This document explains the structure, components, and initialization process of the chart.
4+
5+
## Overview
6+
7+
The DiracX chart is an umbrella chart that orchestrates the deployment of:
8+
9+
- **DiracX services**: The core DiracX API and CLI components
10+
- **DiracX Web**: The web frontend application
11+
- **External dependencies**: Databases, message queues, search engines, and observability tools
12+
- **Authentication services**: Identity providers and certificate management
13+
- **Initialization jobs**: Bootstrap processes for databases, secrets, and configuration
14+
15+
The structure is heavily inspired by the one provided for GitLab due to their similar use case of providing a generic helm chart for a wide variety of installations.
16+
Detailed documentation with justification for technical decisions can be found [here](https://docs.gitlab.com/charts/).
17+
18+
## Chart Dependencies
19+
20+
The chart includes the following external dependencies, all conditionally enabled:
21+
22+
!!! warning "Production Deployments"
23+
The bundled dependencies listed below are primarily intended for development and testing environments. For production deployments, you should use externally managed services. See the [installation guide](../how-to/installing.md) for production deployment recommendations.
24+
25+
### Core Infrastructure
26+
- **MySQL** (`mysql`): Primary database for DiracX data storage
27+
- **RabbitMQ** (`rabbitmq`): Message queue for asynchronous task processing
28+
- **OpenSearch/Elasticsearch**: Search and indexing capabilities
29+
- `opensearch` (preferred): Open source search engine
30+
- `elasticsearch`: Alternative search engine option
31+
32+
### Storage
33+
- **MinIO** (`minio`): S3-compatible object storage for files and artifacts
34+
35+
### Authentication & Security
36+
- **Dex** (`dex`): OpenID Connect identity provider for authentication
37+
- **cert-manager** (`cert-manager`): Automated TLS certificate management
38+
- **cert-manager-issuer**: Custom issuer configuration for cert-manager
39+
40+
### Observability
41+
- **OpenTelemetry Collector** (`opentelemetry-collector`): Telemetry data collection and processing
42+
- **Jaeger** (`jaeger`): Distributed tracing system
43+
- **Grafana** (`grafana`): Metrics visualization and dashboards
44+
- **Prometheus** (`prometheus`): Metrics collection and alerting
45+
46+
## Main Components
47+
48+
### DiracX Services
49+
The core DiracX deployment consists of:
50+
51+
- **Main Deployment** (`diracx/deployment.yaml`): API server pods running DiracX services
52+
- **CLI Deployment** (`diracx/deployment-cli.yaml`): Command-line interface pods for administrative tasks
53+
- **Service** (`diracx/service.yaml`): Kubernetes service exposing DiracX APIs
54+
- **ServiceAccount** (`diracx/serviceaccount.yaml`): Service account with necessary permissions
55+
56+
### DiracX Web
57+
The web frontend deployment includes:
58+
59+
- **Deployment** (`diracx-web/deployment.yaml`): Web application pods serving the DiracX UI
60+
- **Service** (`diracx-web/service.yaml`): Service exposing the web interface
61+
62+
### Configuration and Secrets
63+
- **Secrets** (`diracx/secrets.yaml`): Kubernetes secrets for sensitive configuration
64+
- **Environment Config** (`envconfig.yaml`): Environment-specific configuration
65+
- **Ingress** (`ingress.yaml`): HTTP/HTTPS routing configuration
66+
67+
## Initialization Process
68+
69+
The chart uses a series of initialization jobs to bootstrap the DiracX environment. These jobs run as Kubernetes Jobs before the main services start:
70+
71+
### 1. Secret Initialization (`init-secrets/`)
72+
- **Purpose**: Generates and manages cryptographic secrets and keys
73+
- **Components**:
74+
- ConfigMap with initialization script
75+
- Job that creates necessary secrets
76+
- RBAC configuration for secret management
77+
- **When it runs**: First, before all other initialization jobs
78+
79+
### 2. SQL Database Initialization (`init-sql/`)
80+
- **Purpose**: Sets up database schema and initial data
81+
- **Components**:
82+
- ConfigMap with SQL initialization scripts
83+
- Job that connects to MySQL and creates/updates schemas
84+
- **When it runs**: After secret initialization, before application startup. See [the explanation on how databases are handled](database-management.md) for more details.
85+
86+
### 3. OpenSearch Initialization (`init-os/`)
87+
- **Purpose**: Configures OpenSearch indices and mappings
88+
- **Components**:
89+
- ConfigMap with OpenSearch setup scripts
90+
- Job that creates necessary indices
91+
- **When it runs**: After secret initialization, parallel to SQL initialization
92+
93+
### 4. Configuration Store Initialization (`init-cs/`)
94+
- **Purpose**: Initializes the DiracX Configuration Store with default settings
95+
- **Components**:
96+
- ConfigMap with CS initialization scripts
97+
- Job that populates initial configuration
98+
- **When it runs**: After database and OpenSearch are ready
99+
100+
### 5. Keystore Initialization (`init-keystore/`)
101+
- **Purpose**: Sets up cryptographic keystores and certificates for signing JWTs
102+
- **Components**:
103+
- ConfigMap with keystore management scripts
104+
- Job that generates and loads certificates
105+
- **When it runs**: After secret initialization, before services start
106+
107+
## Secret Management
108+
109+
DiracX uses multiple types of secrets for secure operation.
110+
The chart automatically generates and manages these secrets through the initialization process.
111+
To later update the values of these secrets see the [rotate a secret documentation](../how-to/rotate-a-secret.md).
112+
113+
### Types of Secrets
114+
115+
#### Application Secrets (`diracx-secrets`)
116+
- **DiracX configuration**: Contains all DiracX service settings and configuration values
117+
- **Generated from**: `values.yaml` configuration under `diracx.settings`
118+
- **Usage**: Mounted as environment variables in DiracX service pods
119+
120+
#### Dynamic Secrets (`diracx-dynamic-secrets`)
121+
- **DIRACX_SERVICE_AUTH_STATE_KEY**: 32-byte base64-encoded key for OAuth state management
122+
- **Generated**: Automatically during secret initialization
123+
- **Usage**: Secure token exchange in authentication flows
124+
125+
#### Database Connection Secrets
126+
- **diracx-sql-connection-urls**: Database connection strings for DiracX services
127+
- **diracx-sql-root-connection-urls**: Administrative database connection strings for applying schema changes
128+
- **Format**: MySQL connection URLs with embedded credentials
129+
- **Generated for each database**: AuthDB, JobDB, JobLoggingDB, PilotAgentsDB, SandBoxMetadataDB, TaskQueueDB
130+
131+
#### Search Engine Connection Secrets
132+
- **diracx-os-connection-urls**: OpenSearch/Elasticsearch connection configurations
133+
- **diracx-os-root-connection-urls**: Administrative search engine connections
134+
- **Format**: JSON configuration with hosts, authentication, and SSL settings
135+
- **Generated for each index**: JobParametersDB
136+
137+
#### External Service Secrets
138+
When using bundled dependencies, the chart generates:
139+
- **mysql-secret**: MySQL database passwords (root, replication, application user)
140+
- **rabbitmq-secret**: RabbitMQ credentials (password, Erlang cookie)
141+
142+
#### Certificate and Keystore Secrets
143+
- **Keystore secrets**: Generated by `init-keystore` job for JWT signing
144+
- **TLS certificates**: Managed by cert-manager for HTTPS endpoints
145+
- **CA certificates**: For internal service communication
146+
147+
### Secret Generation Process
148+
149+
1. **Automatic Generation**: Secrets are created if they don't exist during deployment
150+
2. **Preservation**: Existing secrets are preserved during upgrades
151+
3. **Validation**: Missing keys in existing secrets are automatically added
152+
4. **Encoding**: Sensitive values are base64-encoded when stored in Kubernetes
153+
154+
### Secret Access Patterns
155+
156+
- **Environment Variables**: Configuration secrets are injected as environment variables
157+
- **Volume Mounts**: Certificate secrets are mounted as files in containers
158+
159+
## Storage Volumes
160+
161+
The chart creates several persistent volumes:
162+
163+
- **CS Store Volume** (`diracx/cs-store-volume.yml`): Persistent storage for Configuration Store data (development only)
164+
- **DiracX Code Volume** (`diracx-code-volume.yml`): Volume for DiracX application code (used in development mode)
165+
166+
## Configuration Structure
167+
168+
To see the complete set of available values, refer to [the values reference](../reference/values.md).
169+
170+
### Global Configuration
171+
172+
The `global` section in `values.yaml` contains shared settings:
173+
174+
```yaml
175+
global:
176+
batchJobTTL: 600 # Job retention time
177+
imagePullPolicy: Always # Container image pull policy
178+
storageClassName: standard # Kubernetes storage class
179+
activeDeadlineSeconds: 900 # Job timeout
180+
images: # Container image specifications
181+
tag: "dev"
182+
services: ghcr.io/diracgrid/diracx/services
183+
client: ghcr.io/diracgrid/diracx/client
184+
web:
185+
tag: "dev"
186+
repository: ghcr.io/diracgrid/diracx-web/static
187+
```
188+
189+
### Component Configuration
190+
Each component can be configured independently:
191+
192+
- **Replica counts**: Separate settings for DiracX services and web frontend
193+
- **Initialization jobs**: Enable/disable specific initialization steps
194+
- **Developer mode**: Special configuration for development environments
195+
- **Dependencies**: Each external dependency can be enabled/disabled and configured
196+
197+
## Development vs Production Modes
198+
199+
### Development Mode
200+
201+
When `developer.enabled: true`:
202+
203+
- Mounts local source code for live development
204+
- Configures special URLs for local testing
205+
- Uses development image tags
206+
207+
To find more about running in development mode see [here](../../dev/tutorials/run-locally.md).
208+
209+
### Production Mode
210+
211+
For production deployments:
212+
213+
- Uses stable image tags
214+
- Enables all security features
215+
- Configures proper resource limits and requests
216+
- Sets up monitoring and alerting
217+
218+
To find more about running in production mode see [here](../how-to/installing.md).
219+
220+
## Bootstrap Sequence
221+
222+
The complete bootstrap sequence follows this order:
223+
224+
1. **Helm Dependencies**: External charts are deployed (MySQL, RabbitMQ, etc.)
225+
2. **Secret Generation**: `init-secrets` job creates cryptographic materials
226+
3. **Database Setup**: `init-sql` job initializes database schemas
227+
4. **Search Setup**: `init-os` job configures OpenSearch indices
228+
5. **Configuration**: `init-cs` job populates Configuration Store
229+
6. **Certificate Setup**: `init-keystore` job manages certificates
230+
7. **Service Startup**: DiracX services and web frontend start
231+
8. **Health Checks**: Services perform readiness and liveness checks
232+
233+
This staged approach ensures that all dependencies are ready before DiracX services attempt to start, providing a reliable deployment process.

‎docs/admin/explanations/database-management.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Database management
2+
3+
* DIRAC handles most DBs currently, exception AuthDB
4+
* pre-install batch job creates the missing bits of schema
5+
* eventually, alambic

0 commit comments

Comments
 (0)