Skip to content

Commit 34379f1

Browse files
author
Vadym Mudryi
committed
updated readme
1 parent 960a2e1 commit 34379f1

File tree

6 files changed

+198
-131
lines changed

6 files changed

+198
-131
lines changed

‎README.md‎

Lines changed: 152 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@
33

44
Please note that not all features from the Docker Swarm solution are supported yet.
55

6+
**Implemented features**
7+
8+
- Data migration
9+
- Data seeding
10+
611
**Limitations:**
712
- Manual Helm installation and upgrade only
813
- Manual initial user configuration for MinIO, MongoDB, Elasticsearch
@@ -15,63 +20,12 @@ Please note that not all features from the Docker Swarm solution are supported y
1520

1621
This repository is used to store infrastructure code for deploying OpenCRVS.
1722

18-
---
19-
20-
# Running OpenCRVS on Kubernetes
21-
22-
## Prerequisites for Kubernetes Cluster
23-
24-
### Storage
25-
26-
Ensure your cluster has a storage class with encryption, or encryption is implemented at the filesystem level:
27-
28-
- **For existing OpenCRVS installations:**
29-
Make sure the cluster has at least the `hostpath` storage class configured and directories on the filesystem should point to encrypted partitions.
30-
`hostpath` is the best option for migration from Docker Swarm to Kubernetes; it allows data to remain untouched. Data can be migrated to more robust storage later, such as `local` or `nfs` volumes after OpenCRVS migration to Kubernetes.
31-
32-
- **For new installations:**
33-
- Please check the available storage options in the official documentation: [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/) and [Kubernetes Storage Classes Documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner).
34-
- The recommended storage class for new installations is NFS.
3523

36-
Additionally, explore all possible options for CSI (Container Storage Interface) at the [CSI GitHub repository](https://github.com/kubernetes-csi/).
37-
38-
**NOTE:** Depending on your available hardware resources, you may optimize the installation by splitting data across different types of volumes. For example:
39-
- `Hostpath` works better for Elasticsearch.
40-
- `NFS` is the best option for MinIO and Mongo (or Postgres).
4124

4225
---
4326

4427

45-
## [🚧 ] Manual deployment guide
46-
47-
1. Create yaml file with custom values for your installation:
48-
```yaml
49-
# Kubernetes load balancer domain used by traefik as entrypoint
50-
hostname: opencrvs.<you domain>
51-
# OpenCRVS Core image tag
52-
image:
53-
tag: local
54-
# Your country image repository and tag
55-
countryconfig:
56-
image:
57-
name: opencrvs/ocrvs-countryconfig
58-
tag: develop
59-
```
60-
2. Add helm repository:
61-
```
62-
helm repo add ...
63-
```
64-
3. Install OpenCRVS:
65-
```
66-
helm install ... opencrvs
67-
```
68-
**NOTE:** Data seed will run only on `install`, don't use `update --install`.
69-
70-
# [🚧 Coming soon] Server environment migration
71-
72-
TODO: Migration from docker swarm to kubernetes guide
73-
74-
# Development with Kubernetes
28+
# Developing OpenCRVS with Kubernetes
7529

7630
## Prerequisites
7731

@@ -112,7 +66,7 @@ You need to clone the [opencrvs-core](https://github.com/opencrvs/opencrvs-core)
11266
```
11367
6. [Temporary Step] Switch to the k8s-version branch:
11468
```bash
115-
git checkout k8s-version
69+
git checkout k8s-refresh
11670
```
11771
7. Run Tilt:
11872
```bash
@@ -123,27 +77,55 @@ You need to clone the [opencrvs-core](https://github.com/opencrvs/opencrvs-core)
12377

12478
---
12579

126-
## [🚧 Coming soon] For OpenCRVS Country Configuration Developers
80+
## For OpenCRVS Country Config Developers
12781

128-
You need to fork the [opencrvs-countryconfig](https://github.com/opencrvs/opencrvs-countryconfig) repository and clone the [infrastructure](https://github.com/opencrvs/infrastructure) repository. If these repositories are already on your laptop, ensure they are in the same folder.
82+
Please follow official documentation how to setup your own country configuration at [Set-up your own, local, country configuration](https://documentation.opencrvs.org/setup/3.-installation/3.2-set-up-your-own-country-configuration).
83+
You need to fork (clone) the [opencrvs-countryconfig](https://github.com/opencrvs/opencrvs-countryconfig) repository and clone the [infrastructure](https://github.com/opencrvs/infrastructure) repository. If repositories are already on your laptop, ensure they are in the same parent folder, for example:
84+
```
85+
repositories/
86+
infrastructure
87+
opencrvs-countryconfig
88+
...
89+
```
12990
130-
1. Create a new folder or use an existing folder to store the repositories.
91+
**Step by step instruction**
92+
93+
1. Create a new folder or use an existing folder to store the repositories. For example folder could be located at your home directory or in documents:
94+
```bash
95+
mkdir ~/Documents/repository
96+
```
13197
2. Open a terminal (command line) and navigate to the folder.
132-
3. Clone your fork of the OpenCRVS Country Configuration repository:
98+
```bash
99+
cd ~/Documents/repository
100+
```
101+
3. Clone OpenCRVS Country Config repository:
102+
103+
For county config use:
104+
```bash
105+
git clone https://github.com/opencrvs/opencrvs-countryconfig
106+
```
107+
For your own fork use:
133108
```bash
134109
git clone [email protected]:<your-github-account>/<your-repository>.git
135110
```
111+
136112
4. Clone the Infrastructure repository:
137113
```bash
138114
git clone [email protected]:opencrvs/infrastructure.git
139115
```
140-
5. Change directory to your forked repository:
116+
5. Change directory to country config (your own) repository:
117+
118+
For county config use:
119+
```bash
120+
cd opencrvs-countryconfig
121+
```
122+
For your own fork use:
141123
```bash
142124
cd <your-repository>
143125
```
144-
6. [Temporary Step] Switch to the k8s-version branch:
126+
6. [Temporary Step] Switch to the k8s-refresh branch:
145127
```bash
146-
git checkout k8s-version
128+
git checkout k8s-refresh
147129
```
148130
7. Run Tilt:
149131
```bash
@@ -152,6 +134,16 @@ You need to fork the [opencrvs-countryconfig](https://github.com/opencrvs/opencr
152134
8. Navigate to [http://localhost:10350/](http://localhost:10350/)
153135
9. Once all container images are up and running your environment will be available at https://opencrvs.localhost
154136

137+
## Seed data
138+
139+
1. Navigate to file `kubernetes/opencrvs-services/values-dev.yaml` in opencrvs-core (or your country config) repository
140+
2. Change value `data_seeder.enabled` to `true`.
141+
3. Save changes
142+
4. New tilt resource `data-seeder` will be created, check [http://localhost:10350/](http://localhost:10350/)
143+
5. Make sure data-seeder job completed without issues.
144+
6. Change value `data_seeder.enabled` to `false`.
145+
7. Save changes
146+
155147
## Common issues
156148

157149
### Container start is failing with ImagePullBackOff
@@ -162,11 +154,110 @@ Check image tag was set properly, use `kubectl`, adjust value in `kubernetes/ope
162154

163155
### Reset local environment
164156

165-
Restart docker desktop
157+
Draft and working way is to restart docker desktop
158+
159+
### Troubleshooting connectivity inside Kubernetes cluster
160+
161+
1. Issue fresh token:
162+
163+
```bash
164+
USERNAME=o.admin
165+
SUPER_USER_PASSWORD=password
166+
curl -X POST "http://auth.opencrvs-dev.svc.cluster.local:4040/authenticate-super-user" \
167+
-H "Content-Type: application/json" \
168+
-d '{
169+
"username": "'"${USERNAME}"'",
170+
"password": "'"$SUPER_USER_PASSWORD"'"
171+
}'
172+
```
173+
174+
2. Check gateway host:
175+
```bash
176+
GATEWAY_HOST=http://gateway.opencrvs-dev.svc.cluster.local:7070
177+
curl -X GET \
178+
-H "Content-Type: application/json" \
179+
-H "Authorization: Bearer ${token}" \
180+
${GATEWAY_HOST}/locations?type=ADMIN_STRUCTURE&_count=0
181+
```
182+
3. Check config host:
183+
```bash
184+
curl -v -X GET \
185+
-H "Content-Type: application/json" \
186+
-H "Authorization: Bearer ${token}" \
187+
http://config.opencrvs-dev.svc.cluster.local:2021/locations?type=ADMIN_STRUCTURE&_count=0
188+
```
189+
4. Check Hearth:
190+
```bash
191+
curl -v http://hearth.opencrvs-deps-dev.svc.cluster.local:3447/fhir/Location
192+
```
193+
194+
### Login/Client service is not responding: Check login logs
195+
```
196+
2025/03/19 07:53:38 [error] 15#15: *1 upstream timed out (110: Connection timed out) while connecting to upstream, client: 10.1.3.102, server: localhost, request: "GET /api/countryconfig/login-config.js HTTP/1.1", upstream: "http://10.100.14.175:3040/login-config.js", host: "login.opencrvs.localhost", referrer: "https://login.opencrvs.localhost/"
197+
```
198+
199+
Solution: restart nginx inside login container or delete login pod
200+
```
201+
nginx -s reload
202+
```
203+
204+
205+
---
206+
207+
# Running OpenCRVS on Kubernetes
208+
209+
## Prerequisites for Kubernetes Cluster
210+
211+
### Storage
212+
213+
Ensure your cluster has a storage class with encryption, or encryption is implemented at the filesystem level:
214+
215+
- **For existing OpenCRVS installations:**
216+
Make sure the cluster has at least the `hostpath` storage class configured and directories on the filesystem should point to encrypted partitions.
217+
`hostpath` is the best option for migration from Docker Swarm to Kubernetes; it allows data to remain untouched. Data can be migrated to more robust storage later, such as `local` or `nfs` volumes after OpenCRVS migration to Kubernetes.
218+
219+
- **For new installations:**
220+
- Please check the available storage options in the official documentation: [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/) and [Kubernetes Storage Classes Documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner).
221+
- The recommended storage class for new installations is NFS.
222+
223+
Additionally, explore all possible options for CSI (Container Storage Interface) at the [CSI GitHub repository](https://github.com/kubernetes-csi/).
224+
225+
**NOTE:** Depending on your available hardware resources, you may optimize the installation by splitting data across different types of volumes. For example:
226+
- `Hostpath` works better for Elasticsearch.
227+
- `NFS` is the best option for MinIO and Mongo (or Postgres).
166228
167229
---
168230
169231
232+
## [🚧 ] Manual deployment guide
233+
234+
1. Create yaml file with custom values for your installation:
235+
```yaml
236+
# Kubernetes load balancer domain used by traefik as entrypoint
237+
hostname: opencrvs.<you domain>
238+
# OpenCRVS Core image tag
239+
image:
240+
tag: local
241+
# Your country image repository and tag
242+
countryconfig:
243+
image:
244+
name: opencrvs/ocrvs-countryconfig
245+
tag: develop
246+
```
247+
2. Add helm repository:
248+
```
249+
helm repo add ...
250+
```
251+
3. Install OpenCRVS:
252+
```
253+
helm install ... opencrvs
254+
```
255+
**NOTE:** Data seed will run only on `install`, don't use `update --install`.
256+
257+
# [🚧 Coming soon] Server environment migration
258+
259+
TODO: Migration from docker swarm to kubernetes guide
260+
170261
# Useful Links
171262

172263
- [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/)

‎charts/opencrvs-services/README.md‎

Lines changed: 45 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
1818
<tr>
1919
<td>elasticsearch_host</td>
2020
<td>elasticsearch.opencrvs-deps-dev.svc.cluster.local:9200</td>
21-
<td>Elasticsearch configuration, including the hostname and port. TODO: Consider defining the port as a separate variable.</td>
21+
<td>Elasticsearch configuration, including the hostname and port.<br> <b>NOTE</b>: Some services require authentication, please use secrets to redefine ES_HOST variable if needed.</td>
2222
</tr>
2323
<tr>
2424
<td>influxdb.host</td>
@@ -55,6 +55,11 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
5555
<td>mongodb-0.mongodb.opencrvs-deps-dev.svc.cluster.local</td>
5656
<td>MongoDB hostname configuration.</td>
5757
</tr>
58+
<tr>
59+
<td>redis_host</td>
60+
<td>redis-0.redis.opencrvs-deps-dev.svc.cluster.local</td>
61+
<td>Redis hostname configuration.</td>
62+
</tr>
5863
<tr>
5964
<td>hostname</td>
6065
<td>farajaland.com</td>
@@ -80,26 +85,17 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
8085
<td>{}</td>
8186
<td>Mapping kubernetes secrets as environment variables. For more information see [Mapping secrets](#mapping-secrets)</td>
8287
</tr>
88+
<tr>
89+
<td>data_seeder.enabled</td>
90+
<td>true</td>
91+
<td>Seed data as post-install step, data seeder is executed only once while `helm install`. In some cases when data is already seeded, e/g upgrade, this value must be set to false. **Note**: default user is used for data seeding, it will fail anyway on database with non-default data.</td>
92+
</tr>
8393
</tbody>
8494
</table>
8595

86-
# Microservice environment variables configuration
87-
88-
<pre>Do we need this section?</pre>
89-
90-
Helm chart allows to define environment variables in following scopes:
91-
- **Global variables** are defined at top level of values file and is added to all containers. See `env` key in [values.yaml](values.yaml)
92-
- **Service level variables** are defined for each particular service. See `<service_name>.env` key in [values.yaml](values.yaml)
93-
- **Secret environment variables** are defined at service level as `<service_name>.secrets` key, see [values.yaml](values.yaml).
9496

9597
# Mapping secrets
9698

97-
Suppose we need to store ES_HOST variable as a secret since it contains url with login and password for Elastic search.
98-
Kubernetes secret is key/value object usually created from `.env` file, for example:
99-
```
100-
ES_HOST=user:randompass@elasticsearch:9200
101-
```
102-
10399
Mapping needs to be added for particular service to access variable inside workload (service), e/g for `search` service to access ES_HOST following configuration is needed:
104100
```
105101
search:
@@ -117,4 +113,37 @@ secrets:
117113
Summary:
118114
- `secret_name`, name of Kubernetes secret object
119115
- `secret_key`, key (variable name) inside Kubernetes secret data property
120-
- `environment_variable`, environment variable name inside container. If `secret_key` value `environment_variable` are the same, last one can be omitted.
116+
- `environment_variable`, environment variable name inside container. If `secret_key` value `environment_variable` are the same, last one can be omitted.
117+
118+
**Step by step example**
119+
120+
Suppose we need to store ES_HOST variable as a secret and provide variable value to service `search`.
121+
122+
1. Create `.env` like file and put all variables:
123+
```
124+
ES_HOST=user:randompass@elasticsearch:9200
125+
```
126+
2. Create kubernetes secret from `.env` file:
127+
```
128+
kubectl create secret generic elasticsearch-secret --from-env-file=.env
129+
```
130+
3. Make sure the secret was created:
131+
```
132+
kubectl get secret -oyaml elasticsearch-secret
133+
```
134+
Example output:
135+
```yaml
136+
apiVersion: v1
137+
data:
138+
ES_HOST: dXNlcjpyYW5kb21wYXNzQGVsYXN0aWNzZWFyY2g6OTIwMA==
139+
...
140+
```
141+
3. Map variable in your helm chart values file:
142+
```yaml
143+
search:
144+
secrets:
145+
elasticsearch-secret:
146+
- ES_HOST
147+
...
148+
```
149+
4. Redeploy service with `helm upgrade`

0 commit comments

Comments
 (0)