Skip to content

Commit abae3a2

Browse files
author
Vadym Mudryi
committed
updated readme
1 parent 960a2e1 commit abae3a2

File tree

6 files changed

+193
-133
lines changed

6 files changed

+193
-133
lines changed

‎README.md‎

Lines changed: 147 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -17,61 +17,8 @@ This repository is used to store infrastructure code for deploying OpenCRVS.
1717

1818
---
1919

20-
# Running OpenCRVS on Kubernetes
21-
22-
## Prerequisites for Kubernetes Cluster
23-
24-
### Storage
25-
26-
Ensure your cluster has a storage class with encryption, or encryption is implemented at the filesystem level:
27-
28-
- **For existing OpenCRVS installations:**
29-
Make sure the cluster has at least the `hostpath` storage class configured and directories on the filesystem should point to encrypted partitions.
30-
`hostpath` is the best option for migration from Docker Swarm to Kubernetes; it allows data to remain untouched. Data can be migrated to more robust storage later, such as `local` or `nfs` volumes after OpenCRVS migration to Kubernetes.
31-
32-
- **For new installations:**
33-
- Please check the available storage options in the official documentation: [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/) and [Kubernetes Storage Classes Documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner).
34-
- The recommended storage class for new installations is NFS.
35-
36-
Additionally, explore all possible options for CSI (Container Storage Interface) at the [CSI GitHub repository](https://github.com/kubernetes-csi/).
37-
38-
**NOTE:** Depending on your available hardware resources, you may optimize the installation by splitting data across different types of volumes. For example:
39-
- `Hostpath` works better for Elasticsearch.
40-
- `NFS` is the best option for MinIO and Mongo (or Postgres).
41-
42-
---
43-
44-
45-
## [🚧 ] Manual deployment guide
46-
47-
1. Create yaml file with custom values for your installation:
48-
```yaml
49-
# Kubernetes load balancer domain used by traefik as entrypoint
50-
hostname: opencrvs.<you domain>
51-
# OpenCRVS Core image tag
52-
image:
53-
tag: local
54-
# Your country image repository and tag
55-
countryconfig:
56-
image:
57-
name: opencrvs/ocrvs-countryconfig
58-
tag: develop
59-
```
60-
2. Add helm repository:
61-
```
62-
helm repo add ...
63-
```
64-
3. Install OpenCRVS:
65-
```
66-
helm install ... opencrvs
67-
```
68-
**NOTE:** Data seed will run only on `install`, don't use `update --install`.
69-
70-
# [🚧 Coming soon] Server environment migration
71-
72-
TODO: Migration from docker swarm to kubernetes guide
7320

74-
# Development with Kubernetes
21+
# Developing OpenCRVS with Kubernetes
7522

7623
## Prerequisites
7724

@@ -112,7 +59,7 @@ You need to clone the [opencrvs-core](https://github.com/opencrvs/opencrvs-core)
11259
```
11360
6. [Temporary Step] Switch to the k8s-version branch:
11461
```bash
115-
git checkout k8s-version
62+
git checkout k8s-refresh
11663
```
11764
7. Run Tilt:
11865
```bash
@@ -123,27 +70,55 @@ You need to clone the [opencrvs-core](https://github.com/opencrvs/opencrvs-core)
12370

12471
---
12572

126-
## [🚧 Coming soon] For OpenCRVS Country Configuration Developers
73+
## For OpenCRVS Country Config Developers
12774

128-
You need to fork the [opencrvs-countryconfig](https://github.com/opencrvs/opencrvs-countryconfig) repository and clone the [infrastructure](https://github.com/opencrvs/infrastructure) repository. If these repositories are already on your laptop, ensure they are in the same folder.
75+
Please follow official documentation how to setup your own country configuration at [Set-up your own, local, country configuration](https://documentation.opencrvs.org/setup/3.-installation/3.2-set-up-your-own-country-configuration).
76+
You need to fork (clone) the [opencrvs-countryconfig](https://github.com/opencrvs/opencrvs-countryconfig) repository and clone the [infrastructure](https://github.com/opencrvs/infrastructure) repository. If repositories are already on your laptop, ensure they are in the same parent folder, for example:
77+
```
78+
repositories/
79+
infrastructure
80+
opencrvs-countryconfig
81+
...
82+
```
12983
130-
1. Create a new folder or use an existing folder to store the repositories.
84+
**Step by step instruction**
85+
86+
1. Create a new folder or use an existing folder to store the repositories. For example folder could be located at your home directory or in documents:
87+
```bash
88+
mkdir ~/Documents/repository
89+
```
13190
2. Open a terminal (command line) and navigate to the folder.
132-
3. Clone your fork of the OpenCRVS Country Configuration repository:
91+
```bash
92+
cd ~/Documents/repository
93+
```
94+
3. Clone OpenCRVS Country Config repository:
95+
96+
For county config use:
97+
```bash
98+
git clone https://github.com/opencrvs/opencrvs-countryconfig
99+
```
100+
For your own fork use:
133101
```bash
134102
git clone [email protected]:<your-github-account>/<your-repository>.git
135103
```
104+
136105
4. Clone the Infrastructure repository:
137106
```bash
138107
git clone [email protected]:opencrvs/infrastructure.git
139108
```
140-
5. Change directory to your forked repository:
109+
5. Change directory to country config (your own) repository:
110+
111+
For county config use:
112+
```bash
113+
cd opencrvs-countryconfig
114+
```
115+
For your own fork use:
141116
```bash
142117
cd <your-repository>
143118
```
144-
6. [Temporary Step] Switch to the k8s-version branch:
119+
6. [Temporary Step] Switch to the k8s-refresh branch:
145120
```bash
146-
git checkout k8s-version
121+
git checkout k8s-refresh
147122
```
148123
7. Run Tilt:
149124
```bash
@@ -152,6 +127,16 @@ You need to fork the [opencrvs-countryconfig](https://github.com/opencrvs/opencr
152127
8. Navigate to [http://localhost:10350/](http://localhost:10350/)
153128
9. Once all container images are up and running your environment will be available at https://opencrvs.localhost
154129

130+
## Seed data
131+
132+
1. Navigate to file `kubernetes/opencrvs-services/values-dev.yaml` in opencrvs-core (or your country config) repository
133+
2. Change value `data_seeder.enabled` to `true`.
134+
3. Save changes
135+
4. New tilt resource `data-seeder` will be created, check [http://localhost:10350/](http://localhost:10350/)
136+
5. Make sure data-seeder job completed without issues.
137+
6. Change value `data_seeder.enabled` to `false`.
138+
7. Save changes
139+
155140
## Common issues
156141

157142
### Container start is failing with ImagePullBackOff
@@ -162,11 +147,110 @@ Check image tag was set properly, use `kubectl`, adjust value in `kubernetes/ope
162147

163148
### Reset local environment
164149

165-
Restart docker desktop
150+
Draft and working way is to restart docker desktop
151+
152+
### Troubleshooting connectivity inside Kubernetes cluster
153+
154+
1. Issue fresh token:
155+
156+
```bash
157+
USERNAME=o.admin
158+
SUPER_USER_PASSWORD=password
159+
curl -X POST "http://auth.opencrvs-dev.svc.cluster.local:4040/authenticate-super-user" \
160+
-H "Content-Type: application/json" \
161+
-d '{
162+
"username": "'"${USERNAME}"'",
163+
"password": "'"$SUPER_USER_PASSWORD"'"
164+
}'
165+
```
166+
167+
2. Check gateway host:
168+
```bash
169+
GATEWAY_HOST=http://gateway.opencrvs-dev.svc.cluster.local:7070
170+
curl -X GET \
171+
-H "Content-Type: application/json" \
172+
-H "Authorization: Bearer ${token}" \
173+
${GATEWAY_HOST}/locations?type=ADMIN_STRUCTURE&_count=0
174+
```
175+
3. Check config host:
176+
```bash
177+
curl -v -X GET \
178+
-H "Content-Type: application/json" \
179+
-H "Authorization: Bearer ${token}" \
180+
http://config.opencrvs-dev.svc.cluster.local:2021/locations?type=ADMIN_STRUCTURE&_count=0
181+
```
182+
4. Check Hearth:
183+
```bash
184+
curl -v http://hearth.opencrvs-deps-dev.svc.cluster.local:3447/fhir/Location
185+
```
186+
187+
### Login/Client service is not responding: Check login logs
188+
```
189+
2025/03/19 07:53:38 [error] 15#15: *1 upstream timed out (110: Connection timed out) while connecting to upstream, client: 10.1.3.102, server: localhost, request: "GET /api/countryconfig/login-config.js HTTP/1.1", upstream: "http://10.100.14.175:3040/login-config.js", host: "login.opencrvs.localhost", referrer: "https://login.opencrvs.localhost/"
190+
```
191+
192+
Solution: restart nginx inside login container or delete login pod
193+
```
194+
nginx -s reload
195+
```
196+
197+
198+
---
199+
200+
# Running OpenCRVS on Kubernetes
201+
202+
## Prerequisites for Kubernetes Cluster
203+
204+
### Storage
205+
206+
Ensure your cluster has a storage class with encryption, or encryption is implemented at the filesystem level:
207+
208+
- **For existing OpenCRVS installations:**
209+
Make sure the cluster has at least the `hostpath` storage class configured and directories on the filesystem should point to encrypted partitions.
210+
`hostpath` is the best option for migration from Docker Swarm to Kubernetes; it allows data to remain untouched. Data can be migrated to more robust storage later, such as `local` or `nfs` volumes after OpenCRVS migration to Kubernetes.
211+
212+
- **For new installations:**
213+
- Please check the available storage options in the official documentation: [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/) and [Kubernetes Storage Classes Documentation](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner).
214+
- The recommended storage class for new installations is NFS.
215+
216+
Additionally, explore all possible options for CSI (Container Storage Interface) at the [CSI GitHub repository](https://github.com/kubernetes-csi/).
217+
218+
**NOTE:** Depending on your available hardware resources, you may optimize the installation by splitting data across different types of volumes. For example:
219+
- `Hostpath` works better for Elasticsearch.
220+
- `NFS` is the best option for MinIO and Mongo (or Postgres).
166221
167222
---
168223
169224
225+
## [🚧 ] Manual deployment guide
226+
227+
1. Create yaml file with custom values for your installation:
228+
```yaml
229+
# Kubernetes load balancer domain used by traefik as entrypoint
230+
hostname: opencrvs.<you domain>
231+
# OpenCRVS Core image tag
232+
image:
233+
tag: local
234+
# Your country image repository and tag
235+
countryconfig:
236+
image:
237+
name: opencrvs/ocrvs-countryconfig
238+
tag: develop
239+
```
240+
2. Add helm repository:
241+
```
242+
helm repo add ...
243+
```
244+
3. Install OpenCRVS:
245+
```
246+
helm install ... opencrvs
247+
```
248+
**NOTE:** Data seed will run only on `install`, don't use `update --install`.
249+
250+
# [🚧 Coming soon] Server environment migration
251+
252+
TODO: Migration from docker swarm to kubernetes guide
253+
170254
# Useful Links
171255

172256
- [Kubernetes Volumes Documentation](https://kubernetes.io/docs/concepts/storage/volumes/)

‎charts/opencrvs-services/README.md‎

Lines changed: 45 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
1818
<tr>
1919
<td>elasticsearch_host</td>
2020
<td>elasticsearch.opencrvs-deps-dev.svc.cluster.local:9200</td>
21-
<td>Elasticsearch configuration, including the hostname and port. TODO: Consider defining the port as a separate variable.</td>
21+
<td>Elasticsearch configuration, including the hostname and port.<br> <b>NOTE</b>: Some services require authentication, please use secrets to redefine ES_HOST variable if needed.</td>
2222
</tr>
2323
<tr>
2424
<td>influxdb.host</td>
@@ -55,6 +55,11 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
5555
<td>mongodb-0.mongodb.opencrvs-deps-dev.svc.cluster.local</td>
5656
<td>MongoDB hostname configuration.</td>
5757
</tr>
58+
<tr>
59+
<td>redis_host</td>
60+
<td>redis-0.redis.opencrvs-deps-dev.svc.cluster.local</td>
61+
<td>Redis hostname configuration.</td>
62+
</tr>
5863
<tr>
5964
<td>hostname</td>
6065
<td>farajaland.com</td>
@@ -80,26 +85,17 @@ Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
8085
<td>{}</td>
8186
<td>Mapping kubernetes secrets as environment variables. For more information see [Mapping secrets](#mapping-secrets)</td>
8287
</tr>
88+
<tr>
89+
<td>data_seeder.enabled</td>
90+
<td>true</td>
91+
<td>Seed data as post-install step, data seeder is executed only once while `helm install`. In some cases when data is already seeded, e/g upgrade, this value must be set to false. **Note**: default user is used for data seeding, it will fail anyway on database with non-default data.</td>
92+
</tr>
8393
</tbody>
8494
</table>
8595

86-
# Microservice environment variables configuration
87-
88-
<pre>Do we need this section?</pre>
89-
90-
Helm chart allows to define environment variables in following scopes:
91-
- **Global variables** are defined at top level of values file and is added to all containers. See `env` key in [values.yaml](values.yaml)
92-
- **Service level variables** are defined for each particular service. See `<service_name>.env` key in [values.yaml](values.yaml)
93-
- **Secret environment variables** are defined at service level as `<service_name>.secrets` key, see [values.yaml](values.yaml).
9496

9597
# Mapping secrets
9698

97-
Suppose we need to store ES_HOST variable as a secret since it contains url with login and password for Elastic search.
98-
Kubernetes secret is key/value object usually created from `.env` file, for example:
99-
```
100-
ES_HOST=user:randompass@elasticsearch:9200
101-
```
102-
10399
Mapping needs to be added for particular service to access variable inside workload (service), e/g for `search` service to access ES_HOST following configuration is needed:
104100
```
105101
search:
@@ -117,4 +113,37 @@ secrets:
117113
Summary:
118114
- `secret_name`, name of Kubernetes secret object
119115
- `secret_key`, key (variable name) inside Kubernetes secret data property
120-
- `environment_variable`, environment variable name inside container. If `secret_key` value `environment_variable` are the same, last one can be omitted.
116+
- `environment_variable`, environment variable name inside container. If `secret_key` value `environment_variable` are the same, last one can be omitted.
117+
118+
**Step by step example**
119+
120+
Suppose we need to store ES_HOST variable as a secret and provide variable value to service `search`.
121+
122+
1. Create `.env` like file and put all variables:
123+
```
124+
ES_HOST=user:randompass@elasticsearch:9200
125+
```
126+
2. Create kubernetes secret from `.env` file:
127+
```
128+
kubectl create secret generic elasticsearch-secret --from-env-file=.env
129+
```
130+
3. Make sure the secret was created:
131+
```
132+
kubectl get secret -oyaml elasticsearch-secret
133+
```
134+
Example output:
135+
```yaml
136+
apiVersion: v1
137+
data:
138+
ES_HOST: dXNlcjpyYW5kb21wYXNzQGVsYXN0aWNzZWFyY2g6OTIwMA==
139+
...
140+
```
141+
3. Map variable in your helm chart values file:
142+
```yaml
143+
search:
144+
secrets:
145+
elasticsearch-secret:
146+
- ES_HOST
147+
...
148+
```
149+
4. Redeploy service with `helm upgrade`

0 commit comments

Comments
 (0)