Skip to content

Commit aebfca8

Browse files
jnt0rjnt0r
committed
am
Signed-off-by: Jonathan Pollert <38696668+jnt0r@users.noreply.github.com>
1 parent 7a308d0 commit aebfca8

4 files changed

Lines changed: 767 additions & 767 deletions

File tree

‎.features/pending/persistence-schema.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Description: You can now configure a specific postgres database schema for Argo Persistence in the controller config map.
2-
Authors: [Jonathan Pollert](https://github.com/jnt0r)
3-
Component: General
4-
Issues: 2452
5-
1+
Description: You can now configure a specific postgres database schema for Argo Persistence in the controller config map.
2+
Authors: [Jonathan Pollert](https://github.com/jnt0r)
3+
Component: General
4+
Issues: 2452
5+
66
Added support for custom database schemas to improve data isolation and security in shared environments. This allows Argo to operate within a designated logical schema rather than the default. Note: This feature is not applicable to MySQL, as it does not support logical schemas; for MySQL users, database names should continue to be used for application separation.

‎docs/workflow-archive.md‎

Lines changed: 108 additions & 108 deletions
Original file line numberDiff line numberDiff line change
@@ -1,108 +1,108 @@
1-
# Workflow Archive
2-
3-
> v2.5 and after
4-
5-
If you want to keep completed workflows for a long time, you can use the workflow archive to save them in a Postgres (>=9.4) or MySQL (>= 5.7.8) database.
6-
The workflow archive stores the status of the workflow, which pods have been executed, what was the result etc.
7-
The job logs of the workflow pods will not be archived.
8-
If you need to save the logs of the pods, you must setup an [artifact repository](artifact-repository-ref.md) according to [this doc](configure-artifact-repository.md).
9-
10-
The quick-start deployment includes a Postgres database server.
11-
In this case the workflow archive is already enabled.
12-
Such a deployment is convenient for test environments, but in a production environment you must use a production quality database service.
13-
14-
## Enabling Workflow Archive
15-
16-
To enable archiving of the workflows, you must configure database parameters in the `persistence` section of [your configuration](workflow-controller-configmap.yaml) and set `archive:` to `true`.
17-
18-
Example:
19-
20-
persistence:
21-
archive: true
22-
postgresql:
23-
host: localhost
24-
port: 5432
25-
database: postgres
26-
tableName: argo_workflows
27-
userNameSecret:
28-
name: argo-postgres-config
29-
key: username
30-
passwordSecret:
31-
name: argo-postgres-config
32-
key: password
33-
34-
You must also create the secret with database user and password in the namespace of the workflow controller.
35-
36-
Example:
37-
38-
kubectl create secret generic argo-postgres-config -n argo --from-literal=password=mypassword --from-literal=username=argodbuser
39-
40-
Note that IAM-based authentication is not currently supported. However, you can start your database proxy as a sidecar
41-
(e.g. via [CloudSQL Proxy](https://github.com/GoogleCloudPlatform/cloud-sql-proxy) on GCP) and then specify your local
42-
proxy address, IAM username, and an empty string as your password in the persistence configuration to connect to it.
43-
44-
The following tables will be created in the database when you start the workflow controller with enabled archive:
45-
46-
* `argo_workflows`
47-
* `argo_archived_workflows`
48-
* `argo_archived_workflows_labels`
49-
* `schema_history`
50-
51-
## Automatic Database Migration
52-
53-
Every time the Argo workflow-controller starts with persistence enabled, it tries to migrate the database to the correct version.
54-
If the database migration fails, the workflow-controller will also fail to start.
55-
In this case you can delete all the above tables and restart the workflow-controller.
56-
57-
If you know what are you doing you also have an option to skip migration:
58-
59-
persistence:
60-
skipMigration: true
61-
62-
## Required database permissions
63-
64-
### Postgres
65-
66-
The database user/role must have `CREATE` and `USAGE` permissions on the `public` schema of the database so that the tables can be created during the migration.
67-
68-
## Archive TTL
69-
70-
You can configure the time period to keep archived workflows before they will be deleted by the archived workflow garbage collection function.
71-
The default is forever.
72-
73-
Example:
74-
75-
persistence:
76-
archiveTTL: 10d
77-
78-
The `ARCHIVED_WORKFLOW_GC_PERIOD` variable defines the periodicity of running the garbage collection function.
79-
The default value is documented [here](environment-variables.md).
80-
When the workflow controller starts, it sets the ticker to run every `ARCHIVED_WORKFLOW_GC_PERIOD`.
81-
It does not run the garbage collection function immediately and the first garbage collection happens only after the period defined in the `ARCHIVED_WORKFLOW_GC_PERIOD` variable.
82-
83-
## Cluster Name
84-
85-
Optionally you can set a unique name of your Kubernetes cluster. This name will populate the `clustername` field in the `argo_archived_workflows` table.
86-
87-
Example:
88-
89-
persistence:
90-
clusterName: dev-cluster
91-
92-
## Disabling Workflow Archive
93-
94-
To disable archiving of the workflows, set `archive:` to `false` in the `persistence` section of [your configuration](workflow-controller-configmap.yaml).
95-
96-
Example:
97-
98-
persistence:
99-
archive: false
100-
101-
## Customizing PostgreSQL Database Schema
102-
103-
To change the schema for the tables in PostgreSQL, set `schema:` to the desired schema in the `persistence` section of [your configuration](workflow-controller-configmap.yaml). Only available on PostgreSQL.
104-
105-
Example:
106-
107-
persistence:
108-
schema: argo
1+
# Workflow Archive
2+
3+
> v2.5 and after
4+
5+
If you want to keep completed workflows for a long time, you can use the workflow archive to save them in a Postgres (>=9.4) or MySQL (>= 5.7.8) database.
6+
The workflow archive stores the status of the workflow, which pods have been executed, what was the result etc.
7+
The job logs of the workflow pods will not be archived.
8+
If you need to save the logs of the pods, you must setup an [artifact repository](artifact-repository-ref.md) according to [this doc](configure-artifact-repository.md).
9+
10+
The quick-start deployment includes a Postgres database server.
11+
In this case the workflow archive is already enabled.
12+
Such a deployment is convenient for test environments, but in a production environment you must use a production quality database service.
13+
14+
## Enabling Workflow Archive
15+
16+
To enable archiving of the workflows, you must configure database parameters in the `persistence` section of [your configuration](workflow-controller-configmap.yaml) and set `archive:` to `true`.
17+
18+
Example:
19+
20+
persistence:
21+
archive: true
22+
postgresql:
23+
host: localhost
24+
port: 5432
25+
database: postgres
26+
tableName: argo_workflows
27+
userNameSecret:
28+
name: argo-postgres-config
29+
key: username
30+
passwordSecret:
31+
name: argo-postgres-config
32+
key: password
33+
34+
You must also create the secret with database user and password in the namespace of the workflow controller.
35+
36+
Example:
37+
38+
kubectl create secret generic argo-postgres-config -n argo --from-literal=password=mypassword --from-literal=username=argodbuser
39+
40+
Note that IAM-based authentication is not currently supported. However, you can start your database proxy as a sidecar
41+
(e.g. via [CloudSQL Proxy](https://github.com/GoogleCloudPlatform/cloud-sql-proxy) on GCP) and then specify your local
42+
proxy address, IAM username, and an empty string as your password in the persistence configuration to connect to it.
43+
44+
The following tables will be created in the database when you start the workflow controller with enabled archive:
45+
46+
* `argo_workflows`
47+
* `argo_archived_workflows`
48+
* `argo_archived_workflows_labels`
49+
* `schema_history`
50+
51+
## Automatic Database Migration
52+
53+
Every time the Argo workflow-controller starts with persistence enabled, it tries to migrate the database to the correct version.
54+
If the database migration fails, the workflow-controller will also fail to start.
55+
In this case you can delete all the above tables and restart the workflow-controller.
56+
57+
If you know what are you doing you also have an option to skip migration:
58+
59+
persistence:
60+
skipMigration: true
61+
62+
## Required database permissions
63+
64+
### Postgres
65+
66+
The database user/role must have `CREATE` and `USAGE` permissions on the `public` schema of the database so that the tables can be created during the migration.
67+
68+
## Archive TTL
69+
70+
You can configure the time period to keep archived workflows before they will be deleted by the archived workflow garbage collection function.
71+
The default is forever.
72+
73+
Example:
74+
75+
persistence:
76+
archiveTTL: 10d
77+
78+
The `ARCHIVED_WORKFLOW_GC_PERIOD` variable defines the periodicity of running the garbage collection function.
79+
The default value is documented [here](environment-variables.md).
80+
When the workflow controller starts, it sets the ticker to run every `ARCHIVED_WORKFLOW_GC_PERIOD`.
81+
It does not run the garbage collection function immediately and the first garbage collection happens only after the period defined in the `ARCHIVED_WORKFLOW_GC_PERIOD` variable.
82+
83+
## Cluster Name
84+
85+
Optionally you can set a unique name of your Kubernetes cluster. This name will populate the `clustername` field in the `argo_archived_workflows` table.
86+
87+
Example:
88+
89+
persistence:
90+
clusterName: dev-cluster
91+
92+
## Disabling Workflow Archive
93+
94+
To disable archiving of the workflows, set `archive:` to `false` in the `persistence` section of [your configuration](workflow-controller-configmap.yaml).
95+
96+
Example:
97+
98+
persistence:
99+
archive: false
100+
101+
## Customizing PostgreSQL Database Schema
102+
103+
To change the schema for the tables in PostgreSQL, set `schema:` to the desired schema in the `persistence` section of [your configuration](workflow-controller-configmap.yaml). Only available on PostgreSQL.
104+
105+
Example:
106+
107+
persistence:
108+
schema: argo

0 commit comments

Comments
 (0)