Skip to content

Commit 0031ea4

Browse files
mpgxviimpgxvii
authored
Merge pull request #7754 from RADAR-base/feat/radar-output-hpa
feat/add HPA support to radar-output
2 parents 7d3b7f3 + 4d83a58 commit 0031ea4

5 files changed

Lines changed: 73 additions & 4 deletions

File tree

‎charts/radar-output/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ apiVersion: v2
22
appVersion: "3.0.6"
33
description: A Helm chart for RADAR-base output restructure service. This application reads data from intermediate storage and restructure the data into project-> subject-id-> data topic -> data split per hour. This service offers few options to choose the source and target of the pipeline.
44
name: radar-output
5-
version: 1.2.8
5+
version: 1.2.9
66
icon: "http://radar-base.org/wp-content/uploads/2022/09/Logo_RADAR-Base-RGB.png"
77
sources:
88
- https://github.com/RADAR-base/radar-helm-charts/tree/main/charts/radar-output

‎charts/radar-output/README.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# radar-output
44
[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/radar-output)](https://artifacthub.io/packages/helm/radar-base/radar-output)
55

6-
![Version: 1.2.8](https://img.shields.io/badge/Version-1.2.8-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.0.6](https://img.shields.io/badge/AppVersion-3.0.6-informational?style=flat-square)
6+
![Version: 1.2.9](https://img.shields.io/badge/Version-1.2.9-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.0.6](https://img.shields.io/badge/AppVersion-3.0.6-informational?style=flat-square)
77

88
A Helm chart for RADAR-base output restructure service. This application reads data from intermediate storage and restructure the data into project-> subject-id-> data topic -> data split per hour. This service offers few options to choose the source and target of the pipeline.
99

@@ -107,12 +107,17 @@ A Helm chart for RADAR-base output restructure service. This application reads d
107107
| paths.input | string | `"topics"` | Relative path to intermediate storage root to browse for data |
108108
| paths.output | string | `"output"` | Relative path to output storage to write data |
109109
| paths.factory | string | `"org.radarbase.output.path.FormattedPathFactory"` | Output path construction factory |
110+
| paths.path.format | string | `"${projectId}/${userId}/${topic}/${filename}"` | Output path format |
110111
| paths.properties | object | `{}` | Additional properties. For details see https://github.com/RADAR-base/radar-output-restructure/blob/master/restructure.yml |
111112
| topics | object | `{"questionnaire_response":{"pathProperties":{"format":"${projectId}/${userId}/${topic}/${value:name}/${filename}","plugins":"fixed value"}}}` | Individual topic configuration |
112113
| topics.questionnaire_response.pathProperties.format | string | `"${projectId}/${userId}/${topic}/${value:name}/${filename}"` | Alternative path output of the questionnaire_response topic |
113114
| topics.questionnaire_response.pathProperties.plugins | string | `"fixed value"` | Alternative path plugins of the questionnaire_response topic |
114115
| deduplication.enable | bool | `true` | Whether to enable deduplication |
115116
| compression.type | string | `"gzip"` | Compression type to use for output files. Can be one of: gzip, zip, none |
117+
| hpa.enabled | bool | `false` | Enable HPA |
118+
| hpa.maxReplicas | int | `5` | Maximum number of replicas |
119+
| hpa.targetCPU | int | `80` | Target CPU utilization percentage |
120+
| hpa.targetMemory | string | `nil` | Target Memory utilization percentage |
116121

117122
## Cost Considerations
118123

‎charts/radar-output/templates/configmap.yaml‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -140,9 +140,22 @@ data:
140140
output: "{{ .Values.paths.output }}"
141141
# Output path construction factory
142142
factory: {{ .Values.paths.factory }}
143-
# Additional properties
143+
144+
# Global path formatter config (FormattedPathFactory)
145+
path:
146+
format: {{ .Values.paths.path.format | quote }}
147+
{{- if .Values.paths.path.plugins }}
148+
plugins: {{ .Values.paths.path.plugins | quote }}
149+
{{- end }}
150+
{{- if .Values.paths.path.properties }}
151+
properties:
152+
{{ toYaml .Values.paths.path.properties | indent 10 }}
153+
{{- end }}
154+
# Optional: legacy/general plugin properties merged into path.properties by the app
155+
{{- if .Values.paths.properties }}
144156
properties:
145-
{{ .Values.paths.properties | toYaml | indent 8 | trim }}
157+
{{ toYaml .Values.paths.properties | indent 8 }}
158+
{{- end }}
146159

147160
# Individual topic configuration
148161
{{- if .Values.topics }}

‎charts/radar-output/templates/hpa.yaml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
{{- if .Values.hpa.enabled }}
2+
apiVersion: autoscaling/v2
3+
kind: HorizontalPodAutoscaler
4+
metadata:
5+
name: {{ template "common.names.fullname" . }}
6+
namespace: {{ include "common.names.namespace" . | quote }}
7+
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
8+
{{- if .Values.commonAnnotations }}
9+
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
10+
{{- end }}
11+
spec:
12+
scaleTargetRef:
13+
apiVersion: apps/v1
14+
kind: Deployment
15+
name: {{ include "radar-output.fullname" . }}
16+
minReplicas: {{ .Values.replicaCount }}
17+
maxReplicas: {{ .Values.hpa.maxReplicas }}
18+
metrics:
19+
{{- if .Values.hpa.targetCPU }}
20+
- type: Resource
21+
resource:
22+
name: cpu
23+
target:
24+
type: Utilization
25+
averageUtilization: {{ .Values.hpa.targetCPU }}
26+
{{- end }}
27+
{{- if .Values.hpa.targetMemory }}
28+
- type: Resource
29+
resource:
30+
name: memory
31+
target:
32+
type: Utilization
33+
averageUtilization: {{ .Values.hpa.targetMemory }}
34+
{{- end }}
35+
{{- end }}

‎charts/radar-output/values.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,11 @@ paths:
238238
output: output
239239
# -- Output path construction factory
240240
factory: org.radarbase.output.path.FormattedPathFactory
241+
path:
242+
# -- Output path format
243+
format: ${projectId}/${userId}/${topic}/${filename}
244+
# -- Output path plugins
245+
# plugins: fixed time key value
241246
# -- Additional properties. For details see https://github.com/RADAR-base/radar-output-restructure/blob/master/restructure.yml
242247
properties: {}
243248

@@ -270,3 +275,14 @@ deduplication:
270275
compression:
271276
# -- Compression type to use for output files. Can be one of: gzip, zip, none
272277
type: gzip
278+
279+
# Horizontal Pod Autoscaler
280+
hpa:
281+
# -- Enable HPA
282+
enabled: false
283+
# -- Maximum number of replicas
284+
maxReplicas: 5
285+
# -- Target CPU utilization percentage
286+
targetCPU: 80
287+
# -- Target Memory utilization percentage
288+
targetMemory: ~

0 commit comments

Comments
 (0)