Skip to content

Commit 122af26

Browse files
cypresclaude
andauthored
new fix for prometheus ports (#670)
* Properly fix unique Prometheus ports per service (redo of #649) PR #649 fixed port conflicts only in the bazel run scripts, meaning the problem persisted when services were launched directly via their bazel targets. This commit fixes it at the source by overriding metrics_prometheus_port in each service's config class, so the correct port is used regardless of how the service is started. Port assignments (core stays at the base default of 9464): - worker: 9465 - delayed_job_monitor: 9466 - backend_listener: 9467 - backend_worker: 9468 Kubernetes manifests are updated to match these new defaults. Also fixes a pre-existing port name collision: the oauth2-proxy sidecar declared its metrics port as "metrics", conflicting with the OSMO service container port of the same name in the same pod. Renamed to "oauth2-metrics" across all three chart sidecar helpers (service, router, web-ui). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert run script port overrides made redundant by previous commit The --metrics_prometheus_port flags added to the bazel run scripts by #649 are now superseded by the per-service config class defaults. Remove them to keep the scripts clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add oauth2-metrics endpoint to service PodMonitor The oauth2-proxy sidecar port was renamed from "metrics" to "oauth2-metrics" to avoid a pod-level port name collision. Add a corresponding PodMonitor endpoint so Prometheus continues scraping oauth2-proxy metrics when the sidecar is enabled. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix pylint missing-class-docstring in WorkerConfig Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 00b527c commit 122af26

File tree

13 files changed

+36
-14
lines changed

13 files changed

+36
-14
lines changed

deployments/charts/backend-operator/templates/backend-listener.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ spec:
7575
runAsUser: 1001
7676
ports:
7777
- name: metrics
78-
containerPort: 9464
78+
containerPort: 9467
7979
command:
8080
- backend_listener
8181
args:

deployments/charts/backend-operator/templates/backend-worker.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ spec:
7575
runAsUser: 1001
7676
ports:
7777
- name: metrics
78-
containerPort: 9464
78+
containerPort: 9468
7979
command:
8080
- backend_worker
8181
args:

deployments/charts/router/templates/_sidecar-helpers.tpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ OAuth2 Proxy sidecar container
119119
ports:
120120
- name: http
121121
containerPort: {{ .Values.sidecars.oauth2Proxy.httpPort }}
122-
- name: metrics
122+
- name: oauth2-metrics
123123
containerPort: {{ .Values.sidecars.oauth2Proxy.metricsPort }}
124124
livenessProbe:
125125
httpGet:

deployments/charts/service/templates/_sidecar-helpers.tpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,7 @@ OAuth2 Proxy sidecar container
245245
ports:
246246
- name: http
247247
containerPort: {{ .Values.sidecars.oauth2Proxy.httpPort }}
248-
- name: metrics
248+
- name: oauth2-metrics
249249
containerPort: {{ .Values.sidecars.oauth2Proxy.metricsPort }}
250250
livenessProbe:
251251
httpGet:

deployments/charts/service/templates/api-monitor.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,11 @@ spec:
2828
port: envoy-admin
2929
path: /stats/prometheus
3030
{{- end }}
31+
{{- if .Values.sidecars.oauth2Proxy.enabled }}
32+
- interval: 15s
33+
port: oauth2-metrics
34+
path: /metrics
35+
{{- end }}
3136
selector:
3237
matchExpressions:
3338
- { key: app,

deployments/charts/service/templates/delayed-job-monitor.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ spec:
6363
runAsUser: 1001
6464
ports:
6565
- name: metrics
66-
containerPort: 9464
66+
containerPort: 9466
6767
command:
6868
- delayed_job_monitor
6969
imagePullPolicy: {{ .Values.services.delayedJobMonitor.imagePullPolicy }}

deployments/charts/service/templates/worker.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ spec:
7676
runAsUser: 1001
7777
ports:
7878
- name: metrics
79-
containerPort: 9464
79+
containerPort: 9465
8080
imagePullPolicy: {{ .Values.services.worker.imagePullPolicy }}
8181
args:
8282
- --metrics_otel_collector_component

deployments/charts/web-ui/templates/_sidecar-helpers.tpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ OAuth2 Proxy sidecar container
169169
ports:
170170
- name: http
171171
containerPort: {{ .Values.sidecars.oauth2Proxy.httpPort }}
172-
- name: metrics
172+
- name: oauth2-metrics
173173
containerPort: {{ .Values.sidecars.oauth2Proxy.metricsPort }}
174174
livenessProbe:
175175
httpGet:

run/start_backend_bazel.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,6 @@ def _start_backend_operator(service_type: Literal['listener', 'worker'], emoji:
123123

124124
host_ip = get_host_ip()
125125

126-
metrics_port = '9467' if service_type == 'listener' else '9468'
127126
cmd = [
128127
'bazel', 'run', f'@osmo_workspace//src/operator:{service_name}',
129128
'--',
@@ -132,8 +131,7 @@ def _start_backend_operator(service_type: Literal['listener', 'worker'], emoji:
132131
'--backend', 'default',
133132
'--namespace', 'default',
134133
'--username', 'testuser',
135-
'--progress_folder_path', '/tmp/osmo/operator',
136-
'--metrics_prometheus_port', metrics_port
134+
'--progress_folder_path', '/tmp/osmo/operator'
137135
]
138136

139137
process = run_command_with_logging(

run/start_service_bazel.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -315,8 +315,7 @@ def _start_service_worker():
315315
'bazel', 'run', '@osmo_workspace//src/service/worker:worker_binary',
316316
'--',
317317
'--method=dev',
318-
'--progress_file', '/tmp/osmo/service/last_progress_worker',
319-
'--metrics_prometheus_port', '9465'
318+
'--progress_file', '/tmp/osmo/service/last_progress_worker'
320319
]
321320

322321
process = run_command_with_logging(
@@ -388,8 +387,7 @@ def _start_delayed_job_monitor():
388387
'@osmo_workspace//src/service/delayed_job_monitor:delayed_job_monitor_binary',
389388
'--',
390389
'--method=dev',
391-
'--progress_file', '/tmp/osmo/service/last_progress_delayed_job_monitor',
392-
'--metrics_prometheus_port', '9466'
390+
'--progress_file', '/tmp/osmo/service/last_progress_delayed_job_monitor'
393391
]
394392

395393
process = run_command_with_logging(

0 commit comments

Comments
 (0)