Traefik httproute - Troubleshoot dashboard access

[nopunYntendeed]

Hi, i am having trouble getting access to the dashboard.
I've setup a small repo that reproduces the issue: https://gitlab.com/MigTex/argoworkflow-troubleshoot

More context:

argo route

$ kubectl -n argo describe httproutes.gateway.networking.k8s.io argo-httproute 
Name:         argo-httproute
Namespace:    argo
Labels:       <none>
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1
Kind:         HTTPRoute
Metadata:
  Creation Timestamp:  2025-04-05T15:51:24Z
  Generation:          1
  Resource Version:    17359
  UID:                 4c9cbe2b-6141-4c21-91ed-5bea59f30644
Spec:
  Hostnames:
    mysite.localhost.traefik.me
  Parent Refs:
    Group:      gateway.networking.k8s.io
    Kind:       Gateway
    Name:       traefik-gateway
    Namespace:  kube-system
  Rules:
    Backend Refs:
      Group:   
      Kind:    Service
      Name:    argo-server
      Port:    2746
      Weight:  1
    Matches:
      Path:
        Type:   PathPrefix
        Value:  /argo
Status:
  Parents:
    Conditions:
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
    Controller Name:         traefik.io/gateway-controller
    Parent Ref:
      Group:      gateway.networking.k8s.io
      Kind:       Gateway
      Name:       traefik-gateway
      Namespace:  kube-system
     

argo service

$ kubectl -n argo describe svc argo-server
Name:                     argo-server
Namespace:                argo
Labels:                   <none>
Annotations:              <none>
Selector:                 app=argo-server
Type:                     ClusterIP
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.43.236.16
IPs:                      10.43.236.16
Port:                     web  2746/TCP
TargetPort:               2746/TCP
Endpoints:                10.42.0.113:2746
Session Affinity:         None
Internal Traffic Policy:  Cluster
Events:                   <none>

argo deployment

$ kubectl -n argo describe deployment argo-server
Name:                   argo-server
Namespace:              argo
CreationTimestamp:      Sat, 05 Apr 2025 09:22:40 +0100
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision: 5
Selector:               app=argo-server
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           app=argo-server
  Service Account:  argo-server
  Containers:
   argo-server:
    Image:      quay.io/argoproj/argocli:v3.6.5
    Port:       2746/TCP
    Host Port:  0/TCP
    Args:
      server
      --namespaced
      --auth-mode
      server
      --auth-mode
      client
    Readiness:  http-get https://:2746/ delay=10s timeout=1s period=20s #success=1 #failure=3
    Environment:
      ARGO_BASE_HREF:  /argo/
    Mounts:
      /tmp from tmp (rw)
  Volumes:
   tmp:
    Type:          EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:        
    SizeLimit:     <unset>
  Node-Selectors:  kubernetes.io/os=linux
  Tolerations:     <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  argo-server-54cc75567 (0/0 replicas created), argo-server-7c6f8b4cc5 (0/0 replicas created)
NewReplicaSet:   argo-server-75f9fdf448 (1/1 replicas created)
Events:
  Type    Reason             Age                 From                   Message
  ----    ------             ----                ----                   -------
  Normal  ScalingReplicaSet  51m (x2 over 8h)    deployment-controller  Scaled up replica set argo-server-54cc75567 from 0 to 1
  Normal  ScalingReplicaSet  47m (x2 over 111m)  deployment-controller  Scaled up replica set argo-server-7c6f8b4cc5 from 0 to 1

gateway

 $ kubectl -n kube-system describe gateway
Name:         traefik-gateway
Namespace:    kube-system
Labels:       app.kubernetes.io/instance=traefik-kube-system
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=traefik
              helm.sh/chart=traefik-34.2.1_up34.2.0
Annotations:  cert-manager.io/cluster-issuer: selfsigned-issuer
              meta.helm.sh/release-name: traefik
              meta.helm.sh/release-namespace: kube-system
API Version:  gateway.networking.k8s.io/v1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2025-04-05T08:27:08Z
  Generation:          1
  Resource Version:    17360
  UID:                 e2937a39-1017-43a2-83f2-e8ba06a4c0d3
Spec:
  Gateway Class Name:  traefik
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  All
    Hostname:  openwebui.localhost.traefik.me
    Name:      https-openwebui
    Port:      8443
    Protocol:  HTTPS
    Tls:
      Certificate Refs:
        Group:  
        Kind:   Secret
        Name:   selfsigned-tls
      Mode:     Terminate
    Allowed Routes:
      Namespaces:
        From:  Same
    Name:      web
    Port:      8000
    Protocol:  HTTP
    Allowed Routes:
      Namespaces:
        From:  All
    Hostname:  mysite.localhost.traefik.me
    Name:      websecure
    Port:      8443
    Protocol:  HTTPS
    Tls:
      Certificate Refs:
        Group:  
        Kind:   Secret
        Name:   selfsigned-tls
      Mode:     Terminate
Status:
  Addresses:
    Type:   IPAddress
    Value:  192.168.20.72
  Conditions:
    Last Transition Time:  2025-04-05T15:51:24Z
    Message:               Gateway successfully scheduled
    Observed Generation:   1
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2025-04-05T15:51:24Z
    Message:               Gateway successfully scheduled
    Observed Generation:   1
    Reason:                Programmed
    Status:                True
    Type:                  Programmed
  Listeners:
    Attached Routes:  2
    Conditions:
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Name:                    https-openwebui
    Supported Kinds:
      Group:          gateway.networking.k8s.io
      Kind:           HTTPRoute
      Group:          gateway.networking.k8s.io
      Kind:           GRPCRoute
    Attached Routes:  0
    Conditions:
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Name:                    web
    Supported Kinds:
      Group:          gateway.networking.k8s.io
      Kind:           HTTPRoute
      Group:          gateway.networking.k8s.io
      Kind:           GRPCRoute
    Attached Routes:  2
    Conditions:
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-04-05T15:51:24Z
      Message:               No error found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Name:                    websecure
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute
      Group:  gateway.networking.k8s.io
      Kind:   GRPCRoute

Issue

$ kubectl -n kube-system logs traefik-779fdb5f5c-7nb9x 
[05/Apr/2025:16:12:17 +0000] "GET /argo/ HTTP/2.0" 502 11 "-" "-" 2391 "httproute-argo-argo-httproute-gw-kube-system-traefik-gateway-ep-websecure-0-033312df303987759e81@kubernetesgateway" "http://10.42.0.113:2746" 0ms
2025-04-05T16:12:17Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:207 > Service selected by WRR: argo-argo-server-http-2746
2025-04-05T16:12:17Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:207 > Service selected by WRR: f8e1f573ae5511ac
2025-04-05T16:12:17Z DBG github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:117 > 502 Bad Gateway error="read tcp 10.42.0.1:54254->10.42.0.113:2746: read: connection reset by peer"

I have read through https://argo-workflows.readthedocs.io/en/latest/argo-server/, tried with ingress(nothing happens) and ingressRoute(same behaviour)

I'm kinda new and I'm sure I'm doing something dumb, sorry if its a daft question. Any help is appreciated, this is driving me crazy

[nopunYntendeed]

Bumping for aid

[nopunYntendeed]

still asking for helps