Skip to content

Iter8 controller doesn't work #1644

@paloma-rebuelta

Description

@paloma-rebuelta

Describe the bug

I am working on the mirroring trafic use case for modelmesh. I have deployed the traffic controller v0.17.1 as there is no mirroring usecase in the iter8 docs for later versions.
The controller resources get created, the pod spins up, but I get error logs relating to the metrics and A/B?n service and BadgerDB:

time=2023-09-22 12:03:02 level=error msg=cannot read config file: /config/metrics.yaml stack-trace=below ... 
::Trace:: open /config/metrics.yaml: no such file or directory
time=2023-09-22 12:03:02 level=error msg=unable to read metrics configuration: cannot read config file: /config/metrics.yaml
time=2023-09-22 12:03:02 level=error msg=count not start A/B/n metrics service
time=2023-09-22 12:03:02 level=error msg=Unable to configure metrics storage client cannot open BadgerDB
time=2023-09-22 12:03:02 level=error msg=cound not start A/B/n service

This doesn't kill the pod, its status is "Running".
When I deploy a candidate model and use the mirroring traffic template, the virtual service remains intact, it does not get altered based on the weights configmaps.

To Reproduce
the controller was deployed using this kustomization file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- 'https://github.com/iter8-tools/iter8.git/kustomize/controller/clusterScoped?ref=v0.17.1'

namespace: iter8

patches:
- patch: |-
    - op: replace
      path: /data/config.yaml
      value: |
        clusterScoped: true
        defaultResync: 15m
        image: iter8/iter8:0.17
        logLevel: debug
        resourceTypes:
          cm:
            Group: ""
            Resource: configmaps
            Version: v1
          deploy:
            Group: apps
            Resource: deployments
            Version: v1
            conditions:
            - name: Available
              status: "True"
          isvc:
            Group: serving.kserve.io
            Resource: inferenceservices
            Version: v1beta1
            conditions:
            - name: Ready
              status: "True"
          svc:
            Group: ""
            Resource: services
            Version: v1
          vs:
            Group: networking.istio.io
            Resource: virtualservices
            Version: v1alpha3
        storageClassName: ""
  target:
    kind: ConfigMap
    name: iter8

- patch: |-
    - op: replace
      path: /spec/storageClassName
      value: default
  target:
    kind: PersistentVolumeClaim
    name: iter8

Expected behavior
When I deploy a candidate model and the mirroring traffic templates, I would expect traffic to be mirrored to the second model. This does not happen as the virtual service does not get updated with the weights.

Desktop (please complete the following information):

  • OS: MacOS but running in kubernetes
  • Output of the iter8 version command: irrelevant as using controller

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions