[Bug] Expect-check creates a endless loop of patching

[ctschacher]

chainsaw version Version

v0.2.12

Description

When I use the following in my test:

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: v4-wrong-management-context
spec:
  steps:
  - name: apply config
    try:
    - apply:
        file: config.yaml

Everything looks normal in the test output:

    | 11:31:27 | v4-wrong-management-context | @chainsaw    | CREATE    | OK    | v1/Namespace @ chainsaw-frank-bullfrog
    | 11:31:27 | v4-wrong-management-context | apply config | TRY       | BEGIN |
    | 11:31:27 | v4-wrong-management-context | apply config | APPLY     | RUN   | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:31:27 | v4-wrong-management-context | apply config | CREATE    | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:31:27 | v4-wrong-management-context | apply config | APPLY     | DONE  | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:31:27 | v4-wrong-management-context | apply config | TRY       | END   |
    | 11:31:27 | v4-wrong-management-context | apply config | CLEANUP   | BEGIN |
    | 11:31:27 | v4-wrong-management-context | apply config | DELETE    | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:31:27 | v4-wrong-management-context | apply config | CLEANUP   | END   |
    | 11:31:27 | v4-wrong-management-context | @chainsaw    | CLEANUP   | BEGIN |
    | 11:31:27 | v4-wrong-management-context | @chainsaw    | DELETE    | OK    | v1/Namespace @ chainsaw-frank-bullfrog
    | 11:31:32 | v4-wrong-management-context | @chainsaw    | CLEANUP   | END   |

But when I add an expect in the apply section of the test:

spec:
  steps:
  - name: apply config
    try:
    - apply:
        file: config.yaml
        expect:
          - check:
              ($error != null): true

Then it's going crazy with an endless stream of PATCH actions:

    | 11:32:06 | v4-wrong-management-context | @chainsaw    | CREATE    | OK    | v1/Namespace @ chainsaw-still-hornet
    | 11:32:06 | v4-wrong-management-context | apply config | TRY       | BEGIN |
    | 11:32:06 | v4-wrong-management-context | apply config | APPLY     | RUN   | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | CREATE    | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
    | 11:32:06 | v4-wrong-management-context | apply config | PATCH     | OK    | gravitee.io/v1alpha1/ManagementContext @ default/bad-credentials-ctx
[...]

Something seems off here.

Steps to reproduce

1. Apply a YAML file
2. Add expect with a check

Expected behavior

Well, a working check

Screenshots

No response

Logs

Slack discussion

No response

Troubleshooting

• I have searched other issues in this repository and mine is not recorded.

[yonkatian]

Hi
I am also encountering this issue on kyverno 1.16 and Chainsaw 0.2.14 . Which I had put inside the kyverno-chainsaw slack channel
slack message

I had something similar.

• I had a Validating Policy
• In my chainsaw-test I try to apply a resource that violate the validating policy

When the test run I would had the similar output with him.
Oddly enough if i add a match. I would see the kyverno error message

admission webhook "vpol.validate.kyverno.svc-fail" denied the request: Policy dev-ingressclass failed: Please set your ingressClass to be 'dev'

Would be great if someone could pick this up. 😃

Thanks