Something is broken in your `create_fn` (K8s version 1.28.2)

[tgrushka]

Installed with:
helm install clustersecret clustersecret/cluster-secret -n clustersecret --create-namespace

Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2

Kind: ClusterSecret
apiVersion: clustersecret.io/v1
metadata:
  name: postgres-secret
data:
  PGUSERNAME: base64 data...
  PGPASSWORD: base64 data...
  PGDATABASE: base64 data...
  PGHOST: base64 data...
  PGPORT: base64 data...

kubectl describe clustersecret
 
Name:         postgres-secret
Namespace:    default
Labels:       <none>
Annotations:  kopf.zalando.org/create_fn:
                {"started":"2024-02-08T21:04:51.408162","delayed":"2024-02-08T21:08:52.152167","purpose":"create","retries":4,"success":false,"failure":fa...
              kopf.zalando.org/on_field_data.data:
                {"started":"2024-02-08T21:04:51.408105","stopped":"2024-02-08T21:04:51.409285","purpose":"create","retries":1,"success":true,"failure":fal...
API Version:  clustersecret.io/v1
Data:
  PGDATABASE:  base64...
  PGHOST:      base64...
  PGPASSWORD:  base64...
  PGPORT:      base64...
  PGUSERNAME:  base64...
Kind:          ClusterSecret
Metadata:
  Creation Timestamp:  2024-02-08T21:04:51Z
  Finalizers:
    kopf.zalando.org/KopfFinalizerMarker
  Generation:        9
  Resource Version:  30606956
  UID:               8615596a-d289-45cf-a792-83d24428c60d
Status:
  Kopf:
    Progress:
      create_fn:
        Delayed:  2024-02-08T21:08:52.152167
        Failure:  false
        Message:  nothing to repeat at position 0
        Purpose:  create
        Retries:  4
        Started:  2024-02-08T21:04:51.408162
        Success:  false
      on_field_data/data:
        Failure:  false
        Purpose:  create
        Retries:  1
        Started:  2024-02-08T21:04:51.408105
        Stopped:  2024-02-08T21:04:51.409285
        Success:  true
Events:
  Type    Reason   Age    From  Message
  ----    ------   ----   ----  -------
  Normal  Logging  3m11s  kopf  Handler 'on_field_data/data' succeeded.
  Error   Logging  3m11s  kopf  Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 283, in execute_handler_once
    result = await invoke_handler(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 378, in invoke_handler
    result = await invocation.invoke(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/invocation.py", line 117, in invoke
    result = await ...usr/local/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.9/sre_parse.py", line 668, in _parse
    raise source.error("nothing to repeat",
re.error: nothing to repeat at position 0
  Error  Logging  2m11s  kopf  Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 283, in execute_handler_once
    result = await invoke_handler(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 378, in invoke_handler
    result = await invocation.invoke(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/invocation.py", line 117, in invoke
    result = await ...usr/local/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.9/sre_parse.py", line 668, in _parse
    raise source.error("nothing to repeat",
re.error: nothing to repeat at position 0
  Error  Logging  70s  kopf  Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 283, in execute_handler_once
    result = await invoke_handler(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 378, in invoke_handler
    result = await invocation.invoke(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/invocation.py", line 117, in invoke
    result = await ...usr/local/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.9/sre_parse.py", line 668, in _parse
    raise source.error("nothing to repeat",
re.error: nothing to repeat at position 0
  Error  Logging  10s  kopf  Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 283, in execute_handler_once
    result = await invoke_handler(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 378, in invoke_handler
    result = await invocation.invoke(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/invocation.py", line 117, in invoke
    result = await ...usr/local/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.9/sre_parse.py", line 668, in _parse
    raise source.error("nothing to repeat",
re.error: nothing to repeat at position 0

[heyzling]

Caught the same error on chart version '0.2.1' (as in docs).
Can be fixed by setting clustersecret.clustersecret.image.tag="0.0.10" in Helm values.

[wangchaoforever]

@tgrushka I also have this issue. I try to add matchNamespace in my yaml file and it works fine.But i don't know why.
So you can have a try to add matchNamespace in your yaml file. It will looks like:

Kind: ClusterSecret
apiVersion: clustersecret.io/v1
metadata:
  name: postgres-secret
matchNamespace:
  - '.*'
data:
  PGUSERNAME: base64 data...
  PGPASSWORD: base64 data...
  PGDATABASE: base64 data...
  PGHOST: base64 data...
  PGPORT: base64 data...

[ForbiddenEra]

Caught the same error on chart version '0.2.1' (as in docs). Can be fixed by setting clustersecret.clustersecret.image.tag="0.0.10" in Helm values.

Thanks for this!! I still had to redeploy it but working again.

Oddly, at first, I was also getting permission errors that it didn't have get permission for clustersecrets which I manually added but after re-deploying that wasn't needed..?

[sylvaindd]

Same problem here, can't seem to find a way to fix it.

[ForbiddenEra]

Same problem here, can't seem to find a way to fix it.

I had some issues with it myself.

One being, as I mentioned, I was seeing an error about it not having get permissions for clustersecrets. I can confirm in normal operation that this permission doesn't seem to be needed, but when things get messed, it does.

In order to fully resolve the issue, I did the following:

• updated the image version in my deployment as per @heyzling's suggestion
• removed all secrets created by clustersecret (basically just did kubectl get secret -A | grep <secretname> and then manually did kubectl delete secret -n <namespace> <secretname> for each)
• removed all clustersecret related objects (eg. uninstalled it) and reinstalled it (personally, I have it deployed w/agrocd in an ApplicationSet, so simply deleted the Application resource and it instantly repopulated/reinstalled)

The only other issue I had was that I had accidentally added default as one of the namespaces to replicate to, but the secret I was replicating was also coming from default, thus the original secret seemed to have been overwritten; I had to delete and re-create that secret as well in the process while ensuring that default wasn't in the list of namespaces to replicate to, but of course this is an issue unrelated to the core issue here but is something that I hadn't noticed before - and hadn't caused issues - but now was causing an issue.

Once I'd taken all those actions, I hadn't had any other issues and it seems to be working.

That said, if the newer image fixes things than anything that references that image version such as the helm chart should be updated.

I'm also curious as to the permissions issue that I saw; while perhaps the get permission isn't seemingly needed in normal operation, there's obviously code that uses it in some cases. I'm curious what those cases are and if either those cases need to be updated or the permissions need to be updated to include that. As the target in question is the clustersecret CRD, if get is needed, I don't see it being much of an issue.

Anyway, hopefully that helps you get up and running at least!

[sylvaindd]

Removed everything, upgraded to 0.0.10 and it is not working.

Thanks

[dart-mtucker]

We are also seeing this problem with kubernetes ver. 1.28.8 managed with RKE. I have a 4 node cluster which was recently built cleanly and we're still getting this python exception in create_fn. I'm deploying clustersecrets using the kubectl command: kubectl apply -f ./yaml using the "latest" image from the quay image registry as well as a locally built image using ver. 0.4.1.
Once things are in this state, the deployment cannot be removed until all of the clusterSecret objects are deleted, usually requiring patching the object live to remove the finalizers. Very unreliable behavior due to these issues.

Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.3/v/string_type
[2024-05-02 14:08:45,914] kopf.activities.star [INFO ] Found 1 existing cluster secrets.
[2024-05-02 14:08:45,915] kopf.activities.star [ERROR ] Activity 'startup_fn' failed with an exception. Will retry.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 283, in execute_handler_once
result = await invoke_handler(
File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/execution.py", line 378, in invoke_handler
result = await invocation.invoke(
File "/usr/local/lib/python3.9/site-packages/kopf/_core/actions/invocation.py", line 117, in invoke
result = await fn(**kwargs) # type: ignore
File "/src/handlers.py", line 243, in startup_fn
BaseClusterSecret(
File "/usr/local/lib/python3.9/site-packages/pydantic/main.py", line 165, in init
pydantic_self.pydantic_validator.validate_python(data, self_instance=pydantic_self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for BaseClusterSecret
namespace
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.3/v/string_type