Flaky test in the `unittests` job

[srm09]

Bug description
Seen this test fail multiple times when running the unit tests job

• Failure [13.309 seconds]
ClusterBootstrap Reconciler
/home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:42
  when cluster is created from clusterBootstrapTemplate
  /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:781
    from a ClusterBootstrapTemplate
    /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:787
      should perform ClusterBootstrap reconciliation & block reconciliation if ClusterBootstrap gets paused [It]
      /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:788

      Expected
          <bool>: true
      not to be true
      /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:914

Affected product area (please put an X in all that apply)

• ( ) APIs
• ( ) Addons
• ( ) CLI
• ( ) Docs
• ( ) IAM
• ( ) Installation
• ( ) Plugin
• ( ) Security
• (x) Test and Release
• ( ) User Experience
• ( ) Developer Experience

Expected behavior

[srm09]

An example for this one https://github.com/vmware-tanzu/tanzu-framework/actions/runs/4196558719/jobs/7277690816#step:10:2519

[srm09]

Rewriting this assertion https://github.com/vmware-tanzu/tanzu-framework/blame/bd3036ea658d6b7b0fc394ed796c98274b56bdb4/addons/controllers/clusterbootstrap_controller_test.go#L908-L916 might be helpful

// Wait 10 seconds in case reconciliation happens
// time.Sleep(10 * time.Second) <========= Remove this

Eventually(func() bool {
	remoteSecret := &corev1.Secret{}
	if err := k8sClient.Get(ctx, client.ObjectKey{Namespace: constants.TKGSystemNS, Name: util.GenerateDataValueSecretName(clusterName, foobar1CarvelPackageRefName)}, remoteSecret); err != nil {
		return false
	}
	// values.yaml should not update
	if string(s.Data["values.yaml"]) == string(remoteSecret.Data["values.yaml"]) {
		return false
	}
	return true
}, waitTimeout, pollingInterval).Should(BeTrue())// Wait 10 seconds in case reconciliation happens

only to confirm if the above assertion is what the original intention was

Note No need for the timeout since Eventually will keep retrying until the assertion is met or the timeout occurs whichever is first.