[SURE-11131] Allow configuring Requests/Limits via Helm for Rancher components to avoid blocks by customer policy engines (OPA) and improve stability

[kkaempf]

SURE-11131

Business rationale

Several clients enforce governance policies that block or flag Pods without resource configurations.
Currently, these Rancher-installed Pods are failing those compliance checks

Request description:

This goal can be achieved through two potential approaches:

1. Hardcode default Requests/Limits for all Pods created during the Rancher Helm chart installation.
2. Expose configuration options in the Helm chart, allowing users to define custom Requests/Limits via values.

Currently, the Helm chart only allows resource configuration for the core Rancher Pod. This should be extended to other components, such as:

• fleet-controller

Engineering considerations

This would mean:

• exposing chart values for requests and resources (AFAICT Fleet already has this, at least on main and release/v0.14)
• documenting how to set them on the Rancher side, since Rancher already has the ability to install Fleet with arbitrary Helm values, and ensuring that whatever values (or defaults) set at Rancher level are propagated to Fleet.