Inject overlay only working for transformers category

[nbaker-ally]

Viya4 Deployment Version Details

8.2.0

Ansible Variable File Details

No response

Steps to Reproduce

I'm attempting to inject overlays into the base kustomization file for the MRM solution using inject-sas-bases-overlays.yaml and the following entries:

resources:
  - sas-bases/overlays/postgres/cds-postgres
components:
  - sas-bases/components/crunchydata/internal-cds-postgres
transformers:
  - site-config/sas-risk-cirrus-core/resources/core_transform.yaml
  - site-config/sas-risk-cirrus-mrm/resources/mrm_transform.yaml

The transformers are being injected into kustomization.yaml, but the resources and components categories are not.

Expected Behavior

The resources and components entries need to be injected into the base kustomization.yaml file.

Actual Behavior

As a result the resources and components entires are not being injected into the base kustomization file and MRM is not appearing after logging into the Viya platform.

Additional Context

No response

References

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[ajeffowens]

Hi @nbaker-ally, this sounds similar to 634. I have opened a PR that may address it

[ajeffowens]

#640

[nbaker-ally]

Hi Jeff, thanks. I incorporated the contents of your PR and tried again. It still only seems to be injecting the transformers and ignoring the components and resources. I'm surprised it's not loading the resource at least, I would have expected it to even before easing the restriction from sas-bases/overlays to sas-bases/

[ajeffowens]

How are you using the inject-sas-bases-overlays feature? There shouldn't be any site-config in there. If you have kustomizations for site-config, you just put the files in your site-config and viya4-deployment already populates kustomization.yaml accordingly. I suspect that is how you are ending up with the refs in your generated kustomization from your screenshot.

If you want to insert refs to sas-bases overlays in kustomization, you build an inject-sas-bases-overlays file much like the one you have (just minus the transformers since they are in site-config), and then you put that file into a site-config folder itself. When viya4-deployment encounters this file, it inserts the sas-bases ref into kustomization. Did you put your inject-sas-bases-overlays file into a site-config dir?

Lastly, for cds, you shouldn't need to be using inject-sas-bases-overlays. Just specify you want cds-postgres within ansible-vars and viya4-deployment will take care of those refs for you: https://github.com/sassoftware/viya4-deployment/blob/80a98bfc98b83b3358178852c05d10ff8cba1f49/docs/CONFIG-VARS.md#postgresql

[nbaker-ally]

Thank you for the clarification Jeff. The documentation I was following mentioned to edit the base kustomization file directly so it seemed logical. I have followed your guidance and my base kustomization file is populating correctly.

Now I believe that my variables in mrm_transform.yaml are breaking when being passed in during deployment. At least I think that's what I'm discerning based on the logs from the cirrusrcc and cirrusmrm pods. Variables were determined based on the MRM install documentation.

sasriskcirrusmrm.log
sasriskcirrusrcc.log

mrm_transform:

apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-parameter-transform
patch: |-

  #Cirrus deployer environment variables
  - op: replace
    path: /data/SAS_LOG_LEVEL_RISKCIRRUSDEPLOYER
    value: "INFO"
  # For Internal SAS Demo environments,set this value to a empty string. 
  # For customer environments, set this value to load_sample_data(no sample date will be loaded).
  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_SKIP_SPECIFIC_INSTALL_STEPS
    value: "load_sample_data"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_INCLUDE_SAMPLES
    value: "N"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_RUN_SPECIFIC_INSTALL_STEPS
    value: ""

  #Cirrus solution builder environment variables
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_PREFIX
    value: "mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_USER
    value: "nbaker-ally"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_GIT_REPO_FULL_URL
    value: "https://github.com/nbaker-ally/mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_SOLUTION_BRANCH_PREFIX
    value: "DEV"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_APPEND_NAMESPACE_TO_BRANCH_FLG
    value: "Y"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_HOST
    value: "github.com"

target:
  kind: ConfigMap
  name: sas-risk-cirrus-mrm-parameters*

---
apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-secret-transform
patch: |-

  #Cirrus solution builder secrets
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_TOKEN
    value: "XXXXX

target:
  kind: Secret
  name: sas-risk-cirrus-mrm-secret*

[nbaker-ally]

Thank you for the clarification Jeff. The documentation I was following mentioned to edit the base kustomization file directly so it seemed logical. I have followed your guidance and my base kustomization file is populating correctly.

Now I believe that my variables in mrm_transform.yaml are breaking when being passed in during deployment. At least I think that's what I'm discerning based on the logs from the cirrusrcc and cirrusmrm pods. Variables were determined based on the MRM install documentation.

sasriskcirrusmrm.log sasriskcirrusrcc.log

mrm_transform:

apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-parameter-transform
patch: |-

  #Cirrus deployer environment variables
  - op: replace
    path: /data/SAS_LOG_LEVEL_RISKCIRRUSDEPLOYER
    value: "INFO"
  # For Internal SAS Demo environments,set this value to a empty string. 
  # For customer environments, set this value to load_sample_data(no sample date will be loaded).
  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_SKIP_SPECIFIC_INSTALL_STEPS
    value: "load_sample_data"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_INCLUDE_SAMPLES
    value: "N"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_RUN_SPECIFIC_INSTALL_STEPS
    value: ""

  #Cirrus solution builder environment variables
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_PREFIX
    value: "mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_USER
    value: "nbaker-ally"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_GIT_REPO_FULL_URL
    value: "https://github.com/nbaker-ally/mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_SOLUTION_BRANCH_PREFIX
    value: "DEV"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_APPEND_NAMESPACE_TO_BRANCH_FLG
    value: "Y"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_HOST
    value: "github.com"

target:
  kind: ConfigMap
  name: sas-risk-cirrus-mrm-parameters*

---
apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-secret-transform
patch: |-

  #Cirrus solution builder secrets
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_TOKEN
    value: "XXXXX

target:
  kind: Secret
  name: sas-risk-cirrus-mrm-secret*

Any guidance on this? Thanks!

[saschjmil]

@nbaker-ally, Sorry for the delay. I spoke with someone on the Risk Cirrus team about this. From my understanding, if RCC fails to come up, MRM will fail as well. Could you share what changes you made to the RCC transform and which cadence you are using?

Overall, this seems like an issue with the Risk product rather than the viya4-deployment process.

[nbaker-ally]

Thanks Chris! I'll see what I can do with this information. We're using 2024.09 LTS and the following customizations were made.

Core transform:

apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-core-parameter-transform
patch: |-

  #Cirrus deployer environment variables
  - op: replace
    path: /data/SAS_LOG_LEVEL_RISKCIRRUSDEPLOYER
    value: "INFO"

  #Cirrus code library environment variables
  - op: replace
    path: /data/SAS_RISK_CIRRUS_CODE_LIB_GIT_SETUP_NEEDED
    value: "N"

target:
  kind: ConfigMap
  name: sas-risk-cirrus-core-parameters*

MRM transform:

apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-parameter-transform
patch: |-

  #Cirrus deployer environment variables
  - op: replace
    path: /data/SAS_LOG_LEVEL_RISKCIRRUSDEPLOYER
    value: "INFO"
  # For Internal SAS Demo environments,set this value to a empty string. 
  # For customer environments, set this value to load_sample_data(no sample date will be loaded).
  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_SKIP_SPECIFIC_INSTALL_STEPS
    value: ""

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_INCLUDE_SAMPLES
    value: "Y"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_DEPLOYER_RUN_SPECIFIC_INSTALL_STEPS
    value: ""

  #Cirrus solution builder environment variables
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_PREFIX
    value: "mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_USER
    value: "nbaker-ally"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_GIT_REPO_FULL_URL
    value: "https://github.com/nbaker-ally/mrmviya"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_SOLUTION_BRANCH_PREFIX
    value: "DEV"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_APPEND_NAMESPACE_TO_BRANCH_FLG
    value: "Y"

  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_HOST
    value: "github.com"

target:
  kind: ConfigMap
  name: sas-risk-cirrus-mrm-parameters*

---
apiVersion: builtin
kind: PatchTransformer
metadata:
  name: cirrus-mrm-secret-transform
patch: |-

  #Cirrus solution builder secrets
  - op: replace
    path: /data/SAS_RISK_CIRRUS_SOLUTION_BUILDER_REPO_TOKEN
    value: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

target:
  kind: Secret
  name: sas-risk-cirrus-mrm-secret*

[saschjmil]

I think your RCC transform might be missing some values. I would look into the RCC/CORE install documentation to get the required values like you did for MRM. Hopefully that'll get it working.

[github-actions]

This issue is stale because it has been open 30 days with no activity.