Rename RunUnitTests.ps1 RunPesterTests.ps1 to align with their purpose and Keep copy of ModifiedProviderSettingsExport to reference during development

[tkol2022]

🗣 Description

Under the Better Coding Act of 2026 I chartered a quick mini-task to improve the names of the scripts RunUnitTests.ps1 and RunPesterTests.ps1. These scripts are now dubbed RunRegoUnitTests.ps1 and RunPowerShellUnitTests.ps1 so that developers have a better clue on what they are for. I also deleted a phantom script RunFunctionalTests.ps1 since nobody uses it. I moved RunPowerShellUnitTests.ps1 into the ./Testing folder since that script is meant for developer use and it makes sense to put developer scripts in a standard folder instead of deep inside the ScubaGear folder tree.

Closes #1989

Added 2/27/2026
Under the Scope Creep Renewal Act of 2026 I decided to add an important fix to the Products.Tests.ps1 file used by the functional test orchestrator whenever scenarios invoke the ScubaCached test type. This particular functional test type is used extensively by the Entra Id test plan and in smaller quantities in the Defender, EXO and PowerPlatform test plans.
The ScubaCached orchestration code had a defect where it deletes the ModifiedProviderSettingsExport file and this prevents a developer from seeing the file that a specific test scenario creates when developing test scenarios. To rectify this I added code that saves a copy of the ModifiedProviderSettingsExport file created by each test scenario right before the file gets deleted.

💭 Motivation and context

Under BCA2026 developers want to spend less time trying to figure out what a specific script is used for, particularly when the script is helpful and commonly used for unit testing their changes.

🧪 Testing Instructions

Use the sample code below to ensure that the scripts work. Execute these from the root ScubaGear folder.

.\Testing\RunRegoUnitTests.ps1 -p aad

.\Testing\RunPowerShellUnitTests.ps1

✅ Pre-approval checklist

• This PR has an informative and human-readable title.
• PR targets the correct parent branch (e.g., main or release-name) for merge.
• Changes are limited to a single goal - eschew scope creep!
• Changes are sized such that they do not touch excessive number of files.
• All future TODOs are captured in issues, which are referenced in code comments.
• These code changes follow the ScubaGear content style guide.
• Related issues these changes resolve are linked preferably via closing keywords.
• All relevant type-of-change labels added.
• All relevant project fields are set.
• All relevant repo and/or project documentation updated to reflect these changes.
• Unit tests added/updated to cover PowerShell and Rego changes.
• Functional tests added/updated to cover PowerShell and Rego changes.
• All relevant functional tests passed.
• All automated checks (e.g., linting, static analysis, unit/smoke tests) passed.

✅ Pre-merge checklist

• PR passed smoke test check.

• Feature branch has been rebased against changes from parent branch, as needed

  Use Rebase branch button below or use this reference to rebase from the command line.

• Resolved all merge conflicts on branch

• Notified merge coordinator that PR is ready for merge via comment mention

• Demonstrate changes to the team for questions and comments.
  (Note: Only required for issues of size Medium or larger)

✅ Post-merge checklist

• Feature branch deleted after merge to clean up repository.
• Verified that all checks pass on parent branch (e.g., main or release-name) after merge.

[dagarwal-ecs]

One very small update, besides that it looks good

[dagarwal-ecs]

Looks good from my end

[DickTracyII]

When I ran the .\RunPowerShellUnitTests.ps1. I got some failures. but I am sure this in not related to renaming of the script

[DickTracyII]

i ran through both scripts. I like the new names

[tkol2022]

@DickTracyII @dagarwal-ecs I am about to push the added enhancement to the functional test orchestrator and update the PR with new test instructions. I will do an official request for re-review when I finish.

[tkol2022]

Testing instructions for the enhancement that keeps a copy of ModifiedProviderSettingsExport.json

• Follow the instructions at this link and create a script that runs the functional test orchestrator. If you already have a script that is fine too.
• Start by testing a single test case for Entra Id to make sure the code works before you expand to testing multiple test cases. Use the Filter.

• Once Pester finishes running the filtered test scenarios, make sure they Pass at the end of the Pester run. Then go to the output folder and sort the files by name. You should see multiple copies of ModifiedProviderSettingsExport.json with unique file names.

• Ensure there are no unexpected errors during this process.

Examine the output of a single test scenario and verify that you can see the changes in the saved copy of the ModifiedProviderSettingsExport file.

• Execute the functional tests but filter for the policy MS.AAD.3.3v2.

• Go to the output folder and sort by file name. Open the copied ModifiedProviderSettingsExport json file that is first in the list.

• Go to the authentication_method_feature_settings section in the JSON file and find the configuration for Id: MicrosoftAuthenticator.

• Verify that MicrosoftAuthenticator is disabled.

This proves that the orchestrator saved a copy of the configuration created by the first test scenario "MS.AAD.3.3v2 Not-applicable case - MS Auth Disabled"

Expand to try out more test cases.

• Try some of the test cases from Defender, EXO, PowerPlatform, SharePoint but Filter for the test scenarios that use ScubaCached in those test plans.

Final boss test

• To ensure that the functional test plans work on a broader scale run the branch against the GitHub action named "Functional Tests". This will run the code to ensure my new code works against all tenant license types.

[tkol2022]

@DickTracyII @dagarwal-ecs I added a description of the new scope at the top of the issue to save you time looking at file diffs and I added custom test instructions to make your life easier. Feel free to do any tests outside of what I put there and I welcome any questions about the code or design. Thanks for your flexibility. Go for it.

[DickTracyII]

@tkol2022 I ran the functional tests like you stated except I include all AAD policies. I did get alot of runtime exceptions...not sure why but it did produce several MOdifiedProviderSettingsExport json files.

I'll run more tests next week.

[tkol2022]

@tkol2022 I ran the functional tests like you stated except I include all AAD policies. I did get alot of runtime exceptions...not sure why but it did produce several MOdifiedProviderSettingsExport json files.

I'll run more tests next week.

Thanks! Looks like the errors are on GCCHigh? Which specific test scenarios are failing so I can reproduce?

Did you get any errors on GCC, E5 or G3?

@dagarwal-ecs Did you get any errors?

[DickTracyII]

Yes the errors were on gcchigh but I think it was due to my PesterContianer configuration. I didn't receive the errors on any of the tenant after i rebuild my configurations; I had ``TenantDomainasTenantId`. I also specified `gcch` and not `gcchigh`

[DickTracyII]

Everything tested out fine for GCCHIGH, G3, E5 and functional tests passed!