Skip to content

fix: Script reporting error - #8

Merged
Affan2900 merged 1 commit into
mainfrom
fix/loadshed-script-reporting
Jul 26, 2026
Merged

fix: Script reporting error#8
Affan2900 merged 1 commit into
mainfrom
fix/loadshed-script-reporting

Conversation

@Affan2900

@Affan2900 Affan2900 commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • Bug Fixes

    • Improved load testing reliability with persistent HTTP connections and clearer error reporting.
    • Added explicit handling for successful, throttled, and failed analysis requests.
    • Improved replica monitoring and final test verdicts, including inconclusive results when replica data cannot be read.
    • Reduced unnecessary delays while polling job progress.
  • Chores

    • Removed outdated internal documentation from the API startup configuration.

@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The load-shed test now uses persistent HTTP requests, captures correlation headers, samples active revision replicas, conditionally sleeps while polling, and reports PASS, FAIL, or INCONCLUSIVE outcomes. An explanatory Kestrel request-size comment was removed without changing configuration.

Changes

Load-shed test execution

Layer / File(s) Summary
HTTP upload and analyze flow
scripts/loadshed-test.ps1
A persistent HttpClient uploads blobs and triggers /analyze, with explicit handling for success, HTTP 429, response headers, and failures.
Polling and verdict evaluation
scripts/loadshed-test.ps1
Polling reads active revision replica counts, tracks read failures and terminal jobs, conditionally sleeps, and computes the final verdict.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TestScript as loadshed-test.ps1
  participant BlobStorage as Blob storage
  participant AnalyzeEndpoint as /analyze endpoint
  participant AzureCLI as az containerapp revision list

  TestScript->>BlobStorage: PUT blob
  BlobStorage-->>TestScript: Upload response
  TestScript->>AnalyzeEndpoint: POST analyze request
  AnalyzeEndpoint-->>TestScript: Success, correlation ID, or HTTP 429
  TestScript->>AzureCLI: Read active revision replicas
  AzureCLI-->>TestScript: Replica count or read failure
  TestScript->>TestScript: Poll jobs and compute verdict
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing the loadshed script’s reporting behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/loadshed-script-reporting

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

Azure what-if (rg-cognilens-dev)

Show plan
Note: The result may contain false positive predictions (noise).
You can help us improve the accuracy of the result by opening an issue here: https://aka.ms/WhatIfIssues

Resource and property changes are indicated with these symbols:
  - Delete
  + Create
  ~ Modify
  = Nochange
  x Unsupported
  * Ignore
  x Noeffect

The deployment will update the following scopes:

Scope: /















Scope: /subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev

  + Microsoft.Resources/deploymentScripts/cognilens-dev-sql-64sgu7kbbhj5e-bootstrap-users [2023-08-01]

      apiVersion:                                                                                                                                                                                                               "2023-08-01"
      id:                                                                                                                                                                                                                       "/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Resources/deploymentScripts/cognilens-dev-sql-64sgu7kbbhj5e-bootstrap-users"
      identity.type:                                                                                                                                                                                                            "UserAssigned"
      identity.userAssignedIdentities./subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-sql-64sgu7kbbhj5e-deploy-mi: {}
      kind:                                                                                                                                                                                                                     "AzureCLI"
      location:                                                                                                                                                                                                                 "centralus"
      name:                                                                                                                                                                                                                     "cognilens-dev-sql-64sgu7kbbhj5e-bootstrap-users"
      properties.azCliVersion:                                                                                                                                                                                                  "2.65.0"
      properties.cleanupPreference:                                                                                                                                                                                             "OnSuccess"
      properties.environmentVariables: [
        0:

          name:  "SERVER"
          value: "cognilens-dev-sql-64sgu7kbbhj5e.database.windows.net"

        1:

          name:  "DATABASE"
          value: "CogniLens"

        2:

          name:  "API_IDENTITY_NAME"
          value: "cognilens-dev-api-mi"

        3:

          name:  "WORKER_IDENTITY_NAME"
          value: "cognilens-dev-worker-mi"

        4:

          name:  "API_IDENTITY_CLIENT_ID"
          value: "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').clientId]"

        5:

          name:  "WORKER_IDENTITY_CLIENT_ID"
          value: "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').clientId]"

        6:

          name:  "DEPLOY_CLIENT_ID"
          value: "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-sql-64sgu7kbbhj5e-deploy-mi', '2024-11-30').clientId]"

        7:

          name: "CICD_CLIENT_ID"

      ]
      properties.retentionInterval:                                                                                                                                                                                             "PT1H"
      properties.scriptContent:                                                                                                                                                                                                 "      set -e

      # mssql-tools is NOT installed from packages.microsoft.com here: the AzureCLI
      # deployment-script image is Azure Linux, which has no apt-get, so the Debian/Ubuntu repo
      # route fails outright. go-sqlcmd ships a single statically-linked binary instead, so this
      # depends on nothing in the base image beyond curl.
      curl -sSL -o sqlcmd.tar.bz2 https://github.com/microsoft/go-sqlcmd/releases/download/v1.10.0/sqlcmd-linux-amd64.tar.bz2
      # `tar -j` shells out to a bzip2 binary that isn't guaranteed to be present; Python's
      # tarfile decompresses bz2 in-process, so it works even on a stripped-down image.
      tar -xjf sqlcmd.tar.bz2 2>/dev/null || python3 -c "import tarfile; tarfile.open('sqlcmd.tar.bz2').extractall('.')"
      chmod +x ./sqlcmd

      # CREATE USER ... FROM EXTERNAL PROVIDER is deliberately NOT used here. That form makes the
      # SQL server call Microsoft Graph to resolve the identity name, which requires the server to
      # have a managed identity holding the Directory Readers role — granting that needs Privileged
      # Role Administrator in the tenant, which isn't available on this (university-managed) tenant.
      # Creating the user from the identity's client ID converted to a binary SID is the documented
      # equivalent and needs no directory permissions at all.
      cat <<SQL > bootstrap.sql
DECLARE @apiUser sysname = N'$API_IDENTITY_NAME';
DECLARE @workerUser sysname = N'$WORKER_IDENTITY_NAME';
DECLARE @apiSid varbinary(16) = CONVERT(varbinary(16), CAST(N'$API_IDENTITY_CLIENT_ID' AS uniqueidentifier));
DECLARE @workerSid varbinary(16) = CONVERT(varbinary(16), CAST(N'$WORKER_IDENTITY_CLIENT_ID' AS uniqueidentifier));
DECLARE @cmd nvarchar(max);

IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = @apiUser)
BEGIN
  SET @cmd = N'CREATE USER [' + @apiUser + N'] WITH SID = 0x' + CONVERT(varchar(100), @apiSid, 2) + N', TYPE = E;';
  EXEC(@cmd);
END
EXEC('ALTER ROLE db_datareader ADD MEMBER [' + @apiUser + ']');
EXEC('ALTER ROLE db_datawriter ADD MEMBER [' + @apiUser + ']');

IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = @workerUser)
BEGIN
  SET @cmd = N'CREATE USER [' + @workerUser + N'] WITH SID = 0x' + CONVERT(varchar(100), @workerSid, 2) + N', TYPE = E;';
  EXEC(@cmd);
END
EXEC('ALTER ROLE db_datareader ADD MEMBER [' + @workerUser + ']');
EXEC('ALTER ROLE db_datawriter ADD MEMBER [' + @workerUser + ']');
SQL

      # The CI/CD principal applies EF migrations from cd.yml, so it needs DDL rights that no
      # runtime identity should ever have. db_ddladmin covers CREATE/ALTER/DROP; the data roles
      # are for the SeedRubrics migration, which inserts rows. Deliberately not db_owner: the
      # pipeline has no business granting permissions or dropping users.
      if [ -n "$CICD_CLIENT_ID" ]; then
        cat <<SQL >> bootstrap.sql
DECLARE @cicdUser sysname = N'cognilens-cicd';
DECLARE @cicdSid varbinary(16) = CONVERT(varbinary(16), CAST(N'$CICD_CLIENT_ID' AS uniqueidentifier));
DECLARE @cicdCmd nvarchar(max);

IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = @cicdUser)
BEGIN
  SET @cicdCmd = N'CREATE USER [' + @cicdUser + N'] WITH SID = 0x' + CONVERT(varchar(100), @cicdSid, 2) + N', TYPE = E;';
  EXEC(@cicdCmd);
END
EXEC('ALTER ROLE db_ddladmin ADD MEMBER [' + @cicdUser + ']');
EXEC('ALTER ROLE db_datareader ADD MEMBER [' + @cicdUser + ']');
EXEC('ALTER ROLE db_datawriter ADD MEMBER [' + @cicdUser + ']');
SQL
      fi

      # -U carries the user-assigned identity's client ID (how go-sqlcmd disambiguates which MI to
      # use); -b makes a failed T-SQL batch exit non-zero, otherwise sqlcmd returns 0 on SQL errors
      # and a broken bootstrap would silently report success.
      ./sqlcmd -S "tcp:$SERVER,1433" -d "$DATABASE" -l 30 -b \
        --authentication-method=ActiveDirectoryManagedIdentity -U "$DEPLOY_CLIENT_ID" \
        -i bootstrap.sql
    "
      properties.timeout:                                                                                                                                                                                                       "PT15M"
      type:                                                                                                                                                                                                                     "Microsoft.Resources/deploymentScripts"

  ~ Microsoft.App/containerApps/cognilens-dev-api [2025-01-01]
    - properties.configuration.ingress.exposedPort: 0
    - properties.template.revisionSuffix:           "7d20d62"
    - properties.template.scale.cooldownPeriod:     300
    - properties.template.scale.pollingInterval:    30
    ~ properties.configuration.ingress.traffic: [
      ~ 0:

        - revisionName:   "cognilens-dev-api--7d20d62"
        + latestRevision: true

      ]
    ~ properties.template.containers: [
      ~ 0:

        ~ env: [
          ~ 2:

            ~ value: "InstrumentationKey=0bff4d9d-e3df-485a-92a1-b7a9a21d9a9a;IngestionEndpoint=https://eastus2-3.in.applicationinsights.azure.com/;LiveEndpoint=https://eastus2.livediagnostics.monitor.azure.com/;ApplicationId=c678843e-04f4-4447-8dcb-51c6e1abd7e6" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Insights/components/cognilens-dev-appi', '2020-02-02').ConnectionString]"

          ~ 3:

            ~ value: "58ceba81-ddd9-439a-9f3e-15ec5ff32387" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').clientId]"

          ~ 4:

            ~ value: "Server=tcp:cognilens-dev-sql-64sgu7kbbhj5e.database.windows.net,1433;Initial Catalog=CogniLens;Authentication=Active Directory Default;User Id=58ceba81-ddd9-439a-9f3e-15ec5ff32387;Encrypt=True;TrustServerCertificate=False;" => "[format('Server=tcp:{0},1433;Initial Catalog={1};Authentication=Active Directory Default;User Id={2};Encrypt=True;TrustServerCertificate=False;', 'cognilens-dev-sql-64sgu7kbbhj5e.database.windows.net', 'CogniLens', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').clientId)]"

          ~ 5:

            ~ value: "https://cognilensdevst.blob.core.windows.net/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', '2023-05-01').primaryEndpoints.blob]"

          ~ 6:

            ~ value: "https://cognilensdevst.queue.core.windows.net/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', '2023-05-01').primaryEndpoints.queue]"

          ~ 10:

            ~ value: "https://cognilens-speech-dev.cognitiveservices.azure.com/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-speech-dev', '2024-10-01').endpoint]"

          ~ 11:

            ~ value: "https://cognilens-openai-dev.openai.azure.com/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-openai-dev', '2024-10-01').endpoint]"

          ]
        ~ image: "ghcr.io/affan2900/cognilens-api:7d20d62b978dba4b078cce4c6418a5061308d531" => "ghcr.io/affan2900/cognilens-api:latest"

      ]

  ~ Microsoft.App/containerApps/cognilens-dev-worker [2025-01-01]
    - properties.configuration.ingress.exposedPort: 0
    - properties.configuration.ingress.traffic: [
        0:

          latestRevision: true
          weight:         100

      ]
    - properties.template.scale.cooldownPeriod:     300
    - properties.template.scale.pollingInterval:    30
    ~ properties.template.containers: [
      ~ 0:

        ~ env: [
          ~ 2:

            ~ value: "InstrumentationKey=0bff4d9d-e3df-485a-92a1-b7a9a21d9a9a;IngestionEndpoint=https://eastus2-3.in.applicationinsights.azure.com/;LiveEndpoint=https://eastus2.livediagnostics.monitor.azure.com/;ApplicationId=c678843e-04f4-4447-8dcb-51c6e1abd7e6" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Insights/components/cognilens-dev-appi', '2020-02-02').ConnectionString]"

          ~ 3:

            ~ value: "0d11a77c-d181-4239-ada4-c8376afe8be2" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').clientId]"

          ~ 4:

            ~ value: "Server=tcp:cognilens-dev-sql-64sgu7kbbhj5e.database.windows.net,1433;Initial Catalog=CogniLens;Authentication=Active Directory Default;User Id=0d11a77c-d181-4239-ada4-c8376afe8be2;Encrypt=True;TrustServerCertificate=False;" => "[format('Server=tcp:{0},1433;Initial Catalog={1};Authentication=Active Directory Default;User Id={2};Encrypt=True;TrustServerCertificate=False;', 'cognilens-dev-sql-64sgu7kbbhj5e.database.windows.net', 'CogniLens', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').clientId)]"

          ~ 5:

            ~ value: "https://cognilensdevst.blob.core.windows.net/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', '2023-05-01').primaryEndpoints.blob]"

          ~ 6:

            ~ value: "https://cognilensdevst.queue.core.windows.net/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', '2023-05-01').primaryEndpoints.queue]"

          ~ 10:

            ~ value: "https://cognilens-speech-dev.cognitiveservices.azure.com/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-speech-dev', '2024-10-01').endpoint]"

          ~ 11:

            ~ value: "https://cognilens-openai-dev.openai.azure.com/" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-openai-dev', '2024-10-01').endpoint]"

          ]
        ~ image: "ghcr.io/affan2900/cognilens-worker:7d20d62b978dba4b078cce4c6418a5061308d531" => "ghcr.io/affan2900/cognilens-worker:latest"

      ]
    ~ properties.template.scale.rules: [
      ~ 0:

        - azureQueue:

            accountName: "cognilensdevst"
            identity:    "/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi"
            queueLength: 1
            queueName:   "analyze-jobs"

        + custom:

            identity:             "/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi"
            metadata.accountName: "cognilensdevst"
            metadata.queueLength: "1"
            metadata.queueName:   "analyze-jobs"
            type:                 "azure-queue"


      ]

  ~ Microsoft.App/managedEnvironments/cognilens-dev-env [2025-01-01]
    - properties.peerAuthentication:

        mtls.enabled: false

    - properties.peerTrafficConfiguration:

        encryption.enabled: false

    ~ properties.appLogsConfiguration.logAnalyticsConfiguration.customerId: "e03bc1f3-bd5d-4b68-8c5a-3bea7d5d67e2" => "[reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.OperationalInsights/workspaces/cognilens-dev-law', '2023-09-01').customerId]"

  ~ Microsoft.CognitiveServices/accounts/cognilens-openai-dev/deployments/gpt-5-mini [2024-10-01]
    - properties.currentCapacity:      10
    - properties.raiPolicyName:        "Microsoft.DefaultV2"
    - properties.versionUpgradeOption: "OnceNewDefaultVersionAvailable"

  ~ Microsoft.CognitiveServices/accounts/cognilens-openai-dev/deployments/text-embedding-3-small [2024-10-01]
    - properties.currentCapacity:      10
    - properties.raiPolicyName:        "Microsoft.DefaultV2"
    - properties.versionUpgradeOption: "OnceNewDefaultVersionAvailable"

  ~ Microsoft.Consumption/budgets/cognilens-dev-budget [2023-11-01]
    ~ properties.timePeriod.endDate:   "2036-06-28T00:00:00Z" => "[dateTimeAdd(utcNow('yyyy-MM-01''T''00:00:00Z'), 'P10Y', 'yyyy-MM-dd''T''00:00:00Z')]"
    ~ properties.timePeriod.startDate: "2026-07-01T00:00:00Z" => "[utcNow('yyyy-MM-01''T''00:00:00Z')]"

  ~ Microsoft.Insights/components/cognilens-dev-appi [2020-02-02]
    + properties.Flow_Type:      "Bluefield"
    + properties.Request_Source: "rest"

  ~ Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi [2024-11-30]
    - properties:

        isolationScope: "None"


  ~ Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-sql-64sgu7kbbhj5e-deploy-mi [2024-11-30]
    - properties:

        isolationScope: "None"


  ~ Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi [2024-11-30]
    - properties:

        isolationScope: "None"


  ~ Microsoft.Search/searchServices/cognilens-search-dev [2024-06-01-preview]
    - properties.encryptionWithCmk:

        enforcement: "Unspecified"

    - properties.networkRuleSet:

        bypass: "None"

    - properties.semanticSearch: "free"

  ~ Microsoft.Storage/storageAccounts/cognilensdevst/blobServices/default [2023-05-01]
    - properties.deleteRetentionPolicy:

        allowPermanentDelete: false
        enabled:              false


  ~ Microsoft.Storage/storageAccounts/cognilensdevst/blobServices/default/containers/call-audio [2023-05-01]
    - properties.defaultEncryptionScope:      "$account-encryption-key"
    - properties.denyEncryptionScopeOverride: false

  ~ Microsoft.Storage/storageAccounts/cognilensdevst/queueServices/default [2023-05-01]
    - properties:

        logging.delete:                  false
        logging.read:                    false
        logging.retentionPolicy.enabled: false
        logging.version:                 "1.0"
        logging.write:                   false


  = Microsoft.CognitiveServices/accounts/cognilens-openai-dev [2024-10-01]
  = Microsoft.CognitiveServices/accounts/cognilens-speech-dev [2024-10-01]
  = Microsoft.Insights/actionGroups/cognilens-dev-budget-ag [2023-01-01]
  = Microsoft.KeyVault/vaults/cognilens-dev-kv [2023-07-01]
  = Microsoft.OperationalInsights/workspaces/cognilens-dev-law [2023-09-01]
  = Microsoft.Sql/servers/cognilens-dev-sql-64sgu7kbbhj5e [2023-08-01-preview]
  = Microsoft.Sql/servers/cognilens-dev-sql-64sgu7kbbhj5e/databases/CogniLens [2023-08-01-preview]
    x properties.minCapacity: 0.5
    x sku.capacity:           1
    x sku.tier:               "GeneralPurpose"

  = Microsoft.Sql/servers/cognilens-dev-sql-64sgu7kbbhj5e/firewallRules/AllowAllWindowsAzureIps [2023-08-01-preview]
  = Microsoft.Storage/storageAccounts/cognilensdevst [2023-05-01]
  = Microsoft.Storage/storageAccounts/cognilensdevst/queueServices/default/queues/analyze-jobs [2023-05-01]
  = Microsoft.Storage/storageAccounts/cognilensdevst/queueServices/default/queues/analyze-jobs-poison [2023-05-01]
  * Microsoft.Sql/servers/cognilens-dev-sql-64sgu7kbbhj5e/databases/master
  * microsoft.insights/actiongroups/Application Insights Smart Detection

Resource changes: 1 to create, 14 to modify, 11 no change, 13 unsupported, 2 to ignore.

Diagnostics (13): 
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').principalId, 'ba92f5b4-2d11-453d-a403-e96b0029c9fe'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[6]' on line 1 and column 10940 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, 'ba92f5b4-2d11-453d-a403-e96b0029c9fe'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[6]' on line 1 and column 10940 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').principalId, 'db58b8e5-c6ad-4a2a-8342-4190687cbf4a'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[7]' on line 1 and column 11732 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, 'db58b8e5-c6ad-4a2a-8342-4190687cbf4a'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[7]' on line 1 and column 11732 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').principalId, '974c5e8b-45b9-4653-ba55-5f855dd0fb88'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[8]' on line 1 and column 12506 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Storage/storageAccounts/cognilensdevst', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, '974c5e8b-45b9-4653-ba55-5f855dd0fb88'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[1].properties.template.resources[8]' on line 1 and column 12506 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.KeyVault/vaults/cognilens-dev-kv', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.KeyVault/vaults/cognilens-dev-kv', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').principalId, '4633458b-17de-408a-b874-0445c86b69e6'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[3].properties.template.resources[1]' on line 1 and column 18227 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.KeyVault/vaults/cognilens-dev-kv', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.KeyVault/vaults/cognilens-dev-kv', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, '4633458b-17de-408a-b874-0445c86b69e6'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[3].properties.template.resources[1]' on line 1 and column 18227 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-speech-dev', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-speech-dev', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, 'f2dc8367-1007-4938-bd23-fe263f013447'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[4].properties.template.resources[4]' on line 1 and column 22972 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-openai-dev', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.CognitiveServices/accounts/cognilens-openai-dev', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[4].properties.template.resources[5]' on line 1 and column 23666 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, '7ca78c08-252a-4471-8644-bb5ff32d4ba0'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[5].properties.template.resources[1]' on line 1 and column 27141 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-worker-mi', '2024-11-30').principalId, '8ebe5a00-799e-43f5-93ac-243d3dce84a7'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[5].properties.template.resources[2]' on line 1 and column 27814 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.
[extensionResourceId('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', 'Microsoft.Authorization/roleAssignments', guid('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.Search/searchServices/cognilens-search-dev', reference('/subscriptions/163d0954-8f1c-442e-88f9-e11864c54b0f/resourceGroups/rg-cognilens-dev/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cognilens-dev-api-mi', '2024-11-30').principalId, '1407120a-92aa-4202-b7e9-c0e197c71c8f'))] (Unsupported) Changes to the resource declared at 'properties.template.resources[5].properties.template.resources[3]' on line 1 and column 28491 cannot be analyzed because its resource ID or API version cannot be calculated until the deployment is under way. Please see https://aka.ms/WhatIfUnidentifiableResource for more guidance.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/loadshed-test.ps1`:
- Around line 98-105: Dispose the per-iteration PUT request, its
ByteArrayContent, and response in the upload flow around $put and $putResponse,
using try/finally so cleanup occurs on success and failure; ensure response
content is consumed or disposed before releasing the response. Apply the same
disposal pattern to the $analyze/$analyzeResponse POST flow, while preserving
the existing status validation and error behavior.
- Around line 184-208: Update the verdict handling after the $verdict
initialization so PASS remains 0, definite job-loss failures from $stuck remain
1, and scaling-evidence failures from $replicaReadFailures use a distinct
inconclusive code such as 2. Convert the independent outcome checks into a
priority-ordered elseif chain, with the $stuck failure evaluated before the
replica-read inconclusive result, so only one outcome message is printed before
the existing PASS handling.
- Around line 150-158: Wrap the az revision-list invocation in a try/catch
within the replica polling logic so terminating invocation errors, including
executable-not-found, increment replicaReadFailures and emit the existing
warning instead of aborting the loop. Preserve the current success validation
and non-zero LASTEXITCODE fallback, ensuring both failure paths retain
INCONCLUSIVE scaling reporting.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: acef42a7-8571-4ded-b0ce-758990b57abc

📥 Commits

Reviewing files that changed from the base of the PR and between 7d20d62 and 2981e56.

📒 Files selected for processing (2)
  • scripts/loadshed-test.ps1
  • src/CogniLens.Api/Program.cs
💤 Files with no reviewable changes (1)
  • src/CogniLens.Api/Program.cs

Comment thread scripts/loadshed-test.ps1
Comment on lines +98 to +105
$put = New-Object System.Net.Http.HttpRequestMessage([System.Net.Http.HttpMethod]::Put, $create.uploadUrl)
$put.Headers.Add('x-ms-blob-type', 'BlockBlob')
$put.Content = New-Object System.Net.Http.ByteArrayContent(, $audioBytes)
$put.Content.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse('application/octet-stream')
$putResponse = $http.SendAsync($put).GetAwaiter().GetResult()
if (-not $putResponse.IsSuccessStatusCode) {
throw "Upload for call $($create.callId) failed: $([int]$putResponse.StatusCode) $($putResponse.Content.ReadAsStringAsync().GetAwaiter().GetResult())"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Dispose HttpRequestMessage/HttpResponseMessage/content objects created inside the loop.

Every iteration of the for loop (up to $Count, default 50, adjustable via -Count) creates a new HttpRequestMessage, ByteArrayContent, and HttpResponseMessage for the blob PUT (Lines 98-105) and another pair for the analyze POST (Lines 118-135), none of which are disposed. Successful responses never even read/dispose Content, which can keep the underlying connection from being released back to the pool until GC finalizes it — undermining the point of switching to a persistent HttpClient for connection reuse, and adding needless memory/socket pressure at higher $Count.

🔧 Proposed fix (wrap each request/response pair in try/finally)
-    $put = New-Object System.Net.Http.HttpRequestMessage([System.Net.Http.HttpMethod]::Put, $create.uploadUrl)
-    $put.Headers.Add('x-ms-blob-type', 'BlockBlob')
-    $put.Content = New-Object System.Net.Http.ByteArrayContent(, $audioBytes)
-    $put.Content.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse('application/octet-stream')
-    $putResponse = $http.SendAsync($put).GetAwaiter().GetResult()
-    if (-not $putResponse.IsSuccessStatusCode) {
-        throw "Upload for call $($create.callId) failed: $([int]$putResponse.StatusCode) $($putResponse.Content.ReadAsStringAsync().GetAwaiter().GetResult())"
-    }
+    $put = New-Object System.Net.Http.HttpRequestMessage([System.Net.Http.HttpMethod]::Put, $create.uploadUrl)
+    try {
+        $put.Headers.Add('x-ms-blob-type', 'BlockBlob')
+        $put.Content = New-Object System.Net.Http.ByteArrayContent(, $audioBytes)
+        $put.Content.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse('application/octet-stream')
+        $putResponse = $http.SendAsync($put).GetAwaiter().GetResult()
+        try {
+            if (-not $putResponse.IsSuccessStatusCode) {
+                throw "Upload for call $($create.callId) failed: $([int]$putResponse.StatusCode) $($putResponse.Content.ReadAsStringAsync().GetAwaiter().GetResult())"
+            }
+        } finally { $putResponse.Dispose() }
+    } finally { $put.Dispose() }

Apply the same pattern to $analyze/$analyzeResponse at Lines 118-135.

Also applies to: 118-135

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/loadshed-test.ps1` around lines 98 - 105, Dispose the per-iteration
PUT request, its ByteArrayContent, and response in the upload flow around $put
and $putResponse, using try/finally so cleanup occurs on success and failure;
ensure response content is consumed or disposed before releasing the response.
Apply the same disposal pattern to the $analyze/$analyzeResponse POST flow,
while preserving the existing status validation and error behavior.

Comment thread scripts/loadshed-test.ps1
Comment on lines +150 to 158
$replicas = az containerapp revision list --name $WorkerApp --resource-group $ResourceGroup `
--query "[?properties.active].properties.replicas | [0]" -o tsv
if ($LASTEXITCODE -eq 0 -and "$replicas".Trim() -match '^\d+$') {
if ([int]$replicas -gt $maxReplicas) { $maxReplicas = [int]$replicas }
}
else {
$replicaReadFailures++
Write-Warning "Could not read replica count (az exit $LASTEXITCODE). Peak-replica figure will be unreliable."
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "== files =="
git ls-files | rg '(^|/)loadshed-test\.ps1$|scripts/' || true

echo "== target excerpt =="
sed -n '1,220p' scripts/loadshed-test.ps1 2>/dev/null || true

echo "== error action / az call usages in target =="
rg -n "ErrorActionPreference|az [^(]|replicaReadFailures|maxReplicas|Write-Warning|trap|Try|Catch" scripts/loadshed-test.ps1 2>/dev/null || true

Repository: Affan2900/cognilens

Length of output: 10648


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "== PowerShell availability =="
command -v pwsh || true
command -v powershell || true

if command -v pwsh >/dev/null 2>&1; then
  pwsh -NoProfile -Command '$PSVersionTable.PSVersion; Write-Host \"$ErrorActionPreference=$ErrorActionPreference\"; trap {} az containerapp revision list --name bad --resource-group rg --query \"[?properties.active].properties.replicas | [0]\" -o tsv 2>&1; Write-Host \"LASTEXITCODE=$LASTEXITCODE\"; Write-Host \"after\"'
elif command -v powershell >/dev/null 2>&1; then
  powershell -NoProfile -Command '$PSVersionTable.PSVersion; Write-Host \"$ErrorActionPreference=$ErrorActionPreference\"; try { az containerapp revision list --name bad --resource-group rg --query "[?properties.active].properties.replicas | [0]" -o tsv 2>&1 } catch { Write-Host \"caught: $_\" }; Write-Host \"LASTEXITCODE=$LASTEXITCODE\"; Write-Host \"after\"'
else
  echo "No PowerShell runtime available"
fi

Repository: Affan2900/cognilens

Length of output: 592


🌐 Web query:

PowerShell invoke external command az throwing executable not found $ErrorActionPreference Stop LASTEXITCODE documentation

💡 Result:

In PowerShell, an "executable not found" error when calling an external program (like az) is a critical, terminating exception that prevents the command from running [1][2]. This differs from a command that runs but returns a non-zero exit code [3][4]. Handling Executable-Not-Found Errors If PowerShell cannot find the executable (e.g., az), it throws a CommandNotFoundException [2]. Because this occurs before the external program is even invoked, it is a terminating error that obeys the current scope's $ErrorActionPreference [3]. If $ErrorActionPreference is set to 'Stop', the entire script execution will halt immediately upon this error [3]. You can catch this using a try/catch block [1]: try { az --version } catch [System.Management.Automation.CommandNotFoundException] { Write-Error "The Azure CLI executable was not found. Please ensure it is installed and in your PATH." } Distinguishing from Exit Codes External programs do not inherently participate in PowerShell's error handling system [3]. They communicate success or failure via the $LASTEXITCODE automatic variable [3][4]. A non-zero $LASTEXITCODE simply indicates the command ran and failed; it does not trigger a PowerShell exception by default [3][4]. To treat non-zero exit codes as terminating errors that obey $ErrorActionPreference, you can use the $PSNativeCommandUseErrorActionPreference feature (available in PowerShell 7.4+) [4][5]: $PSNativeCommandUseErrorActionPreference = $true $ErrorActionPreference = 'Stop' az account show # Now throws a NativeCommandExitException if exit code is non-zero If you are using an older version of PowerShell or need manual control, the standard approach is to check $LASTEXITCODE immediately after execution [3][4][1]: az account show if ($LASTEXITCODE -ne 0) { throw "Azure CLI command failed with exit code $LASTEXITCODE" } Troubleshooting "Executable Not Found" If you encounter this error frequently, ensure the following: 1. PATH Environment Variable: Verify the directory containing the az executable is included in your system's $env:PATH [6][7]. 2. Execution Policy/Permissions: On some systems (like Linux agents), verify that the file itself is marked as executable [8][9]. 3. Session Environment: If you recently installed the tool, you may need to start a new PowerShell session for the updated PATH environment variable to take effect [6].

Citations:


Wrap the az call so replica-read failures don’t abort the polling loop.

With $ErrorActionPreference = 'Stop', az invocation failures such as executable-not-found throw before a non-zero $LASTEXITCODE is set, bypassing this fallback and losing the script’s graceful INCONCLUSIVE scaling reporting.

🔧 Proposed fix
-    $replicas = az containerapp revision list --name $WorkerApp --resource-group $ResourceGroup `
-        --query "[?properties.active].properties.replicas | [0]" -o tsv
-    if ($LASTEXITCODE -eq 0 -and "$replicas".Trim() -match '^\d+$') {
-        if ([int]$replicas -gt $maxReplicas) { $maxReplicas = [int]$replicas }
-    }
-    else {
-        $replicaReadFailures++
-        Write-Warning "Could not read replica count (az exit $LASTEXITCODE). Peak-replica figure will be unreliable."
-    }
+    try {
+        $replicas = az containerapp revision list --name $WorkerApp --resource-group $ResourceGroup `
+            --query "[?properties.active].properties.replicas | [0]" -o tsv
+        if ($LASTEXITCODE -eq 0 -and "$replicas".Trim() -match '^\d+$') {
+            if ([int]$replicas -gt $maxReplicas) { $maxReplicas = [int]$replicas }
+        }
+        else {
+            $replicaReadFailures++
+            Write-Warning "Could not read replica count (az exit $LASTEXITCODE). Peak-replica figure will be unreliable."
+        }
+    }
+    catch {
+        $replicaReadFailures++
+        Write-Warning "Could not read replica count ($_). Peak-replica figure will be unreliable."
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
$replicas = az containerapp revision list --name $WorkerApp --resource-group $ResourceGroup `
--query "[?properties.active].properties.replicas | [0]" -o tsv
if ($LASTEXITCODE -eq 0 -and "$replicas".Trim() -match '^\d+$') {
if ([int]$replicas -gt $maxReplicas) { $maxReplicas = [int]$replicas }
}
else {
$replicaReadFailures++
Write-Warning "Could not read replica count (az exit $LASTEXITCODE). Peak-replica figure will be unreliable."
}
try {
$replicas = az containerapp revision list --name $WorkerApp --resource-group $ResourceGroup `
--query "[?properties.active].properties.replicas | [0]" -o tsv
if ($LASTEXITCODE -eq 0 -and "$replicas".Trim() -match '^\d+$') {
if ([int]$replicas -gt $maxReplicas) { $maxReplicas = [int]$replicas }
}
else {
$replicaReadFailures++
Write-Warning "Could not read replica count (az exit $LASTEXITCODE). Peak-replica figure will be unreliable."
}
}
catch {
$replicaReadFailures++
Write-Warning "Could not read replica count ($_). Peak-replica figure will be unreliable."
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/loadshed-test.ps1` around lines 150 - 158, Wrap the az revision-list
invocation in a try/catch within the replica polling logic so terminating
invocation errors, including executable-not-found, increment replicaReadFailures
and emit the existing warning instead of aborting the loop. Preserve the current
success validation and non-zero LASTEXITCODE fallback, ensuring both failure
paths retain INCONCLUSIVE scaling reporting.

Comment thread scripts/loadshed-test.ps1
Comment on lines +184 to +208
$verdict = 0

if ($stuck -gt 0) {
Write-Host "`nFAIL: $stuck job(s) never reached a terminal state — check the poison queue." -ForegroundColor Red
$jobs | Where-Object { $null -eq $_.Outcome } | ForEach-Object { Write-Host " $($_.CallId)" }
exit 1
$verdict = 1
}
if ($maxReplicas -le 1) {

if ($replicaReadFailures -gt 0) {
# Not a pass and not a fail: the scaling claim has no evidence behind it either way. Saying
# so beats reporting the initial 0 as though it were an observation.
Write-Host "`nINCONCLUSIVE on scaling: $replicaReadFailures replica read(s) failed, so the peak-replica figure is not trustworthy." -ForegroundColor Yellow
Write-Host " Check the platform's own record instead: ContainerAppSystemLogs_CL, Reason_s == 'KEDAScaleTargetActivated'." -ForegroundColor Yellow
$verdict = 1
}
elseif ($maxReplicas -le 1 -and $jobs.Count -gt 1) {
Write-Host "`nFAIL: worker never scaled past $maxReplicas replica(s) — KEDA rule did not fire." -ForegroundColor Red
exit 1
$verdict = 1
}

if ($verdict -eq 0) {
Write-Host "`nPASS: nothing lost ($completed completed, $failed failed), peak $maxReplicas worker replica(s)." -ForegroundColor Green
}

Write-Host "`nPASS: nothing lost, worker scaled to $maxReplicas replicas." -ForegroundColor Green
exit $verdict

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

FAIL and INCONCLUSIVE collapse to the same exit code, and their messages can both print together.

The whole point of this PR is fixing script reporting, but $verdict only ever becomes 0 or 1 — a definite job-loss FAIL (Line 189) and a scaling-evidence-unreliable INCONCLUSIVE (Line 197) are indistinguishable to any caller (CI, automation) that gates on exit code, even though the script explicitly wants three distinct outcomes. Additionally, since both blocks execute independently, a run with $stuck -gt 0 and $replicaReadFailures -gt 0 prints both the red FAIL banner and the yellow INCONCLUSIVE banner, which muddies what actually happened.

Consider distinct exit codes (e.g., 0=PASS, 1=FAIL, 2=INCONCLUSIVE) and an elseif chain so only the highest-priority outcome is reported.

🔧 Proposed fix
-$verdict = 0
+$verdict = 0 # 0=PASS 1=FAIL 2=INCONCLUSIVE

 if ($stuck -gt 0) {
     Write-Host "`nFAIL: $stuck job(s) never reached a terminal state — check the poison queue." -ForegroundColor Red
     $jobs | Where-Object { $null -eq $_.Outcome } | ForEach-Object { Write-Host "  $($_.CallId)" }
     $verdict = 1
 }

-if ($replicaReadFailures -gt 0) {
+elseif ($replicaReadFailures -gt 0) {
     # Not a pass and not a fail: the scaling claim has no evidence behind it either way. Saying
     # so beats reporting the initial 0 as though it were an observation.
     Write-Host "`nINCONCLUSIVE on scaling: $replicaReadFailures replica read(s) failed, so the peak-replica figure is not trustworthy." -ForegroundColor Yellow
     Write-Host "  Check the platform's own record instead: ContainerAppSystemLogs_CL, Reason_s == 'KEDAScaleTargetActivated'." -ForegroundColor Yellow
-    $verdict = 1
+    $verdict = 2
 }
 elseif ($maxReplicas -le 1 -and $jobs.Count -gt 1) {
     Write-Host "`nFAIL: worker never scaled past $maxReplicas replica(s) — KEDA rule did not fire." -ForegroundColor Red
     $verdict = 1
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
$verdict = 0
if ($stuck -gt 0) {
Write-Host "`nFAIL: $stuck job(s) never reached a terminal state — check the poison queue." -ForegroundColor Red
$jobs | Where-Object { $null -eq $_.Outcome } | ForEach-Object { Write-Host " $($_.CallId)" }
exit 1
$verdict = 1
}
if ($maxReplicas -le 1) {
if ($replicaReadFailures -gt 0) {
# Not a pass and not a fail: the scaling claim has no evidence behind it either way. Saying
# so beats reporting the initial 0 as though it were an observation.
Write-Host "`nINCONCLUSIVE on scaling: $replicaReadFailures replica read(s) failed, so the peak-replica figure is not trustworthy." -ForegroundColor Yellow
Write-Host " Check the platform's own record instead: ContainerAppSystemLogs_CL, Reason_s == 'KEDAScaleTargetActivated'." -ForegroundColor Yellow
$verdict = 1
}
elseif ($maxReplicas -le 1 -and $jobs.Count -gt 1) {
Write-Host "`nFAIL: worker never scaled past $maxReplicas replica(s) — KEDA rule did not fire." -ForegroundColor Red
exit 1
$verdict = 1
}
if ($verdict -eq 0) {
Write-Host "`nPASS: nothing lost ($completed completed, $failed failed), peak $maxReplicas worker replica(s)." -ForegroundColor Green
}
Write-Host "`nPASS: nothing lost, worker scaled to $maxReplicas replicas." -ForegroundColor Green
exit $verdict
$verdict = 0 # 0=PASS 1=FAIL 2=INCONCLUSIVE
if ($stuck -gt 0) {
Write-Host "`nFAIL: $stuck job(s) never reached a terminal state — check the poison queue." -ForegroundColor Red
$jobs | Where-Object { $null -eq $_.Outcome } | ForEach-Object { Write-Host " $($_.CallId)" }
$verdict = 1
}
elseif ($replicaReadFailures -gt 0) {
# Not a pass and not a fail: the scaling claim has no evidence behind it either way. Saying
# so beats reporting the initial 0 as though it were an observation.
Write-Host "`nINCONCLUSIVE on scaling: $replicaReadFailures replica read(s) failed, so the peak-replica figure is not trustworthy." -ForegroundColor Yellow
Write-Host " Check the platform's own record instead: ContainerAppSystemLogs_CL, Reason_s == 'KEDAScaleTargetActivated'." -ForegroundColor Yellow
$verdict = 2
}
elseif ($maxReplicas -le 1 -and $jobs.Count -gt 1) {
Write-Host "`nFAIL: worker never scaled past $maxReplicas replica(s) — KEDA rule did not fire." -ForegroundColor Red
$verdict = 1
}
if ($verdict -eq 0) {
Write-Host "`nPASS: nothing lost ($completed completed, $failed failed), peak $maxReplicas worker replica(s)." -ForegroundColor Green
}
exit $verdict
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/loadshed-test.ps1` around lines 184 - 208, Update the verdict
handling after the $verdict initialization so PASS remains 0, definite job-loss
failures from $stuck remain 1, and scaling-evidence failures from
$replicaReadFailures use a distinct inconclusive code such as 2. Convert the
independent outcome checks into a priority-ordered elseif chain, with the $stuck
failure evaluated before the replica-read inconclusive result, so only one
outcome message is printed before the existing PASS handling.

@Affan2900
Affan2900 merged commit c0e4745 into main Jul 26, 2026
6 checks passed
Affan2900 added a commit that referenced this pull request Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant